flux lora collection

Introduction

The FLUX LoRA Collection provides a set of trained LoRA models for the FLUX.1-dev model developed by Black Forest Labs. These models are designed for text-to-image generation using the Stable Diffusion pipeline and include various artistic styles such as Furry, MJV6, Anime, Disney, Scenery, and Art.

Architecture

The architecture utilizes LoRA (Low-Rank Adaptation) with the FLUX.1-dev model, focusing on image generation. This setup allows for flexible and creative outputs based on textual prompts, leveraging the Stable Diffusion technique.

Training

Training scripts and configurations for fine-tuning the Flux model, including LoRA and ControlNet, are available on the XLabs AI GitHub repository. The training dataset is structured with images paired with JSON files containing text prompts. Contributions for datasets have been acknowledged from users on Civitai.

Guide: Running Locally

To run the model locally, follow these steps:

1. Ensure Environment Setup: Install the necessary dependencies, including Python and required libraries.
2. Clone Repository: Access the scripts and configurations from the XLabs AI GitHub repository.
3. Prepare Dataset: Organize your dataset with images and corresponding JSON files containing prompts.
4. Run Inference: Use the provided Python scripts to generate images with various prompts. For example:

   python3 main.py \
   --prompt "An aerial view of beach with people on it, disney style" \
   --lora_repo_id XLabs-AI/flux-lora-collection --lora_name disney_lora.safetensors \
   --device cuda --offload --use_lora --model_type flux-dev-fp8 --width 1024 --height 1024
   

5. Utilize Cloud GPUs: For optimal performance, consider using cloud GPU services such as AWS, Google Cloud, or Azure.

License

The LoRA models fall under the FLUX.1-dev Non-Commercial License, as provided by Black Forest Labs. The full license details are available on the Hugging Face model page.