Flux Midjourney Mix Lo R A

Introduction

The Flux-Midjourney-Mix-LoRA model is a text-to-image model hosted on Hugging Face. It leverages LoRA (Low-Rank Adaptation) technology to enhance image generation capabilities, integrating styles like realism, photo-realism, and high-fidelity art.

Architecture

The model utilizes the FLUX.1-Dev base model with LoRA weights to augment its performance. It operates using parameters such as a constant learning rate scheduler, AdamW optimizer, and supports image resolutions best at 768 x 1024 or default at 1024 x 1024.

Training

The model is trained using 60+ high-resolution images. Key parameters include:

• LR Scheduler: constant
• Optimizer: AdamW
• Noise Offset: 0.03
• Network Dimensions: 64
• Network Alpha: 32
• Epochs: 22

Training involves various noise and repetition strategies to optimize the model's image generation capabilities.

Guide: Running Locally

1. Setup Environment: Ensure you have Python and PyTorch installed.
2. Install Necessary Libraries:

   pip install torch torchvision
   

3. Download Model:
   • Access and download the model weights from the Files & versions tab.
4. Run the Model:

   import torch
   from pipelines import DiffusionPipeline
   
   base_model = "black-forest-labs/FLUX.1-dev"
   pipe = DiffusionPipeline.from_pretrained(base_model, torch_dtype=torch.bfloat16)
   
   lora_repo = "strangerzonehf/Flux-Midjourney-Mix-LoRA"
   pipe.load_lora_weights(lora_repo)
   
   device = torch.device("cuda")
   pipe.to(device)
   

5. Generate Images: Use the trigger word midjourney mix to produce images.
6. Recommended Hardware: Utilize cloud GPUs such as those provided by AWS, GCP, or Azure for optimal performance.

License

The model is distributed under the CreativeML OpenRAIL-M license, allowing for both personal and commercial use with specific conditions.