Flux Midjourney Mix2 Lo R A

Introduction

The Flux-Midjourney-Mix2-LoRA model, developed by StrangerZoneHF, is a text-to-image diffusion model designed for generating high-quality images from textual descriptions. It utilizes the LoRA (Low-Rank Adaptation) technique to fine-tune the base model for improved realism and detail in various styles, such as portrait, street, and fashion photography.

Architecture

The model is based on the black-forest-labs/FLUX.1-dev architecture, integrating LoRA for enhanced image generation. Key parameters include:

• Learning Rate Scheduler: Constant
• Optimizer: AdamW8
• Network Dimensions: Dim 64, Alpha 32
• Noise Offset: 0.03
• Multires Noise Discount: 0.1

Training

The model was trained using 36 images with a focus on achieving realism and high-quality close-up shots. The training process involved 3660 steps per epoch, repeated over 28 epochs, saving the model every epoch. The label set used includes florence2-en for natural language processing in English.

Guide: Running Locally

Basic Steps

1. Install Required Libraries: Ensure you have torch and DiffusionPipeline installed.
2. Load Model:

   import torch
   from pipelines import DiffusionPipeline
   
   base_model = "black-forest-labs/FLUX.1-dev"
   pipe = DiffusionPipeline.from_pretrained(base_model, torch_dtype=torch.bfloat16)
   
   lora_repo = "strangerzonehf/Flux-Midjourney-Mix2-LoRA"
   pipe.load_lora_weights(lora_repo)
   
   device = torch.device("cuda")
   pipe.to(device)
   

3. Generate Images: Use the trigger word "MJ v6" with prompts to generate images.
4. Download Weights: Access the model weights in Safetensors format from the Files & versions tab.

Suggested Cloud GPUs

• NVIDIA A100
• NVIDIA V100
• NVIDIA T4

License

The model is licensed under the CreativeML Open RAIL-M license, which allows for various uses with some restrictions. Ensure compliance with the license terms when using the model.