Flux Sketch Sized Lo R A

Introduction

Flux-Sketch-Sized-LoRA is a model designed for generating text-to-image outputs using the LoRA (Low-Rank Adaptation) method. It utilizes the FLUX.1-dev base model from Black Forest Labs, optimized for creative and illustrative tasks. This model is particularly suitable for creating stylized images with a sketch-like quality.

Architecture

The model is built on the FLUX.1-dev architecture, with LoRA finetuning to allow efficient adaptation. Key parameters include:

• Optimizer: AdamW
• Learning Rate Scheduler: Constant
• Network Dimensions: 64
• Network Alpha: 32

Training

The model was trained using 30 images in 14-bit raw format. Training parameters were as follows:

• Epochs: 20, with saving every epoch
• Repeat and Steps: 27 repeats and 3490 steps
• Noise Parameters: Noise Offset of 0.03, Multires Noise Discount of 0.1, and 10 Multires Noise Iterations

Guide: Running Locally

1. Setup Environment:

   • Ensure you have torch and Diffusers library installed.
   • Install additional dependencies as required by the pipeline.

2. Initialize the Model:

   import torch
   from pipelines import DiffusionPipeline
   
   base_model = "black-forest-labs/FLUX.1-dev"
   pipe = DiffusionPipeline.from_pretrained(base_model, torch_dtype=torch.bfloat16)
   

3. Load LoRA Weights:

   lora_repo = "strangerzonehf/Flux-Sketch-Sized-LoRA"
   pipe.load_lora_weights(lora_repo)
   

4. Deploy on GPU:

   device = torch.device("cuda")
   pipe.to(device)
   

5. Generate Images: Use the trigger word "Sketch Sized" to initiate image generation.

Note: For optimal performance, using cloud GPUs such as AWS EC2 with NVIDIA GPUs or Google Cloud Platform is recommended.

License

The model is distributed under the CreativeML OpenRAIL-M license, allowing for open use with certain restrictions on use cases, such as prohibiting military applications.