Flux Sketch Ep Lo R A

Introduction

Flux-Sketch-Ep-LoRA is a text-to-image model leveraging LoRA and diffusion technology to generate high-quality sketches based on textual prompts. It is designed to produce images in various artistic styles, focusing on simplicity and vibrancy.

Architecture

The model is based on the "black-forest-labs/FLUX.1-dev" architecture, utilizing LoRA for efficient fine-tuning. It uses the AdamW optimizer and a constant learning rate scheduler. The network dimensions are set to 64, with a network alpha of 32. The model parameters include settings for multiresolution noise and repeat steps to ensure detailed and high-quality image outputs.

Training

The model was trained using 22 images, with a focus on achieving a balance between realistic and cartoon-like styles. The training process involved 20 epochs with specific settings for learning rate and noise offset to optimize performance. Labeling was done using natural language in English.

Guide: Running Locally

To run the Flux-Sketch-Ep-LoRA model locally, follow the steps below:

1. Set Up Environment: Make sure you have Python and PyTorch installed. Use a machine with CUDA support for optimal performance.
2. Install Required Packages: Install necessary packages such as torch and pipelines.
3. Load Model:

   import torch
   from pipelines import DiffusionPipeline
   
   base_model = "black-forest-labs/FLUX.1-dev"
   pipe = DiffusionPipeline.from_pretrained(base_model, torch_dtype=torch.bfloat16)
   
   lora_repo = "strangerzonehf/Flux-Sketch-Ep-LoRA"
   trigger_word = "ep sketch"
   pipe.load_lora_weights(lora_repo)
   
   device = torch.device("cuda")
   pipe.to(device)
   

4. Generate Images: Use the trigger word "ep sketch" in your prompts to generate images.

Cloud GPUs

Consider using cloud-based GPU services like AWS, Google Cloud, or Azure to run the model efficiently if local resources are limited.

License

The model is licensed under the CreativeML OpenRAIL-M license, permitting broad usage with certain restrictions. Ensure compliance with the license terms when deploying the model.