Flux Sketch Smudge Lo R A

Introduction

The Flux-Sketch-Smudge-LoRA is a model designed for text-to-image generation using the LoRA (Low-Rank Adaptation) approach, integrated with the Hugging Face diffusion pipelines. It specializes in creating sketch-like images based on descriptive prompts.

Architecture

The model leverages a base model, "black-forest-labs/FLUX.1-dev," and is fine-tuned using LoRA techniques. It utilizes a specific set of image processing parameters, including:

• LR Scheduler: Constant
• Optimizer: AdamW
• Network Dimensions: 64
• Network Alpha: 32

Training

The model was trained using 26 images with parameters such as a learning rate scheduler set to "constant" and an optimizer "AdamW." The noise offset is set to 0.03, and the multires noise discount is 0.1. The training involved 18 epochs, with model weights saved at each epoch.

Guide: Running Locally

1. Install Dependencies: Ensure you have PyTorch and Hugging Face's Transformers library installed.

   pip install torch transformers
   

2. Setup Model: Use the following code to set up and load the model.

   import torch
   from pipelines import DiffusionPipeline
   
   base_model = "black-forest-labs/FLUX.1-dev"
   pipe = DiffusionPipeline.from_pretrained(base_model, torch_dtype=torch.bfloat16)
   
   lora_repo = "strangerzonehf/Flux-Sketch-Smudge-LoRA"
   trigger_word = "Sketch Smudge"  
   pipe.load_lora_weights(lora_repo)
   
   device = torch.device("cuda")
   pipe.to(device)
   

3. Trigger Words: Use "Sketch Smudge" as a trigger for image generation.

4. Suggested Cloud GPUs: Consider using cloud services like AWS, Google Cloud, or Azure for access to powerful GPUs required for efficient processing.

License

The model is licensed under the CreativeML OpenRAIL-M, allowing for certain uses with specific restrictions and obligations.