Knitted Character Flux Lo R A

Introduction

The Knitted-Character-Flux-LoRA model by prithivMLmods is a text-to-image model, particularly designed for generating images of knitted or crocheted characters. It utilizes LoRA (Low-Rank Adaptation) within the Diffusers library framework, focusing on themes like "Knitted Character" and "Thread".

Architecture

The model is based on the "black-forest-labs/FLUX.1-dev" as the base model and employs LoRA for efficient fine-tuning. Key image processing parameters include:

• LR Scheduler: constant
• Optimizer: AdamW
• Network Dim: 64
• Network Alpha: 32

Training

The model is currently in the training phase, which involves using 30 high-resolution (4K) images to enhance its capabilities. Training specifics include:

• Noise Offset: 0.03
• Multires Noise Discount: 0.1
• Multires Noise Iterations: 10
• Repeat & Steps: 20 & 2000
• Epochs: 10

Guide: Running Locally

To run the model locally, follow these steps:

1. Install Requirements: Ensure you have the necessary libraries installed, including PyTorch and the Diffusers library.

2. Setup: Use the following code snippet to set up the model:

   import torch
   from pipelines import DiffusionPipeline
   
   base_model = "black-forest-labs/FLUX.1-dev"
   pipe = DiffusionPipeline.from_pretrained(base_model, torch_dtype=torch.bfloat16)
   
   lora_repo = "prithivMLmods/Knitted-Character-Flux-LoRA"
   trigger_word = "Knitted Character"  
   pipe.load_lora_weights(lora_repo)
   
   device = torch.device("cuda")
   pipe.to(device)
   

3. Trigger Word: Use "Knitted Character" to initiate image generation.

4. Cloud GPUs: For optimal performance, consider using cloud-based GPU services like AWS, Google Cloud, or Azure.

License

The model is licensed under the Apache-2.0 license, allowing for flexible use and distribution.