Flux Xmas Illustration Lo R A

Introduction

The Flux-Xmas-Illustration-LoRA model enables the creation of festive Christmas-themed illustrations. It leverages the LoRA (Low-Rank Adaptation) technique to enhance image generation capabilities in a text-to-image pipeline.

Architecture

The model is built using the diffusion process, specifically designed for generating high-quality images from textual prompts. It utilizes a base model, "black-forest-labs/FLUX.1-dev," and incorporates LoRA weights for improved performance.

Training

• Parameters:

  • Optimizer: AdamW
  • LR Scheduler: Constant
  • Network Dimension: 64
  • Network Alpha: 32
  • Epochs: 20
  • Noise Offset: 0.03
  • Multires Noise Discount: 0.1
  • Multires Noise Iterations: 10

• Dataset: The model was trained using 21 images with a resolution of 2K, labeled with natural language descriptions.

Guide: Running Locally

To run the Flux-Xmas-Illustration-LoRA model locally, follow these steps:

1. Install Dependencies: Ensure you have Python installed along with the necessary libraries such as torch and pipelines.

2. Setup:

   import torch
   from pipelines import DiffusionPipeline
   
   base_model = "black-forest-labs/FLUX.1-dev"
   pipe = DiffusionPipeline.from_pretrained(base_model, torch_dtype=torch.bfloat16)
   
   lora_repo = "strangerzonehf/Flux-Xmas-Illustration-LoRA"
   trigger_word = "Xmas Illustration"  
   pipe.load_lora_weights(lora_repo)
   
   device = torch.device("cuda")
   pipe.to(device)
   

3. Inference: Use the trigger word "Xmas Illustration" to generate images. The recommended inference steps range from 30 to 35.

4. Resources: For optimal performance, consider using cloud GPUs such as AWS EC2 with Nvidia GPUs or Google Cloud Platform's AI Platform.

License

This model is licensed under the Apache 2.0 License, which allows for both personal and commercial use, modification, and distribution, provided that proper attribution is given to the original creators.