Libre F L U X

Introduction

LibreFLUX is an open-source, de-distilled version of the FLUX.1-schnell model, licensed under Apache 2.0. It offers a full T5 context length, uses attention masking, and restores classifier free guidance. The model is designed to be easily finetuned to new distributions and retains core open-source principles, albeit with a less refined aesthetic compared to proprietary versions.

Architecture

LibreFLUX employs a T5-XXL transformer model, utilizing attention masking to address padding token issues. The model is structured to facilitate easy finetuning and supports parameter-efficient training methods, like LoKr and LyCORIS, to accommodate smaller GPU capacities.

Training

The model underwent training using a combination of LoKr and quanto for int8 precision handling. It was trained on approximately 1,500 H100 hour equivalents and utilized diverse datasets without aesthetic selection, which enhanced its ability to generate varied imagery. The training process included innovative techniques like Beta timestep scheduling and multi-rank stratified sampling to optimize performance.

Guide: Running Locally

To run LibreFLUX locally, ensure you have the necessary libraries:

1. Install Diffusers:

   !pip install diffusers==0.30.3
   

2. Set Up the Pipeline:

   import torch
   from diffusers import DiffusionPipeline
   
   pipe = DiffusionPipeline.from_pretrained(
       "jimmycarter/LibreFLUX",
       custom_pipeline="jimmycarter/LibreFLUX",
       use_safetensors=True,
       torch_dtype=torch.bfloat16,
       trust_remote_code=True,
   )
   

3. Prepare for Inference:
   • Use CFG scales between 2.0 and 5.0 for optimal results.
   • Adjust VRAM usage with quantization if necessary:

     from optimum.quanto import freeze, quantize, qint8
     
     quantize(
         pipe.transformer,
         weights=qint8,
         exclude=["*.norm", "*.norm1", "*.norm2", "*.norm2_context", "proj_out", "x_embedder", "norm_out", "context_embedder"],
     )
     freeze(pipe.transformer)
     pipe.enable_model_cpu_offload()
     

4. Generate Images:

   prompt = "Photograph of a chalk board on which is written: 'I thought what I'd do was, I'd pretend I was one of those deaf-mutes.'"
   negative_prompt = "blurry"
   images = pipe(
     prompt=prompt,
     negative_prompt=negative_prompt,
     return_dict=False,
   )
   images[0][0].save('chalkboard.png')
   

For better performance, consider using cloud GPUs like NVIDIA's A100 or V100 series.

License

LibreFLUX is released under the Apache License 2.0, allowing for free use, modification, and distribution, provided that licensing terms are followed.