SDXL-Lightning Model Documentation

Introduction

SDXL-Lightning is a high-speed text-to-image generation model capable of producing high-quality 1024px images quickly. The model is detailed in the research paper "SDXL-Lightning: Progressive Adversarial Diffusion Distillation". It is open-sourced as part of the research initiative and is based on distilled versions of stabilityai's stable-diffusion-xl-base-1.0. The repository includes checkpoints for models with 1-step, 2-step, 4-step, and 8-step distillation, with varying quality and experimental status.

Architecture

The SDXL-Lightning model is designed to perform text-to-image transformations using a progressive adversarial diffusion approach. It includes both full UNet and LoRA checkpoints. The UNet checkpoints offer superior image quality, while the LoRA checkpoints provide flexibility for integration with other base models.

Training

The model is distilled from the stabilityai/stable-diffusion-xl-base-1.0, with checkpoints available for different step models:

• 2-step, 4-step, 8-step models: High-quality generation with UNet checkpoints.
• 1-step model: Experimental, using "sample" prediction instead of "epsilon".

Guide: Running Locally

To run SDXL-Lightning locally, follow these steps:

1. Install Required Libraries: Ensure you have torch, diffusers, and huggingface_hub installed.
2. Download Checkpoints: Use Hugging Face Hub to download the appropriate checkpoint for your desired step setting.
3. Load and Configure the Model:
   • For UNet models, load the UNet and configure the pipeline using StableDiffusionXLPipeline.
   • For LoRA models, load the pipeline and apply LoRA weights.
4. Set Scheduler: Configure the scheduler with EulerDiscreteScheduler and ensure it uses "trailing" timesteps.
5. Run Inference: Use the pipeline to generate images with specified inference steps and guidance scale.

Cloud GPUs such as those provided by AWS, Google Cloud, or Azure are recommended for optimal performance, especially for high-resolution image generation tasks.

License

The SDXL-Lightning model is released under the OpenRAIL++ license, allowing for open and flexible use with certain conditions.