lcm lora sdv1 5

Introduction

The Latent Consistency Model (LCM) LoRA is designed to accelerate Stable Diffusion models, specifically the runwayml/stable-diffusion-v1-5. By utilizing distilled consistency adapters, it significantly reduces the number of inference steps to just 2-8 steps.

Architecture

LCM-LoRA integrates with the 🤗 Hugging Face Diffusers library and acts as an adapter for the Stable Diffusion model, enabling faster inference. The model operates by loading and fusing LoRA weights with a pre-trained pipeline. It supports tasks such as text-to-image, image-to-image, inpainting, and ControlNet applications with various configurations.

Training

Training details for LCM-LoRA are currently not provided.

Guide: Running Locally

1. Install Required Libraries: Ensure you have the latest versions of necessary libraries.

   pip install --upgrade pip
   pip install --upgrade diffusers transformers accelerate peft
   

2. Text-to-Image Generation:

   • Load the model and adapter.
   • Set the scheduler to LCMScheduler.
   • Reduce the inference steps to 2-8 and adjust the guidance_scale as needed.
   • Use the provided Python code snippet to run the pipeline.

3. Image-to-Image and Inpainting:

   • Similar steps to text-to-image, but use AutoPipelineForImage2Image or AutoPipelineForInpainting.
   • Load initial images and apply transformations as needed.

4. ControlNet:

   • Use StableDiffusionControlNetPipeline with canny edge detection.
   • Load and process images using OpenCV and PIL for ControlNet conditioning.

Suggested Cloud GPUs

For optimal performance, consider using cloud GPU providers such as AWS, Google Cloud, or Azure, which offer powerful GPUs like NVIDIA Tesla V100 or A100.

License

The model is released under the openrail++ license, which is designed for open access and usage.