stable diffusion xl 1.0 tensorrt

Introduction

The Stable Diffusion XL 1.0 TensorRT repository contains TensorRT versions (sdxl, sdxl-lcm, sdxl-lcmlora) of Stable Diffusion XL 1.0, developed in collaboration with NVIDIA. These optimized versions enhance speed and efficiency for text-to-image generation tasks.

Architecture

• Developer: Stability AI
• Model Type: Diffusion-based text-to-image generative model
• Base Model: SDXL base 1.0 and SDXL refiner 1.0 converted for NVIDIA TensorRT optimized inference
• License: CreativeML Open RAIL++-M License

Training

Performance improvements are demonstrated through timing and throughput comparisons at 1024x1024 resolution for 30 steps:

• A10: 13% timing improvement, 20% throughput improvement
• A100: 26% timing improvement, 33% throughput improvement
• H100: 41% timing improvement, 70% throughput improvement

For the Latent Consistency Model (LCM) version:

• A100: Total time 426.16 ms
• H100: Total time 234.22 ms

Guide: Running Locally

1. Setup Environment: Use a TensorRT NGC container.

   git clone https://github.com/rajeevsrao/TensorRT.git
   cd TensorRT
   git checkout release/9.2
   docker run --rm -it --gpus all -v $PWD:/workspace nvcr.io/nvidia/pytorch:23.11-py3 /bin/bash
   

2. Download SDXL TensorRT Files:

   git lfs install 
   git clone https://huggingface.co/stabilityai/stable-diffusion-xl-1.0-tensorrt
   cd stable-diffusion-xl-1.0-tensorrt
   git lfs pull
   cd ..
   

3. Install Libraries:

   cd demo/Diffusion
   python3 -m pip install --upgrade pip
   pip3 install -r requirements.txt
   python3 -m pip install --pre --upgrade --extra-index-url https://pypi.nvidia.com tensorrt
   

4. Inference with TensorRT:

   • SDXL:

     python3 demo_txt2img_xl.py "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" --build-static-batch --use-cuda-graph --num-warmup-runs 1 --width 1024 --height 1024 --denoising-steps 30 --onnx-base-dir /workspace/stable-diffusion-xl-1.0-tensorrt/sdxl-1.0-base --onnx-refiner-dir /workspace/stable-diffusion-xl-1.0-tensorrt/sdxl-1.0-refiner
     

   • SDXL-LCM:

     python3 demo_txt2img_xl.py "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" --version=xl-1.0 --onnx-dir /workspace/stable-diffusion-xl-1.0-tensorrt/lcm --engine-dir /workspace/stable-diffusion-xl-1.0-tensorrt/lcm/engine-sdxl-lcm-nocfg --scheduler LCM --denoising-steps 4 --guidance-scale 0.0 --seed 42
     

   • SDXL-LCMLORA:

     python3 demo_txt2img_xl.py "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" --version=xl-1.0 --onnx-dir /workspace/stable-diffusion-xl-1.0-tensorrt/lcmlora --engine-dir /workspace/stable-diffusion-xl-1.0-tensorrt/lcm/engine-sdxl-lcmlora-nocfg --scheduler LCM --lora-path latent-consistency/lcm-lora-sdxl --lora-scale 1.0 --denoising-steps 4 --guidance-scale 0.0 --seed 42
     

License

The model is licensed under the CreativeML Open RAIL++-M License.