redshift diffusion

Introduction

Redshift Diffusion is a fine-tuned Stable Diffusion model designed for generating high-resolution 3D artworks. The model is optimized to enhance the "redshift style" when used in text prompts. It was developed to improve the quality of renderings associated with the redshift render engine in the context of Cinema4D.

Architecture

Redshift Diffusion utilizes the StableDiffusionPipeline from the Hugging Face Diffusers library. It can be used similarly to other Stable Diffusion models and supports export to ONNX, MPS, and FLAX/JAX formats. The model is trained using diffusers-based DreamBooth training with prior-preservation loss and the train-text-encoder flag.

Training

The model underwent training with 11,000 steps, employing diffusers-based DreamBooth techniques. It was trained with specific settings to capture the "redshift style" effect, using prompts that include negative aspects to avoid, CFG scales, and seeds for reproducibility.

Guide: Running Locally

To use the Redshift Diffusion model locally, follow these steps:

1. Install Dependencies: Ensure you have Python and PyTorch installed. Install the Hugging Face Diffusers library:

   pip install diffusers
   

2. Set Up the Environment: Import the necessary libraries and load the model:

   from diffusers import StableDiffusionPipeline
   import torch
   
   model_id = "nitrosocke/redshift-diffusion"
   pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)
   pipe = pipe.to("cuda")
   

3. Generate Images: Use a prompt to generate an image:

   prompt = "redshift style magical princess with golden hair"
   image = pipe(prompt).images[0]
   image.save("./magical_princess.png")
   

4. Hardware Requirements: Running the model effectively requires a GPU. Consider using cloud GPU providers like AWS, GCP, or Azure for better performance.

License

The model is distributed under the CreativeML OpenRAIL-M license, which includes the following conditions:

1. You cannot use the model to create or distribute illegal or harmful content.
2. The authors do not claim rights over the outputs, which you can freely use, but you are responsible for their usage.
3. Redistribution of the model's weights for commercial purposes must include the same usage restrictions, and a copy of the license must be shared with users.

For full license details, refer to the CreativeML OpenRAIL-M License.