stable zero123

Introduction

Stable Zero123 is a view-conditioned image generation model developed by Stability AI. It is based on the original Zero123 and offers enhanced data rendering and conditioning strategies for improved performance. The model is particularly effective for generating 3D models from 2D images and supports text-to-3D generation.

Architecture

Stable Zero123 is a latent diffusion model finetuned from the lambdalabs/sd-image-variations-diffusers model. It can produce high-quality 3D models using Score Distillation Sampling (SDS). The model supports open research in 3D object generation through improved open-source code of threestudio.

Training

Stable Zero123 was trained on renders from the Objaverse dataset using an enhanced rendering method. Training was performed on Stability AI's cluster on a single node with 8 A100 80GB GPUs. The model builds upon the original zero123 codebase, modified for enhanced performance.

Guide: Running Locally

To run Stable Zero123 locally for 3D mesh generation:

1. Install Threestudio: Follow the installation instructions from the threestudio repository.

2. Download Checkpoint: Save the Stable Zero123 checkpoint (stable_zero123.ckpt) in the load/zero123/ directory.

3. Prepare Image: Use an AI image generator to create an image, such as a 3D render of a subject (e.g., "A friendly dog"). Remove the background and save it to load/images/ with _rgba.png as the suffix.

4. Run Model: Execute the model using the command:

   python launch.py --config configs/stable-zero123.yaml --train --gpu 0 data.image_path=./load/images/your_image_rgba.png
   

For optimal performance, consider using cloud GPUs, such as those provided by AWS or Google Cloud, to handle resource-intensive tasks.

License

Stable Zero123 is available under two licenses. The non-commercial version, which includes some CC-BY-NC 3D objects, is under the Stability AI Non-Commercial Research Community License. The commercial version, Stable Zero123C, trained only on CC-BY and CC0 objects, is released under the Stability AI Community License. For detailed licensing information, refer to the Stability AI license page.