stable video diffusion img2vid xt 1 1 LLM Model

Introduction

Stable Video Diffusion 1.1 is an image-to-video model developed by Stability AI. It is a generative model that transforms a still image into a short video clip using latent diffusion techniques. The model is designed for research and non-commercial use, and specific terms apply for commercial applications.

Architecture

Stable Video Diffusion 1.1 is a latent diffusion model fine-tuned from the SVD Image-to-Video model. It generates 25-frame video clips at a resolution of 1024x576 pixels. The model uses fixed conditioning at 6 FPS with Motion Bucket Id 127 to enhance output consistency. It was developed and funded by Stability AI.

Training

The model was trained with a focus on maintaining output consistency without requiring extensive hyperparameter adjustments. Fine-tuning was performed on the SVD Image-to-Video model with fixed conditioning settings, though these settings can be adjusted if necessary. The model is capable of generating short video clips from a context frame of the same size.

Guide: Running Locally

Clone Repository:
Clone the generative-models repository from GitHub:
```
git clone https://github.com/Stability-AI/generative-models
```
Install Dependencies:
Navigate into the cloned directory and install the required libraries:
```
cd generative-models
pip install -r requirements.txt
```
Download Model Weights:
Obtain the model weights from the Stability AI repository or Hugging Face's model hub.
Run Inference:
Use the provided scripts or notebooks to perform inference with the model, specifying the desired settings for conditioning and frame generation.
Consider Cloud GPUs:
For optimal performance, especially during training or large-scale inference, consider using cloud GPU services such as AWS, Google Cloud, or Azure.

License

The model is licensed under the Stability AI Community License Agreement. It allows for research and non-commercial use without charge, with specific terms for limited commercial use. Entities generating less than USD $1,000,000 in annual revenue may use the model without a commercial license. For more details and to register for commercial use, refer to Stability AI's licensing page.