S D3.5 Lo R A Chinese Line Art

Introduction

The SD3.5-LoRA-Chinese-Line-Art model by Shakker Labs is a text-to-image model that utilizes LoRA (Low-Rank Adaptation) and is based on the Stable Diffusion framework. It is designed to generate images in the style of Chinese line art.

Architecture

This model builds on the stabilityai/stable-diffusion-3.5-large base model. It incorporates LoRA technology to adapt and fine-tune the model towards generating specific artistic styles, such as Chinese line art.

Training

The model was trained using a combination of textual prompts and image outputs, with a focus on generating high-quality Chinese line art. The LoRA technique allows for efficient fine-tuning by using low-rank updates, which enable the model to adapt to this new style without retraining from scratch.

Guide: Running Locally

To run the SD3.5-LoRA-Chinese-Line-Art model locally, follow these steps:

1. Install Required Libraries:

   pip install diffusers>=0.31.0 torch
   

2. Load the Model:

   import torch
   from diffusers import StableDiffusion3Pipeline
   
   pipe = StableDiffusion3Pipeline.from_pretrained(
       "stabilityai/stable-diffusion-3.5-large", 
       torch_dtype=torch.bfloat16
   )
   pipe.load_lora_weights(
       "Shakker-Labs/SD3.5-LoRA-Chinese-Line-Art", 
       weight_name="SD35-lora-Chinese-Line-Art.safetensors"
   )
   pipe.fuse_lora(lora_scale=1.0)
   pipe.to("cuda")
   

3. Generate an Image:

   prompt = "a boat on the river, mountain in the distance, Chinese line art"
   negative_prompt = "(lowres, low quality, worst quality)"
   
   image = pipe(
       prompt=prompt,
       negative_prompt=negative_prompt,
       num_inference_steps=24, 
       guidance_scale=4.0,
       width=960, height=1280
   ).images[0]
   image.save("toy_example.jpg")
   

4. Cloud GPU Recommendation: For optimal performance, running the model on a cloud GPU is recommended. Services such as Google Colab, AWS EC2 instances with GPU support, or Azure's GPU VM are suitable options.

License

The model is distributed under the stabilityai-ai-community license. For more details, refer to the license document.