B R I A 2.3 Inpainting

Introduction

BRIA 2.3 Inpainting is a cutting-edge model developed by BRIA AI, designed for inpainting tasks. It is ideal for commercial use and ensures full legal liability coverage. The model fills masked areas in images based on textual prompts, facilitating object removal, replacement, addition, and image expansion.

Architecture

BRIA 2.3 is a latent diffusion image-to-image model. It features a 'zero-SNR' noise scheduling, which reduces bias towards initial noise and enhances fidelity to the input image, particularly beneficial for image expansion and object removal tasks.

Training

The model was trained on a professional-grade, licensed dataset, ensuring high-quality outputs suitable for commercial applications. This dataset excludes copyrighted materials, public figures, and harmful content, providing legal liability coverage for users.

Guide: Running Locally

To run the BRIA 2.3 Inpainting model locally, follow these steps:

1. Install Dependencies: Ensure you have Python and the necessary libraries installed, including torch, diffusers, and PIL.
2. Download Images: Use URLs to download and resize the initial and mask images to 1024x1024 pixels.
3. Load Model Components: Load the UNet model and DDIM scheduler from the pre-trained BRIA 2.3.
4. Initialize Pipeline: Set up the StableDiffusionXLInpaintPipeline with the loaded components and specify the use of a GPU.
5. Generate Images: Define prompts, utilize the generator for reproducibility, and create inpainted images.

import PIL
import requests
import torch
from io import BytesIO
from diffusers import StableDiffusionXLInpaintPipeline, DDIMScheduler, UNet2DConditionModel

def download_image(url):
    response = requests.get(url)
    return PIL.Image.open(BytesIO(response.content)).convert("RGB")

img_url = "image_url"
mask_url = "mask_url"

init_image = download_image(img_url).resize((1024, 1024))
mask_image = download_image(mask_url).resize((1024, 1024))

unet = UNet2DConditionModel.from_pretrained(
    "briaai/BRIA-2.3-Inpainting",
    subfolder="unet",
    torch_dtype=torch.float16,
)

scheduler = DDIMScheduler.from_pretrained("briaai/BRIA-2.3", subfolder="scheduler", 
                                                        rescale_betas_zero_snr=True,prediction_type='v_prediction',timestep_spacing="trailing",clip_sample=False)

pipe = StableDiffusionXLInpaintPipeline.from_pretrained(
    "briaai/BRIA-2.3",
    unet=unet,
    scheduler=scheduler,
    torch_dtype=torch.float16,
    force_zeros_for_empty_prompt=False
)
pipe = pipe.to("cuda")

prompt = "A ginger cat sitting"
generator = torch.Generator(device='cuda:0').manual_seed(123456)
image = pipe(prompt=prompt, image=init_image, mask_image=mask_image, generator=generator, guidance_scale=5, strength=1).images[0]
image.save("./ginger_cat_on_park_bench.png")

Cloud GPUs: For optimal performance, consider using cloud GPU platforms like AWS, Google Cloud, or Azure.

License

BRIA 2.3 Inpainting is available under the bria-2.3 license, which requires purchase for access. The license provides full legal liability for commercial use. For detailed licensing terms, refer to the BRIA AI license agreement.