F L U X.1 dev Control Net Union Pro
Shakker-LabsIntroduction
FLUX.1-DEV-CONTROLNET-UNION-PRO is a sophisticated text-to-image model jointly developed by the InstantX Team and Shakker Labs. It is a Pro version of the FLUX.1-dev-Controlnet-Union model, equipped with enhanced training steps and datasets. This model is designed to operate with multiple control modes, making it versatile for various image generation applications.
Architecture
The model supports seven control modes: canny, tile, depth, blur, pose, gray, and low quality. It is based on the Stable Diffusion architecture and utilizes the Diffusers library for efficient image generation. The model also allows integration with other ControlNets, enhancing its functionality and flexibility.
Training
FLUX.1-DEV-CONTROLNET-UNION-PRO has undergone extensive training with a variety of datasets and additional steps to improve its performance over the base FLUX.1-dev model. The recommended controlnet_conditioning_scale ranges from 0.3 to 0.8, optimizing the model's output quality across different control modes.
Guide: Running Locally
To run the FLUX.1-DEV-CONTROLNET-UNION-PRO model locally, follow these steps:
- Install Diffusers: Clone the Diffusers repository and install it from the source since the required PR is not yet included in the released version.
git clone https://github.com/huggingface/diffusers cd diffusers pip install -e .
- Setup the Environment: Ensure you have PyTorch installed and a compatible CUDA version for GPU support.
- Load the Model: Use the provided Python code to load the model and perform inference. Replace image URLs and prompts according to your needs.
- Inference with Multi-Control: Utilize the FluxMultiControlNetModel class for handling multiple control inputs.
Suggested Cloud GPUs
For optimal performance, consider using cloud services with GPUs such as NVIDIA V100 or A100, available on platforms like AWS, Google Cloud, or Azure.
License
The model is distributed under the flux-1-dev-non-commercial-license. For more details, refer to the license document. Usage is restricted to non-commercial purposes unless otherwise authorized.