Chinese F L U X.1 adapter

Introduction

The MultilingualFLUX.1-adapter is a multilingual adapter designed for the Flux.1 series models, optimized to support over 100 languages, with additional enhancements for Chinese. It is based on the ECCV 2024 paper titled "PEA-Diffusion."

Architecture

The model utilizes the multilingual encoder byt5-xxl, and the teacher model for the adaptation process is FLUX.1-schnell. It employs a reverse denoising process for distillation training. The adapter is applicable to any FLUX.1 series model, with specific optimizations for Chinese.

Training

The adapter applies a reverse denoising process for distillation training. It requires adjusting parameters such as num_inference_steps and guidance_scale when applied to other FLUX.1 series models.

Guide: Running Locally

1. Environment Setup: Ensure you have Python installed with packages like torch, transformers, and diffusers.
2. Model Preparation: Load the necessary models, including the byt5-xxl for text encoding and the FLUX.1-schnell as the base model.
3. Running the Model: Implement the provided Python script, which processes input text to generate images.
4. Adjust Parameters: Modify num_inference_steps and guidance_scale as needed for desired output.
5. Hardware Recommendations: Use a machine with a GPU for efficient processing. Cloud GPU services like AWS, Google Cloud, or Azure are recommended for handling large models.

License

The adapter is licensed under the Apache License 2.0. However, it must comply with the main model's license, such as the FLUX.1 Non-Commercial License.