Misto Line_ Flux.dev

Introduction

MistoLine_Flux.dev, developed by TheMisto.ai in Shenzhen, China, is a ControlNet network designed for lineart and outline sketches. It is compatible with Flux1.dev and utilizes a scalable Transformer module as its backbone. The model employs a dual-stream Transformer structure to enhance alignment and expressiveness without increasing inference time. It is trained with T5 and clip-l TextEncoders for balanced performance between conditioning images and text prompts.

Architecture

The model's architecture includes a dual-stream Transformer structure, enhancing capabilities for various lineart and outline conditions. It is specifically designed for use with the Flux ControlNet ComfyUI suite and is not compatible with XLabs loaders and samplers. The model supports Flux1.dev's fp16/fp8 formats and other quantized models but has not been tested with ByteDance 8/16-step distilled models.

Training

Training MistoLine_Flux.dev involves significant computational resources, including A100-80GB GPUs with bf16 mixed precision. The training process is costly and requires multi-GPU, multi-node parallel training for larger models. Extensive ablation experiments were conducted to balance performance with training costs.

Guide: Running Locally

1. Download the Model: Access the model from MistoLine_Flux.dev_v1.
2. Setup: Place the model in the ComfyUI\models\TheMisto_model\ directory. This directory is created automatically upon running the ComfyUI suite.
3. Run: Use ComfyUI to execute the model. Ensure that the conditioning image's dimensions are divisible by 16 to prevent errors.
4. Hardware Recommendation: Utilize cloud GPUs, such as NVIDIA A100, for optimal performance.

License

MistoLine_Flux.dev is released under the FLUX.1 [dev] Non-Commercial License. It is intended for research and educational purposes only and is not permitted for commercial use.