FuseChat-Gemma-2-9B-Instruct

Introduction

FuseChat-3.0 models are designed to enhance performance by integrating multiple source large language models (LLMs) into more compact target LLMs, resulting in improved capabilities in tasks like conversation, instruction following, mathematics, and coding.

Architecture

FuseChat-3.0 employs source LLMs such as Gemma-2-27B-It, Mistral-Large-Instruct-2407, Qwen-2.5-72B-Instruct, and Llama-3.1-70B-Instruct to enhance target LLMs like Llama-3.1-8B-Instruct and Gemma-2-9B-It through implicit model fusion (IMF). This involves a two-stage pipeline: Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO).

Training

The training pipeline includes:

Supervised Fine-Tuning (SFT): Targets discrepancies between source and target models.
Direct Preference Optimization (DPO): Learns preferences from multiple source LLMs.

Using Llama-Factory and alignment-handbook libraries, models are fine-tuned and optimized with specific learning rates and batch sizes. The training involves handling datasets for instruction following, mathematics, coding, and Chinese language.

Guide: Running Locally

Setup Environment:
- Clone the GitHub repository: git clone https://github.com/SLIT-AI/FuseChat-3.0
- Install dependencies: pip install -r requirements.txt
Model Download:
- Access the model on Hugging Face: FuseAI on Hugging Face
Run Inference:
- Load the model using Hugging Face Transformers library.
- Execute inference scripts for specific tasks like instruction following or coding.

Cloud GPUs Suggested:

Use services like AWS, Google Cloud, or Azure for GPU instances to handle large model computations efficiently.

License

Details regarding the licensing of FuseChat models can be found in the Hugging Face repository or the associated GitHub page. Ensure compliance with open-source licenses when using the models.