dolphin 2.7 mixtral 8x7b

Introduction

Dolphin 2.7 Mixtral 8x7b is a model retrained from Dolphin-2.5/2.6, incorporating specific fixes in the Transformers library to improve performance. This model excels in coding tasks due to extensive training on coding data and is designed to be highly compliant, yet it lacks DPO tuning.

Architecture

Dolphin 2.7 is based on the Mixtral-8x7b architecture with a 32k context base model, finetuned to 16k. The new version includes specific mixtral fixes and an unfrozen gate layer to address performance issues. The model is uncensored, with a filtered dataset to enhance compliance, but users should apply their own alignment layers to manage ethical compliance.

Training

The model was trained for 1.5 epochs over three days using 4x A100 GPUs, qLoRA, and Axolotl frameworks. It uses the ChatML prompt format, emphasizing compliance and the potential ethical concerns of uncensored models. Users are advised to implement their own ethical and alignment measures when deploying this model.

Guide: Running Locally

1. Setup Environment: Ensure you have a Python environment with PyTorch and the Transformers library installed.
2. Download Model: Clone the Dolphin 2.7 Mixtral 8x7b repository from Hugging Face.
3. Install Dependencies: Install any additional dependencies specified in the repository.
4. Run Model: Load and run the model locally using a script or interactive Python session.
5. Cloud GPUs: Consider using cloud services like AWS, GCP, or Azure with GPU instances (A100 recommended) for efficient model training and inference.

License

The Dolphin 2.7 Mixtral 8x7b model is licensed under the Apache-2.0 license, allowing for wide use and distribution with appropriate credit. Users should comply with the terms and conditions outlined in the license.