dolphin 2.9.4 llama3.1 8b

Introduction

Dolphin-2.9.4-LLAMA3.1-8B is an AI model designed by Eric Hartford and Cognitive Computations. The model is based on Meta Llama 3.1 8B and is known for its instruction-following, conversational, and coding skills. It is capable of executing complex tasks and following instructions in multiple languages, although it is uncensored and thus requires a custom alignment layer to ensure ethical compliance.

Architecture

The Dolphin-2.9.4 model is built on the Meta Llama 3.1-8B framework, featuring a 128K context length with finetuning accomplished using an 8192 sequence length. The model uses the ChatML prompt template format and has been optimized to handle a variety of tasks, including function calling.

Training

The model was trained using nine datasets, including "cognitivecomputations/Dolphin-2.9" and "microsoft/orca-math-word-problems-200k." The training employed a sequence length of 8192 with various hyperparameters such as a learning rate of 5e-6, a cosine learning rate scheduler, and an Adam optimizer. The training was distributed across eight GPUs with gradient accumulation steps set to 16.

Guide: Running Locally

To run the Dolphin-2.9.4 model locally, follow these basic steps:

1. Clone the model repository from Hugging Face.
2. Install dependencies using pip install transformers torch.
3. Load the model using the transformers library.
4. Ensure you have a compatible GPU setup, such as NVIDIA CUDA for optimal performance.

For better performance, consider using cloud GPUs from providers like AWS or Google Cloud, which offer a range of options for deep learning workloads.

License

The model is governed by the Llama 3.1 license. Users must ensure compliance with this license and are responsible for the content generated by the model.