Dolphin3.0 Qwen2.5 3b G G U F

Introduction

Dolphin 3.0 Qwen 2.5 3B is part of the Dolphin 3.0 series, developed to be a versatile instruct-tuned model for coding, math, and general-purpose applications. It aims to provide businesses and users with a customizable AI solution, unlike other AI models that control system prompts and data use.

Architecture

This model is a quantized variant of cognitivecomputations/Dolphin3.0-Qwen2.5-3b, built using llama.cpp. The architecture leverages multiple datasets and a reward model to optimize performance for various applications. It provides users with complete control over system prompts and alignment.

Training

The Dolphin 3.0 model was trained using a variety of open-source datasets, including OpenCoder-LLM, Microsoft, NousResearch, and more. It was supported by several sponsors who provided the necessary computational resources, such as Crusoe Cloud, Akash, Lazarus, and Cerebras, with contributions from Andreessen Horowitz.

Guide: Running Locally

1. Install Dependencies: Ensure you have Python and necessary libraries such as transformers installed.
2. Download Model: Clone the repository and download the model files from Hugging Face.
3. Load Model: Use libraries like Hugging Face Transformers to load and interact with the model.
4. Inference: Customize the system prompt to set the tone and alignment for responses.

For enhanced performance, consider using cloud GPUs such as those from AWS, Google Cloud, or Azure.

License

The model is released under the qwen-research license. For more details, refer to the license page.