EVA-Qwen2.5-32B V0.2 Documentation

Introduction

EVA-Qwen2.5-32B V0.2 is a specialized model for role-playing and story writing. It is a full-parameter fine-tuning of the Qwen2.5-32B model using a mix of synthetic and natural datasets. The model aims to enhance versatility, creativity, and richness in generated content.

Architecture

The model is based on the Qwen2.5-32B architecture with modifications and improvements through fine-tuning. It supports features such as Axolotl integrations and various plugins to optimize performance. Recommended parameter values include a temperature of 1, Min-P of 0.05, Top-A of 0.2, and a repetition penalty of 1.03.

Training

Training involved a diverse set of datasets including Celeste 70B 0.1 data mixture, Opus Instruct, ChatGPT-4o-WritingPrompts, and others. The model was trained for 7 hours using 8xH100 SXM GPUs provided by FeatherlessAI. The training process involved reprocessing datasets to remove data poisoning issues and ensure stability.

Guide: Running Locally

To run EVA-Qwen2.5-32B locally, you can follow these steps:

1. Set Up Environment: Ensure you have Python and necessary dependencies installed.
2. Download Model: Clone the model repository from Hugging Face or download the model files directly.
3. Install Libraries: Install the transformers library and any other required packages.
4. Load Model: Use the transformers library to load the model and tokenizer.
5. Run Inference: Use the model to generate text with specified prompts and parameters.

For optimal performance, consider using cloud GPUs such as those from AWS, Azure, or Google Cloud.

License

The model is licensed under the Apache 2.0 License, allowing for extensive use and modification, provided that proper attribution is given.