Dazzling Star Aurora 32b v0.0 Experimental 1130

Introduction

The Dazzling-Star-Aurora-32B-v0.0-Experimental-1130 is an experimental text generation model resulting from the merging of multiple advanced models. Developed by ProdeusUnity, this model is designed to explore new possibilities in text generation through innovative model merging techniques.

Architecture

The architecture of Dazzling-Star-Aurora-32B involves a combination of several models, specifically the "Qwen" models. The merging was achieved using the TIES method, leveraging the Qwen_Qwen2.5-32B as the base model. The merged models include:

• EVA-UNIT-01/EVA-Qwen2.5-32B-v0.2
• ArliAI/Qwen2.5-32B-ArliAI-RPMax-v1.3

Training

The training process involved a configuration with specific parameters for each model to be merged. The YAML configuration used the following settings:

• EVA-UNIT-01/EVA-Qwen2.5-32B-v0.2 with weight 0.3 and density 0.7
• ArliAI/Qwen2.5-32B-ArliAI-RPMax-v1.3 with weight 0.4 and density 0.8
• Base Model: Qwen/Qwen2.5-32B
• Parameters: epsilon 0.05, lambda 1, int8_mask true, normalize true
• Merge Method: TIES
• Data Type: bfloat16

Guide: Running Locally

To run the Dazzling-Star-Aurora-32B model locally, follow these steps:

1. Clone the model repository from Hugging Face.
2. Install the required dependencies, primarily the transformers library.
3. Load the model using the transformers library and specify the configuration parameters as outlined in the YAML configuration.

Suggested Cloud GPUs

For optimal performance, consider using cloud GPUs such as NVIDIA A100 or V100, available on platforms like AWS, Google Cloud, or Azure.

License

The model's license information is not explicitly mentioned. Users should check the Hugging Face model repository for detailed licensing terms and ensure compliance with any applicable use restrictions.