ULTIIMA-32B Model Documentation

Introduction

ULTIIMA-32B is a pre-trained language model designed for text generation, utilizing advanced merging techniques to enhance performance. The model is hosted on Hugging Face and built using the transformers library with support for Safetensors.

Architecture

The ULTIIMA-32B model is a result of merging two models: Qwen/Qwen2.5-32B-Instruct and Qwen/Qwen2.5-32B. The merge process utilizes the TIES method, which optimizes model integration for improved inference capabilities.

Training

The model was configured using a YAML setup that defines parameters such as weight, density, and normalization, ensuring a balanced and efficient merge. The merging process emphasizes the use of float16 data types to maintain precision while optimizing computational efficiency.

Guide: Running Locally

To run the ULTIIMA-32B model locally, follow these steps:

Install Dependencies: Ensure that you have the transformers library installed.
```
pip install transformers
```
Clone the Repository: Download the model from Hugging Face.
```
git clone https://huggingface.co/Sakalti/ultiima-32B
```

Load the Model: Use the transformers library to load the model in your Python environment.

from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Sakalti/ultiima-32B")
model = AutoModelForCausalLM.from_pretrained("Sakalti/ultiima-32B")

For optimal performance, consider using cloud GPUs offered by providers like AWS, Google Cloud, or Azure.

License

The model is available under the Apache 2.0 license, allowing for both commercial and non-commercial use with attribution.