NemoMix-Unleashed-12B-GGUF

Introduction

NemoMix-Unleashed-12B-GGUF is a merged language model created by MarinaraSpaghetti using the mergekit tool. This model is designed to enhance performance in tasks such as role-playing and storytelling by reducing repetition, especially in higher contextual settings. It incorporates contributions from several notable models, blending their unique strengths.

Architecture

The model is a combination of several pre-trained language models using the della_linear merge method. It utilizes bfloat16 data type for enhanced computational efficiency. The base model used in this merge is mistralaiMistral-Nemo-Base-2407.

Training

The model is not trained from scratch but is a merge of existing models, each contributing different weights and densities:

• Base Model: mistralaiMistral-Nemo-Base-2407
• Merged Models:
  • intervitens_mini-magnum-12b-v1.1
  • nbeerbower_mistral-nemo-bophades-12B
  • Sao10K_MN-12B-Lyra-v1
  • nbeerbower_mistral-nemo-gutenberg-12B
  • mistralaiMistral-Nemo-Instruct-2407

The merging process involved careful parameter tuning to balance the contributions from these models, with specific configurations for weight and density.

Guide: Running Locally

To run NemoMix-Unleashed-12B-GGUF locally, follow these steps:

1. Clone the Repository: Download the model files from the Hugging Face repository.
2. Environment Setup: Ensure you have the transformers library installed. Use a virtual environment for isolation.
3. Load the Model: Use the transformers library to load and interact with the model.
4. Configuration: Adjust the parameters such as Temperature and Top A as recommended to suit your specific needs.

For optimal performance, especially with large models like this one, consider using cloud GPUs from providers like AWS, Google Cloud, or Azure.

License

The model and its components are governed by the licenses of the individual models merged. Users should ensure they comply with these licenses when using the model.