Midnight Miqu 103 B v1.5

Introduction

The Midnight-Miqu-103B-v1.5 model is a 103 billion parameter model created by merging sophosympatheia's Midnight-Miqu-70B-v1.5 with itself. It is designed for text generation with a context length of 32k, leveraging the capabilities of the Miqu framework.

Architecture

The model is built using the transformers library and supports various quantization methods, including GGUF and EXL2, with multiple configurations provided for each. These configurations allow the model to function with different levels of bit precision and memory efficiency.

Training

The Midnight-Miqu-103B-v1.5 was derived by combining layers from the Midnight-Miqu-70B-v1.5 model using the passthrough merge method. This involved detailed layer slicing and merging to enhance performance while maintaining compatibility with the original architecture.

Guide: Running Locally

To run the Midnight-Miqu-103B-v1.5 model locally, follow these simplified steps:

1. Install dependencies: Ensure you have Python and the transformers library installed.
2. Download the model: Access the model from the Hugging Face repository.
3. Load the model: Use the transformers library to load the model into your project.
4. Inference: Execute inference tasks using a suitable script or interactive session.

For optimal performance, it is recommended to use cloud GPUs from providers like AWS, Google Cloud, or Azure due to the model's size and computational demands.

License

The Midnight-Miqu-103B-v1.5 model is based on a leaked version of Mistral's models, and thus, is only suitable for personal use. Users assume any legal risks associated with downloading and using the model. It is advised to consult legal counsel before using this or any Hugging Face model beyond private use.