Lumimaid Magnum v4 12 B G G U F LLM Model

Introduction

LUMIMAID-MAGNUM-V4-12B-GGUF is a merged model combining elements from Lumimaid and Magnum, specifically using Magnum v4. The model is designed for conversational use and has been enhanced using the DELLA merge method with additional fine-tuning on specific inputs.

Architecture

The model architecture merges several base models:

mistralai/Mistral-Nemo-Instruct-2407
NeverSleep/Lumimaid-v0.2-12B
Undi95/LocalC-12B-e2.0
anthracite-org/magnum-v4-12b

It utilizes the transformers library and is tagged with mergekit.

Training

The model was trained using the DELLA merge method within the mergekit framework. It includes fine-tuning of the Nemo component on Claude input, specifically trained with a context size of 16k.

Guide: Running Locally

To run the model locally, follow these basic steps:

Clone the Repository: Ensure you have access to the model files.
Install Dependencies: Use the transformers library for implementation.
Set Up Environment: Prepare your local machine or a cloud environment.
Run Inference: Utilize the provided prompt template for input and output processing.

Suggested Cloud GPUs

For optimal performance, consider using cloud GPUs from providers like AWS, Azure, or Google Cloud.

License

Please check the model's repository for specific licensing details, as it is crucial to comply with any usage restrictions or requirements.

More Related APIs