Hermes Trismegistus Mistral 7 B G G U F

Introduction

The Hermes Trismegistus Mistral 7B model, created by Teknium, is designed for tasks related to the esoteric, occult, and spiritual domains. It has been fine-tuned, incorporating synthetic, GPT-4 generated datasets, to enhance performance across a variety of tasks.

Architecture

The model employs the Mistral architecture and is available in the GGUF format, a new model file format introduced as a replacement for GGML. It supports multiple quantization methods, optimizing for various use cases and hardware configurations.

Training

The model was trained using approximately 10,000 synthetic examples, generated by GPT-4, focusing on esoteric and spiritual knowledge. This training enhances its capability to perform tasks in these specialized areas.

Guide: Running Locally

1. Environment Setup

   • Install the huggingface-hub Python library:

     pip3 install huggingface-hub
     

   • Consider installing hf_transfer for faster downloads on high-speed connections.

2. Downloading Model Files

   • Use the huggingface-cli to download the desired model file:

     huggingface-cli download TheBloke/Hermes-Trismegistus-Mistral-7B-GGUF hermes-trismegistus-mistral-7b.Q4_K_M.gguf --local-dir . --local-dir-use-symlinks False
     

3. Running the Model

   • Use llama.cpp or a compatible interface with GPU support for local inference:

     ./main -ngl 32 -m hermes-trismegistus-mistral-7b.Q4_K_M.gguf --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "USER: {prompt}\nASSISTANT:"
     

4. Cloud GPU Recommendation

   • For improved performance, consider using cloud GPU services such as AWS, Google Cloud, or Azure.

License

The Hermes Trismegistus Mistral 7B model is distributed under the Apache 2.0 license, allowing for wide use and modification under specified terms.