Mistral-7B-v0.3 Model Documentation

Introduction

The Mistral-7B-v0.3 is a large language model developed as an enhancement of the Mistral-7B-v0.2, featuring an extended vocabulary of 32,768 tokens. This model is designed for text generation tasks and can be easily integrated with existing frameworks like Hugging Face Transformers.

Architecture

Mistral-7B-v0.3 builds upon its predecessor by expanding its vocabulary, allowing for more nuanced and diverse text generation. The architecture supports improvements in natural language processing tasks, though it does not incorporate built-in moderation mechanisms.

Training

The model demonstrates the capability of the base structure to be fine-tuned for specific tasks. However, further engagement with the community is encouraged to refine and implement moderation features for broader application, ensuring outputs respect necessary guidelines.

Guide: Running Locally

1. Installation:
   Install the mistral_inference package:

   pip install mistral_inference
   

2. Download Model Files:
   Use the following Python script to download the necessary model files:

   from huggingface_hub import snapshot_download
   from pathlib import Path
   
   mistral_models_path = Path.home().joinpath('mistral_models', '7B-v0.3')
   mistral_models_path.mkdir(parents=True, exist_ok=True)
   
   snapshot_download(repo_id="mistralai/Mistral-7B-v0.3", allow_patterns=["params.json", "consolidated.safetensors", "tokenizer.model.v3"], local_dir=mistral_models_path)
   

3. Run a Demo:
   Execute a demo using the command line:

   mistral-demo $HOME/mistral_models/7B-v0.3
   

4. Generate Text with Transformers:
   Use the Hugging Face Transformers library:

   from transformers import AutoModelForCausalLM, AutoTokenizer
   
   model_id = "mistralai/Mistral-7B-v0.3"
   tokenizer = AutoTokenizer.from_pretrained(model_id)
   
   model = AutoModelForCausalLM.from_pretrained(model_id)
   inputs = tokenizer("Hello my name is", return_tensors="pt")
   
   outputs = model.generate(**inputs, max_new_tokens=20)
   print(tokenizer.decode(outputs[0], skip_special_tokens=True))
   

5. Cloud GPUs:
   For optimal performance, consider using cloud-based GPU services such as AWS EC2, Google Cloud Platform, or Azure.

License

The Mistral-7B-v0.3 model is licensed under the Apache 2.0 License, allowing for wide usage and distribution with minimal restrictions. Please refer to the license text for detailed terms and conditions.