Mistral Small Instruct 2409 LLM Model

Introduction

Mistral-Small-Instruct-2409 is a fine-tuned instruct model developed by Mistral AI, featuring 22 billion parameters and a vocabulary size of 32,768. It supports function calling and has a sequence length of 32k.

Architecture

The model is an instruct fine-tuned version with 22 billion parameters, designed to support advanced functionalities like function calling and handling long sequences of up to 32k tokens.

Training

The model is fine-tuned to understand and generate instructional content, making it suitable for tasks that require following complex instructions or performing specific actions based on contextual prompts.

Guide: Running Locally

Installation:
- Install the vLLM library (>= v0.6.1.post1):
```
pip install --upgrade vllm
```
- Install mistral_common (>= 1.4.1):
```
pip install --upgrade mistral_common
```
Running Offline:
- Use the vLLM library to load and interact with the model.
- Ensure your environment has at least 44 GB of GPU RAM, or use multiple GPUs with tensor_parallel.

Server/Client Setup:

Launch a server using vLLM:

vllm serve mistralai/Mistral-Small-Instruct-2409 --tokenizer_mode mistral --config_format mistral --load_format mistral

Interact with the server using a client, sending requests via HTTP.

Using Mistral Inference:
- Install mistral_inference (>= 1.4.1):
```
pip install mistral_inference --upgrade
```
- Download model files using huggingface_hub.
Cloud GPUs:
- Consider using cloud-based GPUs for efficient model inference, especially if local resources are limited.

License

The Mistral-Small-Instruct-2409 model is licensed under the Mistral AI Research License (MRL). The license allows non-commercial use for research purposes only. Commercial use requires a separate license from Mistral AI. For more details and to obtain a license, visit Mistral AI's license page.

Please note that users must comply with specific usage limitations and attribution requirements as outlined in the license agreement.

More Related APIs