Mini C P M3 4 B

Introduction

MiniCPM3-4B is the third generation in the MiniCPM series, offering superior performance compared to models like Phi-3.5-mini-Instruct and GPT-3.5-Turbo-0125, and is competitive with several recent 7B~9B models. It provides advanced features such as a function call and code interpreter, and is designed for general usage. The model has a 32k context window and utilizes LLMxMapReduce for handling larger contexts efficiently.

Architecture

MiniCPM3-4B is designed for text generation tasks and supports both Chinese and English languages. It is implemented using the Transformers library in PyTorch, and features a 32k context window to manage extensive data inputs.

Training

The MiniCPM3-4B model was trained using scalable strategies to optimize its performance across various benchmarks. These strategies are detailed in its corresponding paper on arXiv.

Guide: Running Locally

1. Environment Setup: Ensure you have Python and PyTorch installed. It's recommended to use a virtual environment.
2. Install Transformers:

   pip install transformers torch
   

3. Clone the Repository: Access the model files via Hugging Face or clone the relevant repository.
4. Load the Model:

   from transformers import AutoModelForCausalLM, AutoTokenizer
   import torch
   
   path = "openbmb/MiniCPM3-4B"
   device = "cuda"
   
   tokenizer = AutoTokenizer.from_pretrained(path, trust_remote_code=True)
   model = AutoModelForCausalLM.from_pretrained(path, torch_dtype=torch.bfloat16, device_map=device, trust_remote_code=True)
   

5. Run Inference: Use the sample code provided to generate text or perform tasks.
6. Hardware Requirements: A GPU is recommended for efficient processing. Consider using cloud GPU services such as AWS, GCP, or Azure for optimal performance.

License

The MiniCPM3-4B model is released under the Apache-2.0 License. Usage of the model weights must comply with the MiniCPM Model License. The model and its weights are free for academic purposes and can be used for commercial purposes following registration through a provided questionnaire.