chatglm3 6b

Introduction

ChatGLM3-6B is the latest open-source model in the ChatGLM series, designed to offer a smooth dialogue experience with a low deployment threshold. Key features include:

1. More Powerful Base Model: Utilizes diverse training data and an optimized strategy, achieving top performance among pre-trained models under 10B.
2. Comprehensive Function Support: Incorporates a new prompt format and supports complex scenarios like function calls and code interpretation.
3. Open-source Series: Offers various models for research and free commercial use upon registration.

Architecture

ChatGLM3-6B is built on a robust base model, ChatGLM3-6B-Base, leveraging a wide range of datasets to enhance capabilities in semantics, mathematics, reasoning, code, and knowledge. It introduces a versatile prompt design and natively supports advanced operations such as code execution and agent tasks.

Training

The model is trained with a diverse dataset and extensive steps, employing reasonable strategies to ensure superior performance in various evaluation metrics. It supports both Chinese and English languages.

Guide: Running Locally

To run the ChatGLM3-6B model locally, follow these steps:

1. Install Dependencies:

   pip install protobuf transformers==4.30.2 cpm_kernels torch>=2.0 gradio mdtex2html sentencepiece accelerate
   

2. Load the Model:

   from transformers import AutoTokenizer, AutoModel
   tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm3-6b", trust_remote_code=True)
   model = AutoModel.from_pretrained("THUDM/chatglm3-6b", trust_remote_code=True).half().cuda()
   model = model.eval()
   

3. Generate Dialogue:

   response, history = model.chat(tokenizer, "你好", history=[])
   print(response)
   

For enhanced performance, consider using cloud GPUs such as AWS EC2 GPU instances or Google Cloud's GPU services.

License

The code is open-sourced under the Apache-2.0 License. The model weights require compliance with the Model License.