Qwen2.5 0.5 B Instruct

Introduction

Qwen2.5 is the latest in the series of large language models developed by Qwen. This release includes base and instruction-tuned models ranging from 0.5 to 72 billion parameters. Key improvements include enhanced knowledge, coding, and mathematics capabilities, better instruction following, long-text generation, structured data comprehension, and multilingual support for over 29 languages. The 0.5B instruction-tuned model features a causal language model architecture with 0.49 billion parameters, supporting a context length of up to 32,768 tokens.

Architecture

• Type: Causal Language Models
• Training Stage: Pretraining & Post-training
• Architecture: Transformers with RoPE, SwiGLU, RMSNorm, Attention QKV bias, and tied word embeddings
• Number of Parameters: 0.49B total, 0.36B non-embedding
• Number of Layers: 24
• Number of Attention Heads (GQA): 14 for Q and 2 for KV
• Context Length: Up to 32,768 tokens, generating up to 8,192 tokens

Training

The Qwen2.5 model series has undergone pretraining and post-training stages, utilizing transformers. The training includes specialized expert models to enhance capabilities in various domains such as coding and mathematics.

Guide: Running Locally

1. Install Transformers Library: Ensure you have the latest version of Hugging Face's Transformers library, as older versions may cause errors.

   pip install transformers
   

2. Load Model and Tokenizer:

   from transformers import AutoModelForCausalLM, AutoTokenizer
   
   model_name = "Qwen/Qwen2.5-0.5B-Instruct"
   
   model = AutoModelForCausalLM.from_pretrained(
       model_name,
       torch_dtype="auto",
       device_map="auto"
   )
   tokenizer = AutoTokenizer.from_pretrained(model_name)
   

3. Generate Text:

   prompt = "Give me a short introduction to large language model."
   messages = [
       {"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."},
       {"role": "user", "content": prompt}
   ]
   text = tokenizer.apply_chat_template(
       messages,
       tokenize=False,
       add_generation_prompt=True
   )
   model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
   
   generated_ids = model.generate(
       **model_inputs,
       max_new_tokens=512
   )
   response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
   

4. Consider Cloud GPUs: For optimal performance, especially with large models, utilize cloud GPUs such as AWS EC2, Google Cloud, or Azure.

License

The Qwen2.5-0.5B-Instruct model is licensed under the Apache-2.0 License. For more details, visit the license link.