Qwen2.5 Coder 14 B Instruct

Introduction

Qwen2.5-Coder is an advanced series of code-specific language models, previously termed CodeQwen. It includes six model sizes ranging from 0.5 to 32 billion parameters, suitable for various developer needs. Improvements over previous versions include enhanced code generation, reasoning, and fixing capabilities, with training using 5.5 trillion tokens. The Qwen2.5-Coder-32B model is state-of-the-art for open-source code LLMs, comparable to GPT-4o. It supports applications in coding and maintains strengths in mathematics and general tasks, with long-context handling up to 128K tokens.

Architecture

• Type: Causal Language Models
• Training Stage: Pretraining & Post-training
• Architecture: Transformers with RoPE, SwiGLU, RMSNorm, and Attention QKV bias
• Number of Parameters: 14.7B
• Non-Embedding Parameters: 13.1B
• Layers: 48
• Attention Heads (GQA): 40 for Q and 8 for KV
• Context Length: 131,072 tokens

Training

The Qwen2.5-Coder models are trained with 5.5 trillion tokens, including diverse data like source code and synthetic data. This extensive training enhances their ability in code generation and reasoning.

Guide: Running Locally

1. Install Requirements:

   • Use the latest Hugging Face transformers library. Versions below 4.37.0 might result in errors.

2. Load Model and Tokenizer:

   from transformers import AutoModelForCausalLM, AutoTokenizer
   
   model_name = "Qwen/Qwen2.5-Coder-14B-Instruct"
   
   model = AutoModelForCausalLM.from_pretrained(
       model_name,
       torch_dtype="auto",
       device_map="auto"
   )
   tokenizer = AutoTokenizer.from_pretrained(model_name)
   

3. Generate Text:

   • Create a prompt and generate responses using the model.
   • Utilize cloud GPUs for efficient processing, such as AWS EC2 instances with NVIDIA GPUs or Google Cloud's GPU offerings.

4. Long Text Processing:

   • For inputs over 32,768 tokens, use YaRN with specific configuration adjustments for optimal performance.

License

The Qwen2.5-Coder model is licensed under the Apache-2.0 License. Details can be found here.