Alchemist Coder D S 6.7 B

Introduction

AlchemistCoder is a series of open-source Code LLMs designed to enhance code generation and generalization capabilities. These models are fine-tuned on multi-source data, employing techniques like AlchemistPrompts and code comprehension tasks to improve instruction-following and code intelligence.

Architecture

AlchemistCoder models focus on harmonizing multiple sources of data through AlchemistPrompts, which address conflicts in style and quality. The models incorporate code comprehension tasks such as instruction evolution, data filtering, and code review to enhance their learning process. These techniques enable AlchemistCoder to outperform other models of similar size and rival larger models across various benchmarks.

Training

The training process for AlchemistCoder involves fine-tuning on 200 million tokens from six types of high-quality data. The model is trained to handle instruction-response pairs effectively, utilizing data-specific prompts to align different data sources.

Guide: Running Locally

To run AlchemistCoder locally, follow these steps:

1. Install Dependencies: Ensure Python and PyTorch are installed on your system.
2. Load the Model: Use the Hugging Face transformers library to load the AlchemistCoder model and tokenizer.

   import torch
   from transformers import AutoModelForCausalLM, AutoTokenizer
   
   tokenizer = AutoTokenizer.from_pretrained("internlm/AlchemistCoder-DS-6.7B", trust_remote_code=True)
   model = AutoModelForCausalLM.from_pretrained("internlm/AlchemistCoder-DS-6.7B", trust_remote_code=True, torch_dtype=torch.bfloat16).cuda()
   model = model.eval()
   

3. Run Inference: Pass input text to the model for code generation.

   input_text = "Implement the Dijkstra algorithm in Python"
   inputs = tokenizer(input_text, return_tensors="pt").to(model.device)
   outputs = model.generate(**inputs, max_length=128)
   print(tokenizer.decode(outputs[0], skip_special_tokens=True))
   

4. Cloud GPUs: It is recommended to use cloud services like AWS, Google Cloud, or Azure for GPU resources to handle the computational load efficiently.

License

AlchemistCoder is licensed under the Apache 2.0 License, which allows for broad use, modification, and distribution of the software.