Code Llama 7b Instruct hf

Introduction

Code Llama is a collection of pretrained and fine-tuned generative text models developed by Meta, designed for code synthesis and understanding. The models are available in various sizes and specializations, including base, Python-specific, and instruction-following variants.

Architecture

Code Llama employs an auto-regressive language model that utilizes an optimized transformer architecture. The models are available in sizes of 7B, 13B, and 34B parameters, designed for general code tasks, Python-specific tasks, and safer instruction-following deployment.

Training

The training and fine-tuning of Code Llama models were conducted using Meta’s Research Super Cluster. The training involved custom libraries and entailed 400K GPU hours on A100-80GB hardware, with an estimated carbon footprint of 65.3 tCO2eq, fully offset by Meta's sustainability efforts.

Guide: Running Locally

1. Install Dependencies: Ensure you have Python installed. Use the following command to install the necessary libraries:

   pip install transformers accelerate
   

2. Model Loading: Use the Hugging Face Transformers library to load the model:

   from transformers import AutoModelForCausalLM, AutoTokenizer
   
   model_name = "meta-llama/CodeLlama-7b-Instruct-hf"
   model = AutoModelForCausalLM.from_pretrained(model_name)
   tokenizer = AutoTokenizer.from_pretrained(model_name)
   

3. Inference: Use the model for text generation by encoding input text and decoding the output:

   inputs = tokenizer("Your code prompt here", return_tensors="pt")
   outputs = model.generate(**inputs)
   print(tokenizer.decode(outputs[0]))
   

Cloud GPUs: For more efficient performance and handling of larger models, consider using cloud-based GPU services like AWS, Google Cloud, or Azure.

License

Code Llama is available under the LLAMA 2 Community License Agreement. Users must agree to the terms, which include non-commercial use and compliance with an Acceptable Use Policy. The license includes provisions for redistribution, modification, and compliance with legal standards. For commercial use above certain user thresholds, additional licensing from Meta may be required. More details can be found at Meta's Llama Downloads.