Llama-2-7B-Chat Technical Documentation

Introduction

Llama 2 is a set of pretrained and fine-tuned generative text models developed by Meta, with parameters ranging from 7 billion to 70 billion. The 7B model is optimized for dialogue applications. These models excel in open-source chat benchmarks and are comparable to popular closed-source models in terms of helpfulness and safety.

Architecture

Llama 2 models are auto-regressive language models utilizing an optimized transformer architecture. They use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety.

Training

• Datasets: Trained on 2 trillion tokens of publicly available data, excluding Meta user data. Fine-tuning involved over one million human-annotated examples.
• Hardware: Training was performed using Meta's Research Super Cluster and other production clusters, with third-party cloud compute used for fine-tuning.
• Carbon Footprint: Training utilized 3.3M GPU hours with an estimated 539 tCO2eq emissions, fully offset by Meta's sustainability program.
• Model Variations: Available in sizes 7B, 13B, and 70B parameters, with models trained using a global batch-size of 4M tokens.

Guide: Running Locally

To run the Llama 2 model locally, follow these steps:

1. Environment Setup: Ensure you have Python and PyTorch installed. Install additional dependencies as required by the Llama 2 repository.
2. Access Model Weights: Visit the Llama 2 download page, accept the license, and request access to the model weights and tokenizer.
3. Load the Model: Use the provided scripts or APIs to load the model into your local environment.
4. Inference: Run inference using the model with your text input.

For enhanced performance, consider using cloud GPUs such as AWS EC2 with NVIDIA A100 instances.

License

Llama 2 is distributed under a custom commercial license. Users must agree to the LLAMA 2 Community License Agreement, which includes conditions on redistribution, usage limitations, and compliance with Meta's Acceptable Use Policy. More details and the full license text are available on the Meta AI website.