Llama 2 70b hf

Introduction

Llama 2 is a series of large language models (LLMs) developed by Meta, with scales ranging from 7 billion to 70 billion parameters. These models are designed for various natural language processing tasks, including text generation, and are available as both pretrained and fine-tuned versions. Llama-2-Chat models, optimized for dialogue, compare favorably with popular chat models like ChatGPT.

Architecture

Llama 2 utilizes an auto-regressive transformer architecture. The fine-tuned models employ supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to better align with human preferences for helpfulness and safety. The models generate text based on input text, and the larger models (70B) incorporate Grouped-Query Attention (GQA) for improved inference scalability.

Training

Llama 2 models were trained on 2 trillion tokens from publicly available sources, with fine-tuning using a mix of public instruction datasets and human-annotated examples. The training process occurred on Meta's infrastructure, including the Research Super Cluster and third-party cloud compute. The carbon footprint of the training was offset by Meta’s sustainability initiatives, with training requiring millions of GPU hours.

Guide: Running Locally

To run Llama 2 models locally, follow these steps:

1. Set Up Environment: Ensure you have Python and PyTorch installed. Use a virtual environment for better dependency management.
2. Install Transformers Library: Run pip install transformers to get the necessary library.
3. Download Model: Access the model via Hugging Face Hub and accept the Meta license terms to download the model weights.
4. Load Model: Use the Hugging Face Transformers library to load the model and tokenizer.
5. Run Inference: Implement a script to input text and generate responses using the model.

Due to substantial computational requirements, utilizing cloud GPUs from providers like AWS, Google Cloud, or Azure is recommended for efficient performance, especially for larger models.

License

Llama 2 is distributed under the LLAMA 2 Community License by Meta. This license allows for non-exclusive, worldwide use, reproduction, and modification of the Llama Materials, subject to specific terms and conditions. Redistribution requires providing a copy of the license to third parties, and use must comply with applicable laws and Meta’s Acceptable Use Policy.