Llama 2 13b

Introduction

Llama 2 is a suite of large language models developed by Meta, featuring pretrained and fine-tuned generative text models ranging from 7 billion to 70 billion parameters. These models are designed for diverse text generation tasks, including dialogue systems. They are available for commercial and research use, subject to Meta's licensing terms.

Architecture

Llama 2 employs an auto-regressive language model with an optimized transformer architecture. The models use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety. Model variants include sizes of 7B, 13B, and 70B parameters, with some utilizing Grouped-Query Attention (GQA) for improved inference scalability.

Training

Llama 2 models were pretrained on 2 trillion tokens from publicly available data sources. Fine-tuning involved publicly available instruction datasets and over a million human-annotated examples. Training utilized Meta’s Research Super Cluster and third-party cloud computing, consuming a cumulative 3.3M GPU hours. Emissions from training were offset by Meta’s sustainability program.

Guide: Running Locally

1. Environment Setup: Ensure you have Python and PyTorch installed.
2. Get the Model: Visit Meta's Llama Downloads and accept the license agreement to download model weights and tokenizer.
3. Install Dependencies: Install necessary libraries using pip install -r requirements.txt.
4. Run Inference: Load the model using the PyTorch library and run text generation scripts.
5. Suggested Resources: Use cloud GPUs such as NVIDIA A100 available on platforms like AWS, Azure, or Google Cloud for efficient computation.

License

Llama 2 is distributed under the LLAMA 2 Community License by Meta. It provides a non-exclusive, worldwide, non-transferable, royalty-free limited license for using, reproducing, and modifying the Llama Materials. Redistribution requires adhering to certain terms, including providing a copy of the license to third parties and retaining attribution notices. Usage must comply with applicable laws and Meta's Acceptable Use Policy. Commercial use for entities with over 700 million monthly active users requires an additional license from Meta.