Llama 3.2 1 B Instruct Spin Quant_ I N T4_ E O8

Introduction

Llama 3.2 is a collection of multilingual large language models (LLMs) developed by Meta. These models are pretrained and instruction-tuned for text generation tasks, optimized for multilingual dialogue, agentic retrieval, and summarization. They are designed to outperform many existing chat models on industry benchmarks and support eight languages: English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.

Architecture

Llama 3.2 utilizes an auto-regressive language model with an optimized transformer architecture. Supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) are employed to align the models with human preferences. The models are available in 1B and 3B parameter sizes, using a mix of publicly available online data for training.

Training

Llama 3.2 was pretrained on up to 9 trillion tokens of data and uses a combination of logits from larger Llama models for pretraining. The training process included supervised fine-tuning, rejection sampling, and direct preference optimization. The training infrastructure involved custom GPU clusters, utilizing 916k GPU hours and achieving net-zero greenhouse gas emissions.

Guide: Running Locally

To run Llama 3.2 models locally:

1. Clone the Repository: Get the model code from Meta's GitHub repository.
2. Set Up Environment: Install necessary dependencies such as PyTorch.
3. Download Model Weights: Obtain the model weights from the Hugging Face model hub.
4. Load and Test Model: Use the provided scripts or APIs to load the model and perform inference.

Cloud GPUs

For optimal performance, consider using cloud-based GPUs like NVIDIA's A100 or H100, available through services like AWS, Google Cloud, or Azure.

License

Llama 3.2 is distributed under the Llama 3.2 Community License. This license grants a non-exclusive, worldwide, non-transferable, and royalty-free limited license to use, reproduce, and modify the Llama Materials. Redistribution requires including the license agreement and "Built with Llama" attribution. Commercial use is subject to additional terms if monthly active users exceed 700 million. The license disclaims warranties and limits liability, with Meta retaining ownership of the Llama Materials and derivatives.