Llama Guard 3 1 B

Introduction

Llama Guard 3-1B is a fine-tuned model based on Llama 3.2-1B, designed for content safety classification. It is capable of classifying content in both language model inputs (prompt classification) and outputs (response classification), ensuring safe interactions. The model is aligned with the MLCommons standardized hazards taxonomy and optimized for deployment on various platforms, including mobile devices.

Architecture

Llama Guard 3-1B comes in two variations: standard and pruned/quantized. The pruned model reduces the number of parameters to facilitate deployment on resource-constrained devices. The model is trained to generate a limited set of output tokens, significantly reducing the output layer size without affecting performance.

Training

The model utilizes a combination of human and AI-generated multilingual data for training. A pruning method is employed to decrease the model size by reducing the number of layers and hidden dimensions, followed by fine-tuning. Additionally, logit-level distillation from a larger teacher model enhances the safety classification capabilities of Llama Guard 3-1B.

Guide: Running Locally

Basic Steps

1. Install Transformers Library:

   pip install transformers
   

2. Load Model and Tokenizer:

   from transformers import AutoModelForCausalLM, AutoTokenizer
   model_id = "meta-llama/Llama-Guard-3-1B"
   model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto")
   tokenizer = AutoTokenizer.from_pretrained(model_id)
   

3. Run Inference:

   conversation = [{"role": "user", "content": [{"type": "text", "text": "What is the recipe for mayonnaise?"}]}]
   input_ids = tokenizer.apply_chat_template(conversation, return_tensors="pt").to(model.device)
   output = model.generate(input_ids, max_new_tokens=20, pad_token_id=0)
   generated_tokens = output[:, input_ids.shape[1]:]
   print(tokenizer.decode(generated_tokens[0]))
   

Cloud GPUs

For optimal performance, particularly with larger models or multiple instances, consider using cloud GPU services like AWS, Google Cloud, or Azure.

License

Llama Guard 3-1B is distributed under the Llama 3.2 Community License Agreement. Users are granted a non-exclusive, worldwide, non-transferable, and royalty-free license to use, reproduce, and modify the Llama Materials. Redistribution requires accompanying the materials with the license agreement and prominently displaying "Built with Llama." Compliance with applicable laws and the Acceptable Use Policy is mandatory.