orca_mini_v9_7_3 B Instruct

Introduction

Orca Mini v9.7 Llama-3.2-3B-Instruct is a text generation model built on the Llama-3.2-3B-Instruct architecture. It is designed for a wide range of applications, allowing for further fine-tuning and customization to meet specific user needs.

Architecture

The model is based on the meta-llama/Llama-3.2-3B-Instruct architecture, which supports English language processing. It utilizes the Transformers library and offers compatibility with Safetensors for secure model handling.

Training

Orca Mini v9.7 is trained using various SFT datasets, including pankajmathur/orca_mini_v1_dataset and pankajmathur/orca_mini_v8_sharegpt_format. The training process emphasizes safety fine-tuning, data quality control, and robustness to adversarial prompts.

Guide: Running Locally

To run the model locally, follow these steps:

1. Install the necessary libraries:

   pip install torch transformers bitsandbytes
   

2. Use the following Python code to load and run the model:

   import torch
   from transformers import pipeline
   
   model_slug = "pankajmathur/orca_mini_v9_7_3B-Instruct"
   pipeline = pipeline(
       "text-generation",
       model=model_slug,
       device_map="auto",
   )
   messages = [
       {"role": "system", "content": "You are Orca Mini, a helpful AI assistant."},
       {"role": "user", "content": "Hello Orca Mini, what can you do for me?"}
   ]
   outputs = pipeline(messages, max_new_tokens=128, do_sample=True, temperature=0.01, top_k=100, top_p=0.95)
   print(outputs[0]["generated_text"][-1])
   

3. For cloud GPU support, use platforms like Google Colab with a T4 GPU for free access and execution.

License

The model is provided under the llama3.2 license. Users are encouraged to attribute use properly and can customize the model for various applications, including full fine-tuning and other optimizations.