Llama 3.2 3 B Instruct abliterated

Introduction

LLAMA-3.2-3B-INSTRUCT-ABLITERATED is an uncensored version of the Llama 3.2 3B Instruct model, created using the technique of abliteration. Special thanks to @FailSpy for the original code and technique. This model is suitable for conversational and text-generation tasks.

Architecture

The model is based on the llama architecture and leverages the Transformers library. It has been enhanced with the abliteration process, focusing on producing uncensored content.

Training

The model was trained using the Llama-3.2-3B-Instruct as the base. It was evaluated using benchmarks like IF_Eval, MMLU Pro, TruthfulQA, BBH, and GPQA, showing slight improvements over the base model. The evaluation script is available in the repository for further analysis.

Guide: Running Locally

1. Download the Model

   huggingface-cli download huihui-ai/Llama-3.2-3B-Instruct-abliterated --local-dir ./huihui-ai/Llama-3.2-3B-Instruct-abliterated
   

2. Get the Reference Model

   ollama pull llama3.2
   

3. Export Model Parameters

   ollama show llama3.2 --modelfile > Modelfile
   

4. Modify the Modelfile
   Remove comment lines before the "FROM" keyword and replace it with:

   FROM huihui-ai/Llama-3.2-3B-Instruct-abliterated
   

5. Create Quantized Model

   ollama create --quantize q4_K_M -f Modelfile Llama-3.2-3B-Instruct-abliterated-q4_K_M
   

6. Run the Model

   ollama run Llama-3.2-3B-Instruct-abliterated-q4_K_M
   

Consider using cloud GPUs for efficient processing, such as those from AWS or Google Cloud.

License

This model is licensed under llama3.2.