E X A O N E 3.5 2.4 B Instruct abliterated G G U F I Q A R M Imatrix Community

Introduction

The EXAONE-3.5-2.4B-Instruct-abliterated model is an uncensored version of the LGAI-EXAONE/EXAONE-3.5-2.4B-Instruct, developed to eliminate refusal responses from language models. This approach, termed "abliteration," is implemented without using TransformerLens.

Architecture

The model is built on the GGUF framework and optimized for conversational tasks. It supports the English language and is part of the EXAONE series, which utilizes a base model from huihui-ai.

Training

This version of the model has undergone a process called "abliteration," which removes refusals to certain instructions. This process is a proof-of-concept aimed at enhancing the responsiveness of language models in conversational applications.

Guide: Running Locally

1. Clone the Repository:
   Download the model from the Hugging Face Hub.

2. Set Up Environment:
   Ensure you have Python and necessary libraries installed, such as transformers and torch.

3. Load the Model:

   from transformers import AutoModel, AutoTokenizer
   
   tokenizer = AutoTokenizer.from_pretrained("huihui-ai/EXAONE-3.5-2.4B-Instruct-abliterated")
   model = AutoModel.from_pretrained("huihui-ai/EXAONE-3.5-2.4B-Instruct-abliterated")
   

4. Inference: Run the model locally on your data. For best performance, consider using a cloud GPU service like AWS, Google Cloud, or Azure.

License

The model is released under the "Exaone" license. Specific terms and conditions can be found in the LICENSE file provided in the repository.