Qwen2.5 7 B Instruct abliterated v2

Introduction

The QWEN2.5-7B-INSTRUCT-ABLITERATED-V2 is an uncensored version of the Qwen2.5-7B-Instruct model, created using a method called abliteration. This model is optimized for conversational AI applications, offering improved performance over its predecessor.

Architecture

The model is based on the Qwen/Qwen2.5-7B-Instruct architecture, designed for text generation tasks. It leverages the transformers library and supports conversational contexts, making it suitable for chat applications. This version incorporates techniques to allow uncensored responses.

Training

The model has been refined through a process of abliteration, a methodology designed to enhance its capabilities in generating uncensored conversational text. More details on this technique can be found in the related article.

Guide: Running Locally

To run the model locally:

1. Install Dependencies: Ensure you have Python installed, along with the Hugging Face transformers library.

   pip install transformers
   

2. Load the Model and Tokenizer:

   from transformers import AutoModelForCausalLM, AutoTokenizer
   
   model_name = "huihui-ai/Qwen2.5-7B-Instruct-abliterated-v2"
   model = AutoModelForCausalLM.from_pretrained(
       model_name,
       torch_dtype="auto",
       device_map="auto"
   )
   tokenizer = AutoTokenizer.from_pretrained(model_name)
   

3. Initialize and Run the Chat:

   initial_messages = [
       {"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."}
   ]
   

4. Cloud GPU Recommendation: For optimal performance, especially with large models like Qwen2.5-7B, consider using cloud services that offer GPU support such as AWS EC2, Google Cloud, or Azure.

License

The model is licensed under the Apache 2.0 License. This allows for both personal and commercial use, with appropriate credit given to the original creators.