Huatuo G P T o1 8 B G G U F

Introduction

HuatuoGPT-o1 is a medical language model (LLM) designed for advanced medical reasoning. It generates a comprehensive thought process by reflecting and refining its reasoning before delivering a final response. This model is particularly useful in medical contexts where complex reasoning is required.

Architecture

HuatuoGPT-o1 is built on the LLaMA-3.1 architecture, specifically the 8B variant for this quantized version. The model supports English and is derived from the FreedomIntelligence HuatuoGPT series, which includes models like HuatuoGPT-o1-70B and HuatuoGPT-o1-7B, supporting both English and Chinese.

Training

The HuatuoGPT-o1 model utilizes datasets such as FreedomIntelligence/medical-o1-reasoning-SFT and FreedomIntelligence/medical-o1-verifiable-problem to enhance its medical reasoning capabilities. The model follows a "thinks-before-it-answers" approach, producing outputs with a detailed reasoning process followed by a final response.

Guide: Running Locally

To run HuatuoGPT-o1 locally, you can use libraries like transformers from Hugging Face for model deployment. Here are the basic steps:

1. Install Required Libraries:

   pip install transformers
   

2. Load and Use the Model:

   from transformers import AutoModelForCausalLM, AutoTokenizer
   
   model = AutoModelForCausalLM.from_pretrained("FreedomIntelligence/HuatuoGPT-o1-8B", torch_dtype="auto", device_map="auto")
   tokenizer = AutoTokenizer.from_pretrained("FreedomIntelligence/HuatuoGPT-o1-8B")
   
   input_text = "How to stop a cough?"
   messages = [{"role": "user", "content": input_text}]
   
   inputs = tokenizer(tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True), return_tensors="pt").to(model.device)
   outputs = model.generate(**inputs, max_new_tokens=2048)
   print(tokenizer.decode(outputs[0], skip_special_tokens=True))
   

3. Cloud GPU Recommendation: For optimal performance, especially with large models like HuatuoGPT-o1-8B, consider using cloud GPUs from providers such as AWS, Google Cloud, or Azure.

License

HuatuoGPT-o1 is distributed under the Apache 2.0 license, which allows for both personal and commercial use, distribution, and modification, provided that the license terms are adhered to.