Bio Medical Llama 3 8 B

Introduction

The Bio-Medical-Llama-3-8B is a specialized large language model designed for applications in the biomedical field. It is fine-tuned from the Meta-Llama-3-8B-Instruct model using a custom dataset, "BioMedData," which includes over 500,000 synthetic and manually curated entries. The model is intended for use in research, clinical decision support, and education, providing comprehensive coverage of biomedical topics.

Architecture

• Model Name: Bio-Medical-Llama-3-8B
• Base Model: Llama-3-8B-Instruct
• Parameter Count: 8 billion
• Training Data: High-quality biomedical dataset with over 500,000 entries, comprising synthetic and curated samples.

Training

• Learning Rate: 0.0002
• Train Batch Size: 12
• Eval Batch Size: 8
• Seed: 42
• Gradient Accumulation Steps: 4
• Total Train Batch Size: 32
• Optimizer: Adam with betas (0.9, 0.999) and epsilon 1e-08
• Scheduler Type: Cosine with a warmup ratio of 0.03
• Training Steps: 2000
• Mixed Precision Training: Native AMP
• Framework Versions:
  • PEFT 0.11.0
  • Transformers 4.40.2
  • Pytorch 2.1.2
  • Datasets 2.19.1
  • Tokenizers 0.19.1

Guide: Running Locally

To run the Bio-Medical-Llama-3-8B model locally, follow these steps:

1. Installation: Ensure you have Python installed and set up a virtual environment. Install necessary libraries:

   pip install torch transformers
   

2. Model Setup:

   import transformers
   import torch
   
   model_id = "ContactDoctor/Bio-Medical-Llama-3-8B"
   pipeline = transformers.pipeline(
       "text-generation",
       model=model_id,
       model_kwargs={"torch_dtype": torch.bfloat16},
       device_map="auto",
   )
   

3. Inference: Create a prompt and generate text:

   messages = [
       {"role": "system", "content": "You are an expert trained on healthcare and biomedical domain!"},
       {"role": "user", "content": "I'm a 35-year-old male and for the past few months, I've been experiencing fatigue, increased sensitivity to cold, and dry, itchy skin. What is the diagnosis here?"},
   ]
   prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
   outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.6, top_p=0.9)
   print(outputs[0]["generated_text"][len(prompt):])
   

4. Resource Recommendations:

   • Utilize cloud GPUs such as AWS EC2 instances with NVIDIA GPUs, Google Cloud Platform, or Azure for efficient processing.

License

This model is licensed under the Bio-Medical-Llama-3-8B (Non-Commercial Use Only) license. Please review the terms and conditions before using the model. For more information, contact info@contactdoctor.in or visit ContactDoctor's website.