Llama 2 G G M L Medical Chatbot LLM Model

Introduction

The Llama-2-GGML-Medical-Chatbot is an advanced AI-driven medical chatbot designed for answering medical questions, providing summaries, and generating medical text using the Llama-2-7B-Chat-GGML model. It is based on the Gale Encyclopedia of Medicine and is suitable for patients, healthcare professionals, and researchers. However, it should not be used as a substitute for professional medical advice.

Architecture

The chatbot leverages the Llama-2-7B-Chat-GGML large language model, which is fine-tuned and quantized with the q2_K method. The model file is named llama-2-7b-chat.ggmlv3.q2_K.bin, with a size of 2.87 GB and requiring a maximum of 5.37 GB RAM. It uses different tensor types such as GGML_TYPE_Q4_K and GGML_TYPE_Q2_K for various operations. The system is powered by Langchain and Chainlit, and operates on a minimum hardware setup of an Intel® Core™ i3 CPU with 8 GB RAM.

Training

The chatbot is trained using the Gale Encyclopedia of Medicine, Volume 1, 2nd Edition, which is a comprehensive 637-page PDF. This training set allows the chatbot to cover a wide range of medical topics.

Guide: Running Locally

Open Git Bash.
Navigate to the desired directory where you want to clone the repository.

Clone the repository by running:

git clone https://github.com/ThisIs-Developer/Llama-2-GGML-Medical-Chatbot.git

Install required packages by executing:
```
pip install -r requirements.txt
```
Run the chatbot using:
```
chainlit run model.py -w
```

For enhanced performance, consider using cloud GPUs from providers like AWS, GCP, or Azure.

License

The Llama-2-GGML-Medical-Chatbot is licensed under the MIT License.