Che Xagent 8b

Introduction

The CheXagent-8B is a model developed by Stanford AIMI for the interpretation of chest X-rays. It utilizes advanced text generation and transformer architectures to aid in medical imaging analysis. This project is available on Hugging Face and is accompanied by a comprehensive research paper.

Architecture

CheXagent-8B leverages the transformer architecture tailored for causal language modeling. It incorporates a processor for handling image and text inputs, enabling it to generate textual descriptions of medical images, specifically chest X-rays. The model is designed to be efficient, employing torch.float16 data type and intended for use on CUDA-enabled devices.

Training

Details on the training process are not explicitly provided in the available documentation. However, the model likely underwent extensive training on a large dataset of chest X-ray images and corresponding textual data to fine-tune its ability to interpret and describe medical images accurately.

Guide: Running Locally

1. Setup Environment: Ensure you have Python and necessary libraries installed (torch, transformers, requests, PIL).
2. Load Model and Processor:

   from transformers import AutoModelForCausalLM, AutoProcessor, GenerationConfig
   
   device = "cuda"
   dtype = torch.float16
   
   processor = AutoProcessor.from_pretrained("StanfordAIMI/CheXagent-8b", trust_remote_code=True)
   generation_config = GenerationConfig.from_pretrained("StanfordAIMI/CheXagent-8b")
   model = AutoModelForCausalLM.from_pretrained("StanfordAIMI/CheXagent-8b", torch_dtype=dtype, trust_remote_code=True)
   

3. Fetch Image: Use the requests library to download images for analysis.
4. Generate Findings:

   prompt = f'Describe "Airway"'
   inputs = processor(images=images, text=f" USER: <s>{prompt} ASSISTANT: <s>", return_tensors="pt").to(device=device, dtype=dtype)
   output = model.generate(**inputs, generation_config=generation_config)[0]
   response = processor.tokenizer.decode(output, skip_special_tokens=True)
   

5. Suggestion: For optimal performance, use a cloud GPU service like AWS, Google Cloud, or Azure.

License

The licensing information for the CheXagent-8B model is not explicitly mentioned in the provided details. For accurate information, refer to the model's page on Hugging Face or its associated GitHub repository.