Med Image Insights LLM Model

Introduction

MedImageInsight is an open-source medical imaging embedding model designed for tasks such as zero-shot classification, image embedding, and text embedding. It simplifies the usage of the MedImageInsight model initially presented in the paper "MedImageInsight: An Open-Source Embedding Model for General Domain Medical Imaging" by Noel C. F. Codella et al. This repository offers a standalone implementation, removing unnecessary files and code to enhance accessibility.

Architecture

The MedImageInsight model combines a vision model and a language model to perform various tasks related to medical imaging. This repository provides necessary modifications, including dependency management through uv, and supports multi-label classification. Additionally, an example FastAPI service is included to demonstrate practical applications.

Training

The repository does not include specific details on training the model as it focuses on providing a streamlined implementation of the pre-trained MedImageInsight model. Users can leverage the pre-trained models for tasks like zero-shot classification and embeddings without additional training.

Guide: Running Locally

Clone the Repository: Ensure git-lfs is installed and clone the repository.

git lfs install
git clone https://huggingface.co/lion-ai/MedImageInsights

Install Dependencies: Use the uv package manager to handle dependencies.
- To create a virtual environment and sync dependencies:
```
uv sync
```
- To run a specific script:
```
uv run example.py
```
Running Examples: The example.py script provides usage examples, including zero-shot classification and embedding tasks.
FastAPI Server: Start the FastAPI server for API access.
```
uv run fastapi_app.py
```
Access the Swagger documentation at localhost:8000/docs.

Cloud GPUs: For performance improvements, consider using cloud GPU services such as AWS EC2, Google Cloud Platform, or Azure.

License

This project is licensed under the MIT License, allowing open-source use and distribution with minimal restrictions.

More Related APIs