Biomed Parse

Introduction

BiomedParse is a foundation model designed for joint segmentation, detection, and recognition of biomedical objects across nine imaging modalities. It enhances accuracy for individual tasks and enables applications such as segmenting all relevant objects in an image using text prompts. This model is broadly applicable in the biomedical field, particularly in cell biology, pathology, and radiology.

Architecture

BiomedParse employs a transformer-based architecture optimized for large biomedical corpora. Utilizing multi-head attention mechanisms, it efficiently identifies and understands biomedical terminology, extracting contextually relevant information. Pre-trained on extensive datasets, it generalizes effectively across various biomedical domains.

Training

Training details are not explicitly provided in the document, but the model is pre-trained on vast biomedical datasets to ensure high accuracy and generalizability across different biomedical imaging modalities.

Guide: Running Locally

1. Clone the Repository:

   git clone https://github.com/microsoft/BiomedParse.git
   cd BiomedParse
   

2. Create and Activate a Conda Environment:

   conda create -n biomedparse python=3.9.19
   conda activate biomedparse
   

3. Install PyTorch:

   conda install pytorch torchvision torchaudio pytorch-cuda=12.4 -c pytorch -c nvidia
   

4. Install Dependencies:

   pip install -r assets/requirements/requirements.txt
   

5. Model Setup:

   from modeling.BaseModel import BaseModel
   from modeling import build_model
   from utilities.distributed import init_distributed
   from utilities.arguments import load_opt_from_config_files
   
   opt = load_opt_from_config_files(["configs/biomedparse_inference.yaml"])
   opt = init_distributed(opt)
   pretrained_pth = 'hf_hub:microsoft/BiomedParse'
   model = BaseModel(opt, build_model(opt)).from_pretrained(pretrained_pth).eval().cuda()
   

6. Example Segmentation:

   from PIL import Image
   image = Image.open('examples/Part_1_516_pathology_breast.png').convert('RGB')
   prompts = ['neoplastic cells', 'inflammatory cells']
   pred_mask = interactive_infer_image(model, image, prompts)
   

Cloud GPUs

For improved performance, consider using cloud GPU services like AWS, Azure, or Google Cloud.

License

The BiomedParse model is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (cc-by-nc-sa-4.0). It is intended for research and development use only and not for clinical decision-making or other clinical uses. Users bear sole responsibility for any use, including incorporation into any product intended for clinical use.