Multi Label Classification of Pub Med Articles

Introduction

The project focuses on multi-label classification of PubMed articles using transfer learning techniques. By leveraging the pre-existing BioBERT model, the project fine-tunes it on the PubMed MultiLabel Text Classification Dataset. The project also explores the RoBERTa and XLNet models for sequence classification. Weight and Bias are integrated for model visualization and logging.

Architecture

The architecture involves using the BioBERT model, a variant of BERT optimized for biomedical text, for transfer learning. The model is fine-tuned on a dataset specifically curated for multi-label classification tasks in the medical domain. The project also utilizes variations such as RoBERTaForSequenceClassification and XLNetForSequenceClassification.

Training

Training involves fine-tuning the BioBERT model on the PubMed MultiLabel dataset. This approach leverages transfer learning to adapt the model to classify multiple labels per PubMed article. The project utilizes the BCEWithLogitsLoss function from PyTorch for handling multi-label classification tasks.

Guide: Running Locally

1. Clone the Repository:
   Clone the repository containing the code and dataset to your local machine.

2. Install Dependencies:
   Ensure that PyTorch and the Hugging Face Transformers library are installed. Additional dependencies might include the Weights & Biases library for tracking experiments.

3. Prepare Dataset:
   Download and preprocess the PubMed MultiLabel Text Classification Dataset as outlined in the Kaggle notebook.

4. Run the Model:
   Execute the training script to fine-tune the BioBERT model on your dataset. Use the provided Kaggle notebook for detailed steps.

5. Utilize Cloud GPUs:
   For efficient training, consider using cloud GPU services such as AWS, GCP, or Azure to handle the computational load.

License

The project does not specify a license, so it is advisable to contact the author for terms of use or assume a standard open-source license until otherwise stated.