m B E R T base Biomedical N E R

Introduction

The BERT-BASE-MULTILINGUAL-CASED-FINETUNED-NER-4 model is a fine-tuned version of bert-base-multilingual-cased, designed for multilingual Biomedical Named Entity Recognition (NER) tasks. It combines data from the CRAFT, BC4CHEMD, and BioNLP09 datasets. The model demonstrates high performance with precision, recall, F1 score, and accuracy all around 98%.

Architecture

This model is based on the BERT architecture, specifically the bert-base-multilingual-cased variant. It is structured to handle token classification tasks and has been adapted to handle biomedical data through fine-tuning.

Training

The model was trained using the following hyperparameters:

• Learning Rate: 3e-05
• Training Batch Size: 8
• Evaluation Batch Size: 8
• Seed: 42
• Optimizer: Adam (betas=(0.9,0.999), epsilon=1e-08)
• Learning Rate Scheduler Type: Linear
• Number of Epochs: 4

The training procedure involved a linear learning rate scheduler and an Adam optimizer. The model achieved notable results with a training loss of 0.1027 and a validation loss of 0.0751.

Guide: Running Locally

1. Install Dependencies: Ensure you have the necessary libraries installed, including Transformers, PyTorch, Datasets, and Tokenizers.
2. Clone the Repository: Use Git to clone the model repository to your local machine.
3. Load the Model: Utilize the Transformers library to load the pre-trained model and tokenizer.
4. Prepare Data: Format your dataset according to the input requirements of the model.
5. Run Inference: Use the model to perform token classification on your data.

Cloud GPU Suggestion

Consider using cloud-based GPUs from providers such as AWS, Google Cloud, or Azure to accelerate training and inference, especially for large datasets or complex models.

License

The model is released under the Apache 2.0 License. This permits usage, modification, and distribution of the model, provided that the same license terms are included with any distribution.