matscibert

Introduction

MatSciBERT is a pretrained BERT model specifically designed for text mining and information extraction in the materials science domain. It leverages research papers related to materials such as alloys, glasses, metallic glasses, cement, and concrete.

Architecture

MatSciBERT is based on the BERT architecture and is trained using a corpus from ScienceDirect. The model focuses on handling materials science texts, utilizing both abstracts and full texts when available.

Training

The model is trained on a dataset of materials science research papers accessed via the Elsevier API. The training involves pretraining on the specific corpus followed by finetuning for various downstream tasks. Detailed methodologies and codes for pretraining and finetuning are available on the GitHub repository.

Guide: Running Locally

1. Clone Repository: Download the codes from the GitHub repository.
2. Environment Setup: Ensure you have Python and PyTorch installed. Set up a virtual environment if necessary.
3. Install Dependencies: Use pip install -r requirements.txt to install the necessary libraries.
4. Run Model: Load the model using the Transformers library and execute your tasks.
5. Consider Cloud GPUs: For large-scale experiments, consider using cloud GPU services such as AWS, Google Cloud, or Azure for efficient computation.

License

MatSciBERT is distributed under the MIT License, allowing for freedom in use, modification, and distribution.