math roberta

Introduction

MathRoBERTa is a fine-tuned version of RoBERTa-large, specifically trained to handle NLP tasks within math learning environments. It is developed using data from Algebra Nation and aims to assist in tasks such as text classification, semantic search, and question answering.

Architecture

MathRoBERTa is based on the RoBERTa-large architecture, consisting of 24 layers and 355 million parameters. The model's weights require approximately 1.5 gigabytes of storage.

Training

The model was trained using 3,000,000 math discussion posts from students and facilitators on Algebra Nation. This training was conducted on 8 Nvidia RTX 1080Ti GPUs.

Guide: Running Locally

To use MathRoBERTa with your text inputs, follow these steps:

1. Install the transformers library if you haven't already:

   pip install transformers
   

2. Use the following code to load and utilize the model:

   from transformers import RobertaTokenizer, RobertaModel
   
   tokenizer = RobertaTokenizer.from_pretrained('uf-aice-lab/math-roberta')
   model = RobertaModel.from_pretrained('uf-aice-lab/math-roberta')
   
   text = "Replace me by any text you'd like."
   encoded_input = tokenizer(text, return_tensors='pt')
   output = model(**encoded_input)
   

3. Consider using cloud GPUs for more efficient processing, such as those offered by Google Cloud, AWS, or Azure.

License

The MathRoBERTa model is released under the MIT License, allowing for both personal and commercial use.