Emo Ro B E R Ta

Introduction

EmoRoBERTa is a text classification model developed to identify emotions in text. It leverages the RoBERTa architecture and is trained on the GoEmotions dataset, which includes 58,000 Reddit comments labeled with 28 distinct emotions.

Architecture

RoBERTa is an advanced transformer model that builds on BERT's architecture by modifying key hyperparameters. It removes BERT’s next-sentence pretraining objective, uses larger mini-batches and learning rates, and is trained on significantly more data. This enhances RoBERTa's ability to generalize to downstream tasks.

Training

The model was trained using the following hyperparameters:

• Learning Rate: 5e-5
• Epochs: 10
• Max Sequence Length: 50
• Batch Size: 16
• Warmup Proportion: 0.1
• Epsilon: 1e-8

These settings led to a best Macro F1 score of 49.30%.

Guide: Running Locally

To use EmoRoBERTa locally, follow these steps:

1. Install the Transformers library:

   pip install transformers
   

2. Load the tokenizer and model:

   from transformers import RobertaTokenizerFast, TFRobertaForSequenceClassification, pipeline
   
   tokenizer = RobertaTokenizerFast.from_pretrained("arpanghoshal/EmoRoBERTa")
   model = TFRobertaForSequenceClassification.from_pretrained("arpanghoshal/EmoRoBERTa")
   

3. Perform sentiment analysis:

   emotion = pipeline('sentiment-analysis', model='arpanghoshal/EmoRoBERTa')
   emotion_labels = emotion("Thanks for using it.")
   print(emotion_labels)
   

   Output:

   [{'label': 'gratitude', 'score': 0.9964383244514465}]
   

4. Recommendation: For better performance, consider using cloud GPUs from services like AWS, Google Cloud, or Azure.

License

EmoRoBERTa is released under the MIT License, allowing for flexibility in usage and modification.