distilbert base uncased emotion

Introduction

The distilbert-base-uncased-emotion model is a text classification model based on DistilBERT, fine-tuned for emotion detection on a dataset derived from Twitter. DistilBERT is a smaller and faster version of BERT, retaining 97% of its language understanding capability while reducing its size by 40%.

Architecture

DistilBERT is trained using knowledge distillation, which involves transferring the knowledge from a larger BERT model to a smaller one during the pre-training phase. The distilbert-base-uncased-emotion model is specifically fine-tuned for classifying emotions in text.

Training

The model was fine-tuned on the emotion dataset using the Hugging Face Trainer with the following hyperparameters:

• Learning rate: 2e-5
• Batch size: 64
• Number of epochs: 8

Performance metrics on the emotion dataset include:

• Accuracy: 93.8%
• F1 Score: 93.79%
• Test samples per second: 398.69

Guide: Running Locally

To use the model locally, follow these steps:

1. Install Transformers Library:

   pip install transformers
   

2. Load and Use the Model:

   from transformers import pipeline
   classifier = pipeline("text-classification", model='bhadresh-savani/distilbert-base-uncased-emotion', return_all_scores=True)
   prediction = classifier("I love using transformers. The best part is wide range of support and its easy to use.")
   print(prediction)
   

   The output will display the emotion scores for the input text.

3. Hardware Recommendation:

   • For efficient performance, especially with large datasets, consider using cloud GPU services such as AWS, GCP, or Azure.

License

The model is licensed under the Apache 2.0 License.