rubert tiny2 cedr emotion detection

Introduction

The rubert-tiny2-cedr-emotion-detection model is a fine-tuned version of the cointegrated/rubert-tiny2 model, designed for detecting emotions within Russian sentences. The model performs multilabel classification, enabling it to identify multiple emotions within a single sentence.

Architecture

The model utilizes a fine-tuned version of the RuBERT-Tiny2 architecture. It is trained on the CEDR dataset, which focuses on emotion detection in Russian texts as described in the paper "Data-Driven Model for Emotion Detection in Russian Texts" by Sboev et al.

Training

Training was conducted using the Adam optimizer over 40 epochs. The learning rate was set at 1e-5, and a batch size of 64 was used. The training process is documented in a Google Colab notebook. Performance metrics on the test dataset include:

• AUC: 0.9286 for no emotion, 0.9512 for joy, and various other scores for sadness, surprise, fear, and anger.
• F1 Scores:
  • Micro: 0.8624 for no emotion, 0.9575 for fear, and other scores for the remaining emotions.
  • Macro: 0.8562 for no emotion, 0.9017 for sadness, and others.

Guide: Running Locally

To run the model locally, follow these steps:

1. Install Dependencies: Ensure you have Python and PyTorch installed. You can install the Hugging Face Transformers library using pip:

   pip install transformers
   

2. Download the Model: Use the Hugging Face model hub to download the rubert-tiny2-cedr-emotion-detection model.
3. Load the Model: In your Python script, load the model using the Transformers library:

   from transformers import AutoModelForSequenceClassification, AutoTokenizer
   
   tokenizer = AutoTokenizer.from_pretrained("cointegrated/rubert-tiny2-cedr-emotion-detection")
   model = AutoModelForSequenceClassification.from_pretrained("cointegrated/rubert-tiny2-cedr-emotion-detection")
   

4. Run Inference: Prepare your input text and run it through the model to get predictions.

For enhanced performance, particularly with large datasets or batch processing, consider using cloud GPU services like AWS or Google Cloud.

License

The model and the CEDR dataset usage are subject to the applicable licenses. Users should consult the Hugging Face model page and dataset documentation for specific licensing details.