B E R T tiny emotion intent LLM Model

Introduction

BERT-TINY-EMOTION-INTENT is a fine-tuned version of Google's BERT model, specifically google/bert_uncased_L-2_H-128_A-2, on an emotion dataset. The model is designed for text classification tasks, achieving an accuracy of 0.91 on the evaluation set with a loss of 0.3620.

Architecture

The model is based on the BERT architecture, known for its transformer-based approach to natural language processing tasks. It uses a compact version of BERT, with reduced layers and hidden states, suitable for scenarios requiring lower computational resources.

Training

The model was trained using the following hyperparameters:

Learning Rate: 5e-05
Train Batch Size: 16
Eval Batch Size: 16
Seed: 33
Optimizer: Adam with betas (0.9, 0.999) and epsilon 1e-08
LR Scheduler Type: Linear
Number of Epochs: 50
Mixed Precision Training: Native AMP

The training results showed a consistent improvement in accuracy, reaching 0.91 by the 12th epoch with a validation loss of 0.3620.

Guide: Running Locally

To run the BERT-TINY-EMOTION-INTENT model locally, follow these basic steps:

Clone the Repository:

git clone https://huggingface.co/gokuls/BERT-tiny-emotion-intent
cd BERT-tiny-emotion-intent

Install Dependencies: Ensure you have Python installed, and then run:
```
pip install transformers torch datasets
```

Load the Model: Use the Hugging Face Transformers library to load the model:

from transformers import BertForSequenceClassification, BertTokenizer
model = BertForSequenceClassification.from_pretrained("gokuls/BERT-tiny-emotion-intent")
tokenizer = BertTokenizer.from_pretrained("gokuls/BERT-tiny-emotion-intent")

Inference: Prepare your text input and perform inference:

inputs = tokenizer("Your text here", return_tensors="pt")
outputs = model(**inputs)

For improved performance, consider using cloud GPU services, such as AWS EC2 with GPU instances, Google Cloud Platform, or Microsoft Azure.

License

The BERT-TINY-EMOTION-INTENT model is licensed under the Apache 2.0 License, allowing for broad usage and modification with proper attribution.

More Related APIs in Text Classification