Finetuned-RoBERTa-Depression

Introduction

This model is a fine-tuned version of the roberta-base model, designed for text classification tasks related to detecting depression. It was trained on an unspecified dataset and achieves a loss of 0.1385 and an accuracy of 0.9745 on the evaluation set.

Architecture

The model is based on the roberta-base architecture, which is a variant of the BERT model designed to improve upon the language understanding capabilities of BERT by training on longer sequences of text. This version has been fine-tuned specifically for identifying expressions of depression in text.

Training

Training Procedure

The model was trained using the following hyperparameters:

• Learning Rate: 5e-05
• Train Batch Size: 8
• Eval Batch Size: 8
• Seed: 42
• Optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• Learning Rate Scheduler Type: Linear
• Number of Epochs: 3.0

Training Results

The training results are as follows:

• Epoch 1: Training Loss = 0.0238, Validation Loss = 0.1385, Accuracy = 0.9745
• Epoch 2: Training Loss = 0.0333, Validation Loss = 0.1385, Accuracy = 0.9745
• Epoch 3: Training Loss = 0.0263, Validation Loss = 0.1385, Accuracy = 0.9745

Framework Versions

• Transformers: 4.17.0
• PyTorch: 1.10.0+cu111
• Datasets: 2.0.0
• Tokenizers: 0.11.6

Guide: Running Locally

To run this model locally, follow these steps:

1. Install Dependencies:

   • Install PyTorch, Transformers, and other dependencies.

   pip install torch transformers datasets
   

2. Load the Model:

   • Use the Hugging Face transformers library to load the model.

   from transformers import RobertaForSequenceClassification, RobertaTokenizer
   
   model = RobertaForSequenceClassification.from_pretrained("ShreyaR/finetuned-roberta-depression")
   tokenizer = RobertaTokenizer.from_pretrained("roberta-base")
   

3. Inference:

   • Tokenize your input text and perform inference.

   inputs = tokenizer("Your input text here", return_tensors="pt")
   outputs = model(**inputs)
   

4. Cloud GPUs:

   • For faster and more efficient processing, consider using cloud services such as AWS, GCP, or Azure to access GPUs.

License

This model is distributed under the MIT License, allowing for free use, modification, and distribution with proper attribution.