distilbert imdb

Introduction

DistilBERT-IMDB is a fine-tuned version of the distilbert-base-uncased model, tailored for text classification tasks on the IMDB dataset. It achieves an accuracy of 92.8% on the evaluation set, making it suitable for sentiment analysis tasks.

Architecture

The model is based on the DistilBERT architecture, a smaller and faster variant of the BERT model, designed to retain its performance while reducing resource consumption.

Training

The model was fine-tuned on the IMDB dataset with the following hyperparameters:

• Learning Rate: 5e-05
• Train Batch Size: 16
• Eval Batch Size: 16
• Seed: 42
• Optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• Learning Rate Scheduler Type: Linear
• Number of Epochs: 1

The training utilized the following framework versions:

• Transformers: 4.15.0
• PyTorch: 1.10.0+cu111
• Datasets: 1.17.0
• Tokenizers: 0.10.3

Guide: Running Locally

To run the DistilBERT-IMDB model locally, follow these steps:

1. Clone the Repository:

   git clone https://huggingface.co/lvwerra/distilbert-imdb
   cd distilbert-imdb
   

2. Install Dependencies: Ensure you have Python and pip installed. Then, run:

   pip install -r requirements.txt
   

3. Run the Model: Use the provided scripts in the repository to perform inference or further training.

4. Cloud GPUs: For optimal performance, especially during training, it is recommended to use cloud GPUs such as those provided by AWS, GCP, or Azure.

License

This model is licensed under the Apache 2.0 License.