distilbert base uncased finetuned imdb

Introduction

The distilbert-base-uncased-finetuned-imdb model is a fine-tuned version of distilbert-base-uncased on the IMDB dataset. This model is designed for sentiment analysis tasks and has been trained to classify movie reviews as positive or negative.

Architecture

The model is based on the DistilBERT architecture, which is a smaller, faster, and lighter version of BERT that retains 97% of its language understanding. This makes it well-suited for NLP tasks requiring efficiency.

Training

Training Hyperparameters

• Learning Rate: 2e-05
• Train Batch Size: 64
• Eval Batch Size: 64
• Seed: 42
• Optimizer: Adam with betas=(0.9, 0.999) and epsilon=1e-08
• LR Scheduler Type: Linear
• Number of Epochs: 3.0
• Mixed Precision Training: Native AMP

Training Results

• Loss on Evaluation Set: 2.4264
• Training involved multiple epochs with validation losses recorded at each step.

Framework Versions

• Transformers: 4.12.0.dev0
• PyTorch: 1.9.1+cu111
• Datasets: 1.12.2.dev0
• Tokenizers: 0.10.3

Guide: Running Locally

1. Install Dependencies: Ensure you have Python and pip installed. Use pip install transformers torch datasets to install necessary libraries.
2. Download Model: Use the Hugging Face Transformers library to load the model.

   from transformers import AutoTokenizer, AutoModelForSequenceClassification
   
   tokenizer = AutoTokenizer.from_pretrained("huggingface-course/distilbert-base-uncased-finetuned-imdb")
   model = AutoModelForSequenceClassification.from_pretrained("huggingface-course/distilbert-base-uncased-finetuned-imdb")
   

3. Run Inference: Tokenize input text and run through the model to get predictions.
4. Setup Cloud GPU: For intensive tasks, consider using cloud GPUs from providers like AWS, Google Cloud, or Azure to speed up computations.

License

This model is licensed under the Apache License 2.0. Users are free to use, modify, and distribute the model, subject to the license terms.