distilbert base uncased finetuned natural questions LLM Model

Introduction

The model DistilBERT-Base-Uncased-Finetuned-Natural-Questions is a fine-tuned version of the DistilBERT architecture, optimized for question-answering tasks using the Natural Questions dataset. The model exhibits a loss of 0.6267 on the evaluation set.

Architecture

DistilBERT is a smaller, faster, and lighter version of BERT, retaining much of BERT's language understanding capabilities. This specific model is finetuned for the task of question answering, leveraging the Natural Questions dataset.

Training

The model was trained using the following hyperparameters:

Learning Rate: 2e-05
Train Batch Size: 64
Eval Batch Size: 64
Seed: 42
Optimizer: Adam with betas=(0.9, 0.999) and epsilon=1e-08
Learning Rate Scheduler: Linear
Number of Epochs: 40

Training Results

The training loss decreased progressively over the epochs, starting from 2.0532 in the first epoch to 0.6267 by the final epoch.

Guide: Running Locally

To run the model locally, follow these steps:

Install Dependencies: Ensure you have Python installed and set up a virtual environment. Install the necessary libraries:
```
pip install transformers torch datasets
```

Download the Model: Use the transformers library to load the model:

from transformers import AutoModelForQuestionAnswering, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("datarpit/distilbert-base-uncased-finetuned-natural-questions")
model = AutoModelForQuestionAnswering.from_pretrained("datarpit/distilbert-base-uncased-finetuned-natural-questions")

Run Inference: Tokenize your input and get predictions from the model.

For optimal performance, especially with larger datasets or more intensive tasks, consider using cloud GPUs from providers like AWS, Google Cloud, or Azure.

License

The model is licensed under the Apache 2.0 License, allowing for both personal and commercial use, modifications, and distribution under the same license.