BERT-FINETUNED-SQUAD

Introduction

The BERT-FINETUNED-SQUAD model is a fine-tuned version of the BERT model, optimized for question-answering tasks using the SQuAD dataset. This model is part of the Hugging Face course, providing users with a practical example of fine-tuning transformer models for specific applications.

Architecture

The model utilizes the BERT architecture, originally designed by Google, which is pre-trained on a large corpus of text and then fine-tuned on the SQuAD dataset. This approach leverages transfer learning, allowing the model to perform effectively on question-answering tasks.

Training

Training Procedure

The model was trained using the following hyperparameters:

Learning Rate: 2e-05
Train Batch Size: 8
Eval Batch Size: 8
Seed: 42
Optimizer: Adam with betas=(0.9, 0.999) and epsilon=1e-08
Learning Rate Scheduler: Linear
Number of Epochs: 3
Mixed Precision Training: Native AMP

Framework Versions

The following framework versions were used:

Transformers: 4.11.0.dev0
PyTorch: 1.8.1+cu111
Datasets: 1.12.2.dev0
Tokenizers: 0.10.3

Guide: Running Locally

To run the BERT-FINETUNED-SQUAD model locally, follow these steps:

Setup Environment
Install the required libraries:
```
pip install transformers datasets torch
```

Download Model
Use the Hugging Face transformers library to load the model:

from transformers import pipeline

qa_pipeline = pipeline("question-answering", model="huggingface-course/bert-finetuned-squad")

Run Inference
Provide a context and question to get answers:

context = "Your context here."
question = "Your question here."
result = qa_pipeline(question=question, context=context)
print(result)

Optional: Use Cloud GPUs
For faster inference or when dealing with large datasets, consider using cloud GPUs from providers such as AWS, GCP, or Azure.

License

The model is available under a license specified by Hugging Face. Ensure to review the license terms on the Hugging Face model card page before usage.

bert finetuned squad