albert base v2 squad LLM Model

Introduction

The ALBERT-BASE-V2-SQUAD model is a version of the ALBERT v2 model fine-tuned on the SQuAD v1 dataset for the task of question answering. This model focuses on providing improved results over the original ALBERT model by enhancing metrics such as Exact Match (EM) and F1 scores.

Architecture

The model is based on the ALBERT (A Lite BERT) architecture, which is designed to be more parameter-efficient and faster than BERT, making it suitable for large-scale training and deployment. ALBERT utilizes techniques such as parameter sharing and factorized embedding parameterization to achieve this efficiency.

Training

The model was trained using the nn_pruning script from Hugging Face, although pruning features were disabled during this process. The training aimed to improve the model's performance on the SQuAD v1 dataset, resulting in an Exact Match score of 83.75 and an F1 score of 90.79, surpassing the original results of F1=90.2 and EM=83.2.

Guide: Running Locally

To run the ALBERT-BASE-V2-SQUAD model locally, follow these steps:

Install the Hugging Face Transformers library by running:
```
pip install transformers
```

Clone the model repository from Hugging Face:

git clone https://huggingface.co/madlag/albert-base-v2-squad

Load the model in your Python script:

from transformers import AutoTokenizer, AutoModelForQuestionAnswering

tokenizer = AutoTokenizer.from_pretrained("madlag/albert-base-v2-squad")
model = AutoModelForQuestionAnswering.from_pretrained("madlag/albert-base-v2-squad")

If extensive computation is needed, consider using cloud-based GPU services such as AWS, GCP, or Azure to enhance performance and reduce training time.

License

The model is provided under the Apache 2.0 License, which allows for both personal and commercial use. Users should ensure they comply with the terms of this license when using the model.