bert base cased squad v1.1 portuguese

Introduction

The model is a Portuguese BERT base, cased for question answering, finetuned on SQUAD v1.1 by the Deep Learning Brasil group. It utilizes the BERTimbau Base from Neuralmind.ai, a pretrained BERT model for Brazilian Portuguese, achieving state-of-the-art performance on several NLP tasks.

Architecture

The underlying architecture is based on BERTimbau Base, a variant of BERT tailored for Brazilian Portuguese. This model has been finetuned specifically for question answering tasks, using the SQUAD v1.1 dataset translated into Portuguese.

Training

Training was conducted using the SQUAD v1.1 dataset in Portuguese. The model was trained and evaluated on Google Colab with performance metrics including an F1 score of 82.50 and an exact match score of 70.49. Further information on the training method can be found in a detailed blog post.

Guide: Running Locally

1. Install Transformers Library:

   pip install transformers
   

2. Set Up the Model:
   Use the pipeline for question answering:

   from transformers import pipeline
   model_name = 'pierreguillou/bert-base-cased-squad-v1.1-portuguese'
   nlp = pipeline("question-answering", model=model_name)
   

3. Ask Questions:
   Pass a question and context to get an answer.

   question = "Quando começou a pandemia de Covid-19 no mundo?"
   context = "A pandemia de COVID-19, também conhecida como pandemia de coronavírus..."
   result = nlp(question=question, context=context)
   print(f"Answer: '{result['answer']}', score: {round(result['score'], 4)}")
   

4. Clone the Model Repository (Optional):

   git lfs install
   git clone https://huggingface.co/pierreguillou/bert-base-cased-squad-v1.1-portuguese
   

5. Cloud GPU Recommendation:
   For intensive tasks, consider using cloud GPUs from providers like Google Colab or AWS.

License

The model is released under the MIT license, allowing for wide usage and modification.