vi mrc base

Introduction

The VI-MRC-BASE model is a Vietnamese question-answering model created by Binh Nguyen. It leverages the XLM-RoBERTa language model, fine-tuned for extractive question answering tasks. It is designed for use with Vietnamese datasets but can also work with English inputs.

Architecture

The model is based on the XLM-RoBERTa architecture, which splits words into sub-words. In this implementation, sub-word representations are recombined into word representations using a sum strategy, enhancing the model's ability to understand the context in Vietnamese.

Training

The model is trained on a combination of datasets, including SQuAD 2.0, mailong25, UIT-ViQuAD, and MultiLingual Question Answering. The evaluation results using 10% of the Vietnamese dataset show an EM (Exact Match) score of 76.43 and an F1 score of 84.16 for the base model.

Guide: Running Locally

1. Set Up Environment:

   • Ensure you have Python installed.
   • Install the necessary packages: transformers and torch.

2. Using Pre-trained Model via Hugging Face Pipeline:

   from transformers import pipeline
   
   model_checkpoint = "nguyenvulebinh/vi-mrc-base"
   nlp = pipeline('question-answering', model=model_checkpoint, tokenizer=model_checkpoint)
   QA_input = {
     'question': "Bình là chuyên gia về gì ?",
     'context': "Bình Nguyễn là một người đam mê với lĩnh vực xử lý ngôn ngữ tự nhiên . Anh nhận chứng chỉ Google Developer Expert năm 2020"
   }
   res = nlp(QA_input)
   print('pipeline: {}'.format(res))
   

3. Running on Cloud GPUs:

   • Consider using cloud services like Google Colab or AWS for GPU support, which can significantly speed up inference and training.

License

The VI-MRC-BASE model is licensed under the Creative Commons Attribution-NonCommercial 4.0 International License (cc-by-nc-4.0). This allows for adaptation and use of the model for non-commercial purposes with proper attribution.