Indobert Q A

Introduction

IndoBERT-QA is a fine-tuned version of the IndoBERT model, specifically adapted for question-answering tasks in the Indonesian language. The model is fine-tuned on the Translated SQuAD 2.0 dataset, making it capable of understanding and responding to questions based on a given context.

Architecture

IndoBERT is based on the BERT architecture, tailored for the Indonesian language. It is trained with over 220 million words from diverse sources, including Indonesian Wikipedia, news articles, and an Indonesian Web Corpus. The fine-tuned model, IndoBERT-QA, is designed for question-answering tasks, leveraging the complexities of the SQuAD 2.0 dataset, which includes both answerable and unanswerable questions.

Training

The IndoBERT-QA model was trained using a Tesla T4 GPU with 12GB of RAM. The training involved using the SQuAD 2.0 dataset, which includes 130,000 training samples and 12,300 evaluation samples. The model achieved an Exact Match (EM) score of 51.61 and an F1 score of 69.09.

Guide: Running Locally

To run the IndoBERT-QA model locally:

1. Install Transformers Library: Ensure you have the Hugging Face Transformers library installed.

   pip install transformers
   

2. Set Up the Pipeline: Use the following Python code to create a question-answering pipeline:

   from transformers import pipeline
   
   qa_pipeline = pipeline(
       "question-answering",
       model="Rifky/Indobert-QA",
       tokenizer="Rifky/Indobert-QA"
   )
   
   result = qa_pipeline({
       'context': """Your context here""",
       'question': "Your question here"
   })
   
   print(result)
   

3. Consider Using Cloud GPUs: For improved performance, consider running the model on cloud platforms that offer GPU support, such as AWS, Google Cloud, or Azure.

License

The IndoBERT-QA model is licensed under the Apache 2.0 License, allowing for both personal and commercial use, provided that proper attribution is maintained.