chinese pert base mrc LLM Model

Introduction

The Chinese PERT-Base MRC is a machine reading comprehension (MRC) model designed to understand and answer questions in Chinese. It is built upon the PERT-base model, which utilizes a permuted language model (PerLM) for semantic learning without using mask tokens.

Architecture

The PERT model is a pre-trained model that focuses on learning text semantics in a self-supervised manner. It is competitive in tasks such as reading comprehension and sequence labeling. The model architecture does not rely on traditional [MASK] tokens, which differentiates it from models like BERT.

Training

This model is fine-tuned on a mixture of Chinese MRC datasets. It achieves the following results on various Chinese MRC datasets:

CMRC 2018 Dev: EM 73.2 / F1 90.6
DRCD Dev: EM 88.7 / F1 94.1
SQuAD-Zen Dev (Answerable): EM 59.7 / F1 76.5
Average: EM 73.9 / F1 87.1

Guide: Running Locally

Install the Transformers Library: Ensure you have the transformers library installed, preferably via pip.
```
pip install transformers
```

Loading the Model: Use the BertForQuestionAnswering class in the Transformers library to load the model.

from transformers import BertForQuestionAnswering

model = BertForQuestionAnswering.from_pretrained("hfl/chinese-pert-base-mrc")

Running Inference: Use the model for question-answering tasks with relevant input data.
Hardware Requirements: For optimal performance, running this model on a cloud-based GPU is recommended. Providers like AWS, Google Cloud, and Azure offer suitable instances.

License

The Chinese PERT-Base MRC model is licensed under the Apache-2.0 License. This permissive license allows for both personal and commercial use, distribution, and modification of the model.