roberta classical chinese large char

Introduction

roberta-classical-chinese-large-char is a RoBERTa model pre-trained on Classical Chinese texts, derived from GuwenBERT-large. It is designed to handle traditional and simplified Chinese characters, suitable for tasks such as sentence segmentation, POS tagging, and dependency parsing.

Architecture

The model is based on the RoBERTa architecture, which is a robustly optimized BERT approach. It utilizes character embeddings enhanced to support both traditional and simplified Chinese characters.

Training

The model is pre-trained on Classical Chinese corpora. It can be further fine-tuned for specific tasks like sentence segmentation, POS tagging, and dependency parsing. The base model used for this pre-training is ethanyt/guwenbert-large.

Guide: Running Locally

To run the model locally, follow these steps:

1. Install the Transformers Library:

   pip install transformers
   

2. Load the Model and Tokenizer:

   from transformers import AutoTokenizer, AutoModelForMaskedLM
   
   tokenizer = AutoTokenizer.from_pretrained("KoichiYasuoka/roberta-classical-chinese-large-char")
   model = AutoModelForMaskedLM.from_pretrained("KoichiYasuoka/roberta-classical-chinese-large-char")
   

3. Inference: Use the model for fill-mask tasks or fine-tune for other applications.

For enhanced performance and efficiency, consider using cloud GPU services such as AWS, Google Cloud, or Azure.

License

The roberta-classical-chinese-large-char model is licensed under the Apache-2.0 License, allowing for both commercial and non-commercial use.