roberta base latin v2

Introduction

The RoBERTa Base Latin V2 model is a language model designed to work with Latin text. It is based on the RoBERTa architecture and is fine-tuned for tasks involving Latin language processing, such as fill-mask tasks.

Architecture

The model follows the base RoBERTa architecture with modifications to the vocabulary size to accommodate Latin language requirements. It employs a Byte Pair Encoding (BPE) tokenizer with a vocabulary size of 50,000 tokens.

Training

The model was trained on a subset of the CC-100 dataset, specifically the Latin portion. This dataset contains monolingual data gathered from web crawls, providing a rich source of Latin text for training.

Guide: Running Locally

To run the model locally, ensure the following prerequisites are met:

1. Install the Transformers Library
   Ensure you have transformers==4.19.2 installed. You can install it via pip:

   pip install transformers==4.19.2
   

2. Load and Use the Model
   Use the pipeline method from Hugging Face's Transformers library to perform fill-mask tasks:

   from transformers import pipeline
   
   unmasker = pipeline('fill-mask', model='ClassCat/roberta-base-latin-v2')
   result = unmasker("vita brevis, ars <mask>")
   print(result)
   

3. Cloud GPU Recommendation
   For large-scale or performance-intensive tasks, consider using cloud GPU services such as AWS, Google Cloud, or Azure to enhance processing speed and efficiency.

License

This model is distributed under the Creative Commons Attribution-ShareAlike 4.0 International License (cc-by-sa-4.0), allowing for sharing and adaptation with appropriate credit.