bert base finnish cased v1

Introduction

The BERT-BASE-FINNISH-CASED-V1 model is a Finnish adaptation of Google's BERT deep transfer learning model, designed to achieve high performance on various Finnish NLP tasks. It was developed by the TurkuNLP research group and features a custom vocabulary tailored for the Finnish language, enabling superior coverage and performance compared to multilingual BERT models.

Architecture

FinBERT utilizes a custom 50,000 wordpiece vocabulary that enhances its ability to process Finnish text. The model was pre-trained on 3 billion tokens from diverse Finnish text sources, including news, online discussions, and internet crawls, over 1 million training steps. This extensive training allows it to outperform multilingual BERT in Finnish-specific tasks.

Training

FinBERT's training process involved a significant amount of Finnish text, which is substantially larger than the Finnish portion of the multilingual BERT's training data. This extensive pre-training enables FinBERT to deliver superior results in natural language processing tasks such as document classification, named entity recognition, and part-of-speech tagging.

Guide: Running Locally

To run the BERT-BASE-FINNISH-CASED-V1 model locally, follow these steps:

1. Installation: Ensure you have Python and the Hugging Face Transformers library installed. You can install the library using pip:

   pip install transformers
   

2. Download the Model: Access the model from Hugging Face's model hub:

   from transformers import BertModel, BertTokenizer
   tokenizer = BertTokenizer.from_pretrained('TurkuNLP/bert-base-finnish-cased-v1')
   model = BertModel.from_pretrained('TurkuNLP/bert-base-finnish-cased-v1')
   

3. Run Inference: Tokenize and run inference using the model on your Finnish text data.

4. Consider Cloud GPUs: For enhanced performance and faster inference, consider using cloud-based GPUs such as those offered by AWS, Google Cloud, or Azure.

License

For information regarding the licensing of the BERT-BASE-FINNISH-CASED-V1 model, you should refer to the Hugging Face model card or contact the TurkuNLP research group.