Czert A base uncased

Introduction

The CZERT repository hosts the CZERT-A model, a Czech BERT-like model designed for language representation. It is associated with the paper "Czert – Czech BERT-like Model for Language Representation" which is available on arXiv.

Architecture

CZERT offers two versions of pre-trained models: CZERT-A and CZERT-B. They are designed for a variety of NLP tasks and come with corrected tokenizer configurations in version 2. The models are available in different configurations, including those fine-tuned for tasks like sentiment classification, named entity recognition, morphological tagging, and semantic role labeling.

Training

CZERT models are evaluated on several NLP tasks across sentence, document, and token levels:

• Sentence Level Tasks: Sentiment Classification and Semantic Text Similarity.
• Document Level Tasks: Multi-label Document Classification.
• Token Level Tasks: Named Entity Recognition, Morphological Tagging, and Semantic Role Labeling.

The models have demonstrated competitive results compared to other pre-trained models like mBERT, Pavlov, and Albert-random.

Guide: Running Locally

1. Download Model: Select and download the desired CZERT model (e.g., CZERT-A-v2) from the provided links.
2. Set Up Environment: Install the necessary libraries, primarily PyTorch or TensorFlow, depending on your preference for running the model.
3. Load Model: Use the Hugging Face Transformers library to load the model into your environment.
4. Run Inference: Input data and run inference for your specific task, such as sentiment analysis or named entity recognition.

For efficient model training and inference, consider using cloud GPUs from providers such as AWS, Google Cloud, or Azure.

License

This project is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License, allowing sharing and adaptation under similar terms for non-commercial purposes. More details can be found here.