sbert_large_mt_nlu_ru

Introduction

The SBERT_LARGE_MT_NLU_RU model is a BERT-based large model designed for multitask sentence embeddings in the Russian language. It is developed by the SberDevices team and aims to achieve high-quality sentence embeddings by utilizing mean token embeddings.

Architecture

The model is based on the BERT architecture and is implemented using PyTorch and the Transformers library. It is designed to handle multitask sentence embeddings for the Russian language.

Training

The model achieves its performance by focusing on sentence embeddings and uses mean pooling to process embeddings accurately. It is evaluated using metrics from the Russian SuperGLUE benchmark.

Guide: Running Locally

1. Installation:

   • Ensure Python and PyTorch are installed.
   • Install Hugging Face Transformers with pip install transformers.

2. Usage:

   • Import relevant classes from Transformers and PyTorch.
   • Load the model and tokenizer using:

     from transformers import AutoTokenizer, AutoModel
     tokenizer = AutoTokenizer.from_pretrained("ai-forever/sbert_large_mt_nlu_ru")
     model = AutoModel.from_pretrained("ai-forever/sbert_large_mt_nlu_ru")
     

   • Tokenize your sentences and compute embeddings using the example code provided.

3. Hardware Requirements:

   • For optimal performance, consider using cloud GPU services such as AWS, Google Cloud Platform, or Azure.

License

The model is developed by the SberDevices team, with contributions from Aleksandr Abramov and Denis Antykhov. Specific licensing details are not provided in the README.