xlm roberta large xnli

Introduction

The XLM-RoBERTa-Large-XNLI model is a fine-tuned version of the xlm-roberta-large model for zero-shot text classification tasks across multiple languages. It is designed to work with the Hugging Face ZeroShotClassificationPipeline and is particularly effective for multilingual use cases.

Architecture

The model is based on the xlm-roberta-large architecture and has been fine-tuned on a combination of multilingual Natural Language Inference (NLI) data from the XNLI dataset, which includes 15 languages such as English, French, Spanish, and more. The base model was pre-trained on 100 languages, allowing it to perform well beyond the languages explicitly fine-tuned upon.

Training

XLM-RoBERTa-Large-XNLI was initially pre-trained on a dataset of 100 languages. It was subsequently fine-tuned on the MNLI train set combined with the XNLI validation and test sets. An additional training epoch was conducted on shuffled language pairs in the XNLI dataset to enhance multilingual understanding.

Guide: Running Locally

1. Install Libraries:

   pip install transformers
   

2. Load the Model with Zero-Shot Classification Pipeline:

   from transformers import pipeline
   classifier = pipeline("zero-shot-classification",
                         model="joeddav/xlm-roberta-large-xnli")
   

3. Classify Text:

   sequence_to_classify = "За кого вы голосуете в 2020 году?"
   candidate_labels = ["Europe", "public health", "politics"]
   results = classifier(sequence_to_classify, candidate_labels)
   print(results)
   

4. Cloud GPUs: For faster performance, consider using cloud GPU services such as AWS EC2, Google Cloud, or Azure.

License

This model is available under the MIT License, allowing for a wide range of uses with minimal restrictions.