Modern B E R T base nli

Introduction

ModernBERT-base-nli is a multi-task model fine-tuned on a variety of Natural Language Inference (NLI) datasets. It excels in reasoning tasks, sentiment analysis, and zero-shot classification. The model is based on "ModernBERT" architecture and was trained for 200k steps using an Nvidia A30 GPU.

Architecture

The model is built using Hugging Face's transformers library and leverages the ModernBERT architecture. It is fine-tuned for NLI tasks and supports zero-shot classification pipelines. This model can serve as a robust backbone for further fine-tuning due to its enhanced reasoning capabilities.

Training

ModernBERT-base-nli was trained on a diverse set of NLI datasets, including MNLI, ANLI, and others, achieving notable test accuracies across various benchmarks. The training utilized an Nvidia A30 GPU, focusing on improving reasoning and long-context processing abilities.

Guide: Running Locally

1. Install Transformers Library: Ensure you have the transformers library installed via pip:

   pip install transformers
   

2. Zero-Shot Classification Pipeline:

   from transformers import pipeline
   
   classifier = pipeline("zero-shot-classification", model="tasksource/ModernBERT-base-nli")
   text = "one day I will see the world"
   candidate_labels = ['travel', 'cooking', 'dancing']
   classifier(text, candidate_labels)
   

3. Natural Language Inference Pipeline:

   from transformers import pipeline
   
   pipe = pipeline("text-classification", model="tasksource/ModernBERT-base-nli")
   pipe([dict(text='there is a cat', text_pair='there is a black cat')])
   

4. Cloud GPUs: For optimal performance, consider using cloud-based GPU services such as AWS EC2, Google Cloud, or Azure.

License

ModernBERT-base-nli is licensed under the Apache-2.0 License.