Modern B E R T large zeroshot v1

Introduction

ModernBERT-large-zeroshot-v1 is a fine-tuned BERT variant designed for zero-shot classification in English. It leverages the MoritzLaurer/synthetic_zeroshot_mixtral_v0.1 dataset to perform Natural Language Inference tasks.

Architecture

• Model Type: ModernBERT-large (BERT variant)
• Task: Zero-shot Classification
• Language: English
• Dataset: MoritzLaurer/synthetic_zeroshot_mixtral_v0.1

Training

The model was fine-tuned using the following configuration:

• Framework: PyTorch
• Batch Size: 32
• Learning Rate: 2e-5
• Optimizer: AdamW
• Hardware: RTX 4090

Guide: Running Locally

1. Install Dependencies:

   pip install transformers torch datasets
   

2. Use the Model:

   from transformers import pipeline
   
   classifier = pipeline("zero-shot-classification", model="r-f/ModernBERT-large-zeroshot-v1")
   sequence_to_classify = "I want to be an actor."
   candidate_labels = ["space", "economy", "entertainment"]
   output = classifier(sequence_to_classify, candidate_labels, multi_label=False)
   print(output)
   

3. Cloud GPUs: For efficient processing, consider using cloud platforms like AWS, Google Cloud, or Azure with GPU support.

License

This model is licensed under the MIT License. See the LICENSE file for more details.