ms marco electra base

Introduction

The Cross-Encoder model for MS MARCO is designed for passage ranking tasks, specifically for information retrieval. It encodes queries alongside possible passages and ranks the passages by relevance. This model is useful for tasks where retrieving and re-ranking documents is critical, such as search engines.

Architecture

The Cross-Encoder is based on the ELECTRA architecture, a transformer model optimized for sequence classification tasks. It evaluates pairs of query and passage text to determine relevance, leveraging the pre-trained capabilities of the ELECTRA model for efficient processing.

Training

The model is trained using the MS Marco Passage Ranking dataset. Training involves encoding a query with all possible passages and sorting them based on relevance. For detailed training methodology, refer to the SBERT.net Training MS Marco resource.

Guide: Running Locally

Using Transformers

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model = AutoModelForSequenceClassification.from_pretrained('model_name')
tokenizer = AutoTokenizer.from_pretrained('model_name')

features = tokenizer(['How many people live in Berlin?', 'How many people live in Berlin?'], 
                     ['Berlin has a population of 3,520,031 registered inhabitants in an area of 891.82 square kilometers.', 
                      'New York City is famous for the Metropolitan Museum of Art.'], 
                     padding=True, truncation=True, return_tensors="pt")

model.eval()
with torch.no_grad():
    scores = model(**features).logits
    print(scores)

Using SentenceTransformers

from sentence_transformers import CrossEncoder

model = CrossEncoder('model_name', max_length=512)
scores = model.predict([('Query', 'Paragraph1'), ('Query', 'Paragraph2'), ('Query', 'Paragraph3')])

Cloud GPU Suggestion

For optimal performance, especially for large datasets or models, consider using a cloud-based GPU service such as AWS EC2 with GPU instances, Google Cloud Compute Engine, or NVIDIA's GPU Cloud.

License

This model is licensed under the Apache 2.0 License, which allows for both personal and commercial use, modification, and distribution of the software.