ModernBERT-base-msmarco

Introduction

ModernBERT-base-msmarco is a model for sentence similarity tasks, built on the BERT architecture. It is designed for feature extraction and is optimized for English. The model benefits from the advancements of modern BERT implementations and is trained with a large dataset to enhance performance on sentence similarity evaluations.

Architecture

The model utilizes the BERT architecture, specifically tailored for modern adaptations that improve sentence similarity tasks. It leverages the framework provided by sentence-transformers and employs safetensors for efficient handling of model data.

Training

The model was trained using the CachedMultipleNegativesRankingLoss strategy, which is effective for tasks requiring high-quality ranking of multiple sentence pairs. The training dataset consists of 11,662,655 samples, ensuring robust and comprehensive learning. The training process was guided by the methodologies detailed in research papers such as arXiv:1908.10084 and arXiv:2101.06983.

Guide: Running Locally

To run ModernBERT-base-msmarco locally:

1. Clone the Repository: Use the provided GitHub link to clone the model repository.
2. Install Dependencies: Ensure you have the sentence-transformers library installed. Use the command:

   pip install sentence-transformers
   

3. Load the Model: Utilize the sentence-transformers library to load the model.
4. Run Inference: Use your data to perform sentence similarity tasks by providing input sentences to the model.

For optimal performance, consider using cloud GPUs such as those from AWS, Google Cloud, or Azure, which support deep learning frameworks and can handle large model computations efficiently.

License

The ModernBERT-base-msmarco model is subject to the licensing agreements outlined by the model creator, Joe32140. Ensure compliance with these terms when using or distributing the model.