splade cocondenser ensembledistil

Introduction

The SPLADE COCONDENSER ENSEMBLEDISTIL model, developed by NAVER LABS EUROPE, is designed for passage retrieval tasks. It incorporates techniques such as query expansion, document expansion, and knowledge distillation to enhance retrieval performance. This model is evaluated on the MS MARCO dataset, achieving an MRR@10 of 38.3 and R@1000 of 98.3.

Architecture

This model leverages the SPLADE architecture, which utilizes sparse representations for information retrieval. It integrates components like bag-of-words and passage-retrieval methods, alongside knowledge distillation to improve its retrieval accuracy and efficiency.

Training

The model is trained on the MS MARCO dataset and employs strategies such as distillation and hard negative sampling to refine its performance. These techniques aim to make sparse neural IR models more effective by enhancing their ability to retrieve relevant passages.

Guide: Running Locally

1. Clone the Repository: Access the code at GitHub.
2. Install Dependencies: Ensure you have PyTorch and Transformers libraries installed.
3. Download Model Weights: Obtain the model weights from Hugging Face's model hub.
4. Run Inference: Utilize the provided code to perform passage retrieval tasks.

For optimal performance, consider using cloud GPU services like AWS, Google Cloud, or Azure.

License

The SPLADE COCONDENSER ENSEMBLEDISTIL model is released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) license. This allows for adaptation and sharing under similar terms for non-commercial purposes.