splade v3 distilbert LLM Model

Introduction

SPLADE-v3-DistilBERT is a version of DistilBERT developed by NAVER Labs Europe, optimized for specific tasks using the SPLADE framework. It is designed for tasks in natural language processing and information retrieval.

Architecture

SPLADE-v3-DistilBERT is built upon the DistilBERT architecture and is part of the SPLADE framework, which focuses on sparse representations for efficient information retrieval. The model is designed to be lightweight while maintaining performance.

Training

The model achieves notable performance metrics such as MRR@10 of 38.7 on the MS MARCO dev dataset and an average nDCG@10 of 50.0 on BEIR-13. These metrics highlight its effectiveness in retrieval tasks.

Guide: Running Locally

Installation: First, clone the SPLADE repository from GitHub:
```
git clone https://github.com/naver/splade
```
Dependencies: Navigate to the cloned directory and install the required packages:
```
cd splade
pip install -r requirements.txt
```

Model Loading: Use the Hugging Face Transformers library to load the model:

from transformers import AutoModelForMaskedLM, AutoTokenizer

model = AutoModelForMaskedLM.from_pretrained("naver/splade-v3-distilbert")
tokenizer = AutoTokenizer.from_pretrained("naver/splade-v3-distilbert")

Inference: Perform inference using the model and tokenizer for fill-mask tasks.

For enhanced performance, consider using cloud GPU services such as AWS, Google Cloud, or Azure, which provide scalable and powerful computing resources.

License

SPLADE-v3-DistilBERT is licensed under the Creative Commons Attribution Non Commercial Share Alike 4.0 International (CC BY-NC-SA 4.0). This license allows for sharing and adapting the material, but not for commercial purposes.

More Related APIs in Fill Mask