mxbai embed large v1

Introduction

The MXBAI-EMBED-LARGE-V1 model by Mixedbread-AI is designed for generating sentence embeddings and supports tasks such as retrieval, clustering, and classification. It includes features like Matryoshka Representation Learning and binary quantization to enhance performance and efficiency. The model is optimized for English and is part of the sentence-transformers library.

Architecture

The model architecture supports BERT-based embeddings and various pooling strategies, such as CLS pooling. It is compatible with different libraries, including Transformers, ONNX, and Transformers.js, allowing for flexibility in deployment and execution. The model works well with retrieval prompts to enhance the relevance of sentence embeddings in search tasks.

Training

The model achieves state-of-the-art (SOTA) performance for Bert-large sized models on the MTEB benchmark, outperforming some larger commercial models. It was trained without overlapping MTEB data to ensure generalization across different domains and tasks.

Guide: Running Locally

1. Installation:

   • For Python, install the sentence-transformers library:

     python -m pip install -U sentence-transformers
     

   • For JavaScript, install the @xenova/transformers package:

     npm i @xenova/transformers
     

2. Loading the Model:

   • Python: Use the SentenceTransformer class to load the model with specified dimensions.
   • JavaScript: Use the pipeline function to create a feature extraction pipeline.

3. Encoding and Similarity:

   • Encode sentences using the model's encode function.
   • Compute cosine similarities between query and document embeddings.

4. Optional Quantization:

   • Use binary quantization to reduce memory usage for embeddings.

5. Suggested Cloud GPUs:

   • Consider using cloud services such as AWS, GCP, or Azure for GPU resources to enhance performance for large-scale deployments.

License

The model is licensed under the Apache 2.0 License, allowing for wide usage and modification with proper attribution.