bge m3

Introduction

BGE-M3 is a versatile model developed to address multiple functionalities, languages, and granularities in text retrieval tasks. It integrates dense retrieval, multi-vector retrieval, and sparse retrieval, supporting over 100 languages. It can process both short sentences and long documents up to 8192 tokens.

Architecture

BGE-M3 is built to accommodate various retrieval needs:

• Multi-Functionality: Performs dense, multi-vector, and sparse retrieval.
• Multi-Linguality: Supports more than 100 languages.
• Multi-Granularity: Handles inputs ranging from short texts to long documents.

Training

The model employs techniques such as self-knowledge distillation and efficient batching. These methods enhance performance across different retrieval modes and improve efficiency when fine-tuning on long texts. Additionally, MCLS is used to boost performance on long texts without the need for fine-tuning.

Guide: Running Locally

1. Installation:

   • Clone the repository and install dependencies:

     git clone https://github.com/FlagOpen/FlagEmbedding.git
     cd FlagEmbedding
     pip install -e .
     

   • Alternatively, install directly:

     pip install -U FlagEmbedding
     

2. Generating Embeddings:

   • Load the model and encode your sentences:

     from FlagEmbedding import BGEM3FlagModel
     
     model = BGEM3FlagModel('BAAI/bge-m3', use_fp16=True)
     sentences = ["Example sentence 1", "Example sentence 2"]
     embeddings = model.encode(sentences)['dense_vecs']
     

3. Cloud GPUs: For faster processing, consider using cloud services like AWS, GCP, or Azure that offer GPUs.

License

The BGE-M3 model is released under the MIT license, allowing for flexible use and modification.