sentence bert swedish cased

Introduction

The SENTENCE-BERT-SWEDISH-CASED model is a sentence-transformers model designed to map Swedish sentences and paragraphs to a 768-dimensional dense vector space. It is suitable for tasks such as clustering and semantic search. Developed by KBLab, a data lab at the National Library of Sweden, this model is a bilingual Swedish-English model created using knowledge distillation techniques.

Architecture

The model architecture consists of two main components: a Transformer model and a pooling layer. The Transformer model is based on BertModel with a maximum sequence length of 384 and no lower casing. The pooling layer uses mean pooling to aggregate token embeddings into sentence embeddings.

Training

The model was trained using approximately 14.6 million sentences from English-Swedish parallel corpuses, sourced from OPUS datasets like JW300, Europarl, and others. Training used a batch size of 64 and an MSELoss function. Parameters included 2 epochs, a learning rate of 8e-06, and a warmup phase of 5000 steps.

Guide: Running Locally

1. Install Dependencies:

   pip install -U sentence-transformers
   

2. Load the Model:

   from sentence_transformers import SentenceTransformer
   model = SentenceTransformer('KBLab/sentence-bert-swedish-cased')
   

3. Encode Sentences:

   sentences = ["Det här är en exempelmening", "Varje exempel blir konverterad"]
   embeddings = model.encode(sentences)
   print(embeddings)
   

4. Using Cloud GPUs: For improved performance, consider using cloud services such as AWS, Google Cloud, or Azure that offer GPU instances.

License

The model is licensed under the Apache-2.0 License, allowing for both personal and commercial use with proper attribution.