F R I D A

Introduction

FRIDA is a comprehensive text embedding model that utilizes a denoising architecture inspired by T5. It is based on the encoder part of the FRED-T5 model and has been finetuned for tasks involving Russian and English texts. The model is designed for various applications, including retrieval, paraphrasing, sentiment analysis, and topic classification.

Architecture

FRIDA's architecture is built upon the encoder segment of the FRED-T5 model, which is optimized for text embedding tasks. The model supports multiple prefixes to guide the embedding process based on the task, such as "search_query," "paraphrase," and "categorize_topic." The choice of prefix influences the embedding strategy, aiding in task-specific optimizations.

Training

The model has been pre-trained on a Russian-English dataset and further fine-tuned to enhance performance on specific tasks. The finetuning process involves using high-quality datasets to adapt the model to different text classification and retrieval tasks. The model's training encompasses various metrics to evaluate and improve its performance across tasks like classification, clustering, and retrieval.

Guide: Running Locally

To run FRIDA locally, follow these steps:

1. Setup Environment: Ensure Python and PyTorch are installed. Use a virtual environment for easy dependency management.

2. Install Required Libraries:

   pip install transformers sentence-transformers torch
   

3. Download and Initialize the Model:

   from transformers import AutoTokenizer, T5EncoderModel
   
   tokenizer = AutoTokenizer.from_pretrained("ai-forever/FRIDA")
   model = T5EncoderModel.from_pretrained("ai-forever/FRIDA")
   

4. Encode Texts: Use the tokenizer and model to encode texts with the appropriate prefix:

   tokenized_inputs = tokenizer(["paraphrase: Your text here"], return_tensors="pt")
   outputs = model(**tokenized_inputs)
   

5. Cloud GPUs: For large-scale tasks or faster processing, consider using cloud services like AWS, Google Cloud, or Azure, which offer GPU instances.

License

FRIDA is licensed under the MIT License, allowing for free use, modification, and distribution, subject to the license terms.