sanstib

Introduction
The SANSTIB model is designed for creating sentence embeddings for Sanskrit and Tibetan languages. It is particularly suited for tasks involving semantic similarity.

Architecture
The model utilizes transformers and is compatible with PyTorch. It is based on the RoBERTa architecture, optimized for feature extraction.

Training
The model requires pre-segmentation and transliteration of Sanskrit into an internal format, and Tibetan into Wylie transliteration before training or inference. A script for Sanskrit segmentation and transliteration will be provided.

Guide: Running Locally

1. Set Up Environment: Ensure Python and PyTorch are installed.
2. Install Transformers: Use pip install transformers.
3. Download Model: Retrieve the model from Hugging Face's model hub.
4. Prepare Data: Segment and transliterate Sanskrit, and transliterate Tibetan accordingly.
5. Run Inference: Use the model to generate sentence embeddings.
6. Cloud GPUs: For enhanced performance, consider using cloud GPU services like AWS, GCP, or Azure.

License
The model is licensed under LGPL-LR, allowing for both personal and commercial use under specific conditions.