Mutual_ Implication_ Score

Introduction

The Mutual Implication Score (MIS) is a symmetric measure for evaluating text semantic similarity. It is based on a RoBERTa model fine-tuned for paraphrase detection, leveraging natural language inference datasets. This measure is particularly effective for paraphrase detection and applicable to other semantic similarity tasks, such as text style transfer.

Architecture

MIS employs a modified RoBERTa-Large NLI model, which has been fine-tuned using the Quora Question Pairs (QQP) paraphrase dataset. This architecture allows the model to excel in tasks requiring evaluation of semantic similarity between pairs of texts.

Training

The model was fine-tuned on the QQP paraphrase dataset. It was developed to improve upon existing measures for paraphrase detection and text style transfer, showing superior performance in modeling semantic similarity across various evaluated datasets.

Guide: Running Locally

To use the Mutual Implication Score model locally, you can follow these steps:

1. Install the package:

   !pip install mutual-implication-score
   

2. Run the model:

   from mutual_implication_score import MIS
   mis = MIS(device='cpu') # Use 'cuda:0' for GPU
   source_texts = ['I want to leave this room', 'Hello world, my name is Nick']
   paraphrases = ['I want to go out of this room', 'Hello world, my surname is Petrov']
   scores = mis.compute(source_texts, paraphrases)
   print(scores)  # Expected output: [0.9748, 0.0545]
   

3. Cloud GPUs: For better performance, especially with large datasets, consider using cloud-based GPUs such as those from AWS, GCP, or Azure.

License

This model is distributed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. For more details, visit the Creative Commons website.