rubert tiny toxicity

Introduction

The cointegrated/rubert-tiny-toxicity model is a fine-tuned classifier designed to identify toxicity and inappropriateness in short informal Russian texts, such as social media comments. It operates as a multilabel classifier with categories including non-toxic, insult, obscenity, threat, and dangerous.

Architecture

This model is built on the cointegrated/rubert-tiny architecture and utilizes the Transformers library and PyTorch framework. It is designed to work effectively with Russian text, leveraging a multilabel classification approach.

Training

The training process involved a combined dataset from the OK ML Cup and Babakov et.al., using an Adam optimizer with a learning rate of 1e-5 and a batch size of 64 across 15 epochs. Training was conducted in a Colab notebook. The model achieves high ROC AUC scores across different labels, demonstrating robust performance in identifying non-toxic, insulting, obscene, threatening, and dangerous content.

Guide: Running Locally

1. Installation: Ensure you have transformers and sentencepiece installed:

   pip install transformers sentencepiece --quiet
   

2. Import Libraries:

   import torch
   from transformers import AutoTokenizer, AutoModelForSequenceClassification
   

3. Load Model and Tokenizer:

   model_checkpoint = 'cointegrated/rubert-tiny-toxicity'
   tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
   model = AutoModelForSequenceClassification.from_pretrained(model_checkpoint)
   if torch.cuda.is_available():
       model.cuda()
   

4. Define Toxicity Function:

   def text2toxicity(text, aggregate=True):
       with torch.no_grad():
           inputs = tokenizer(text, return_tensors='pt', truncation=True, padding=True).to(model.device)
           proba = torch.sigmoid(model(**inputs).logits).cpu().numpy()
       if isinstance(text, str):
           proba = proba[0]
       if aggregate:
           return 1 - proba.T[0] * (1 - proba.T[-1])
       return proba
   

5. Usage Example:

   print(text2toxicity('я люблю нигеров', True))
   

6. Cloud GPUs: For enhanced performance, consider using cloud GPU services such as Google Colab, AWS, or Azure.

License

The model and its implementations are subject to the licensing terms defined by the authors at the Hugging Face repository. Ensure compliance with these terms when using or distributing the model.