distilbert base multilingual cased toxicity

Introduction

The DistilBERT-Base-Multilingual-Cased-Toxicity model is a multilingual sequence classifier developed by CitizenLab. It is designed to classify text as toxic or not toxic, using the Jigsaw Toxic Comment Classification Challenge dataset for training. The model supports multiple languages and provides high accuracy in identifying toxic content.

Architecture

The model is based on the DistilBERT architecture, a smaller, faster, and more efficient version of BERT, which retains most of its language understanding capabilities. The multilingual aspect allows the model to handle text in various languages, including English, Dutch, French, Portuguese, Italian, Spanish, German, Danish, Polish, and Afrikaans.

Training

The model was trained using the Jigsaw Toxic Comment Classification Challenge dataset. It achieved an accuracy score of 0.9425, a micro F1 score of 0.945, and a macro F1 score of 0.849. These metrics demonstrate the model's effectiveness in classifying text for toxicity across different languages.

Guide: Running Locally

To run the model locally, follow these steps:

1. Install the Transformers library: Ensure you have the transformers library installed. You can install it via pip:

   pip install transformers
   

2. Load and use the model:

   from transformers import pipeline
   
   model_path = "citizenlab/distilbert-base-multilingual-cased-toxicity"
   toxicity_classifier = pipeline("text-classification", model=model_path, tokenizer=model_path)
   
   # Classify text
   result = toxicity_classifier("this is a lovely message")
   print(result)
   

3. Suggested Infrastructure: It is recommended to use a cloud GPU service for faster inference, especially if processing large datasets or deploying at scale. Services like AWS, GCP, or Azure offer flexible GPU instances.

License

The model is distributed under the Apache License 2.0, allowing for both personal and commercial use with proper attribution.