toxic bert

Introduction

TOXIC-BERT is a model designed for toxic comment classification, developed by Unitary. It is built using PyTorch and Transformers to predict toxic comments in various contexts, including the Jigsaw challenges. The model processes comments in multiple languages and offers insights into identifying toxicity, minimizing bias, and handling multilingual input.

Architecture

TOXIC-BERT utilizes a Transformer-based architecture, leveraging BERT and RoBERTa models for different tasks. The model is available in several configurations: original, unbiased, and multilingual. Each configuration is trained on specific datasets to address distinct challenges, such as identifying toxicity and reducing bias.

Training

Training involves datasets from Jigsaw challenges, including:

• Toxic Comment Classification
• Unintended Bias in Toxicity Classification
• Multilingual Toxic Comment Classification

Training requires setting up an environment, downloading datasets from Kaggle, and executing specific training scripts. The training process can be monitored using TensorBoard, and various configurations are used depending on the task, such as configs/Toxic_comment_classification_BERT.json.

Guide: Running Locally

To run TOXIC-BERT locally:

1. Clone the Repository

   git clone https://github.com/unitaryai/detoxify
   cd detoxify
   

2. Set Up Environment

   python3 -m venv toxic-env
   source toxic-env/bin/activate
   pip install -e .
   

3. Install Dependencies for Training

   pip install -r requirements.txt
   

4. Run Predictions

   • Load a model using torch.hub or from a checkpoint:

     python run_prediction.py --input 'example' --model_name original
     

5. Cloud GPUs Suggestion

   • For faster training and inference, consider using cloud-based GPUs such as those available from AWS, Google Cloud, or Azure.

License

This project is licensed under the Apache 2.0 License. For more detailed information, refer to the license document.