unbiased toxic roberta

Introduction

The Unbiased Toxic RoBERTa model by Unitary is designed for toxic comment classification across various challenges, including detecting unintended bias and handling multilingual data. This initiative aims to mitigate harmful online content by leveraging advanced machine learning techniques.

Architecture

The model employs several transformer-based architectures:

• bert-base-uncased for the Toxic Comment Classification Challenge.
• roberta-base for the Unintended Bias in Toxicity Classification.
• xlm-roberta-base for the Multilingual Toxic Comment Classification.

These models are built using PyTorch and the Hugging Face Transformers library, allowing for flexible deployment and fine-tuning.

Training

To train these models, you must first obtain datasets from Kaggle, requiring an account and an API token. The training involves:

• Setting up a virtual environment and installing necessary dependencies.
• Downloading Jigsaw competition datasets.
• Running specific training scripts for each challenge, with configurations detailed in JSON files.

Guide: Running Locally

1. Clone the Repository

   git clone https://github.com/unitaryai/detoxify
   cd detoxify
   

2. Set Up Environment

   python3 -m venv toxic-env
   source toxic-env/bin/activate
   

3. Install Dependencies

   pip install -e .
   pip install -r requirements.txt
   

4. Download Datasets

   • Use Kaggle API to download datasets for the Jigsaw challenges.

5. Run Predictions

   python run_prediction.py --input 'example' --model_name original
   

6. Cloud GPUs
   For efficient training and inference, consider using cloud GPUs available through services like AWS, Google Cloud, or Azure.

License

The Unbiased Toxic RoBERTa model is released under the Apache 2.0 License, which allows for broad use and modification with proper attribution.