E S G B E R T

Introduction

ESG-BERT is a domain-specific BERT model developed for text mining in sustainable investing. It was created by Mukut Mukherjee, Charan Pothireddi, and Parabole.ai. The model is designed to perform various NLP tasks, particularly in the context of Environmental, Social, and Governance (ESG) topics.

Architecture

ESG-BERT is based on the BERT architecture. It is a language model pre-trained on unstructured text data and fine-tuned for specific tasks in sustainable investing. The model leverages BERT's capabilities to understand and process the nuances of the ESG domain.

Training

The model was trained on unstructured text data, achieving high accuracy in tasks like Next Sentence Prediction and Masked Language Modeling. Fine-tuning for text classification resulted in an F-1 score of 0.90, outperforming the general BERT model and traditional machine learning approaches.

Guide: Running Locally

1. Install Required Packages:

   pip install torchserve torch-model-archiver torchvision transformers
   

2. Set Up the Handler Script: Save the handler script as handler.py and customize it as needed for text classification.

3. Create Model Archive:

   torch-model-archiver --model-name "bert" --version 1.0 --serialized-file ./bert_model/pytorch_model.bin --extra-files "./bert_model/config.json,./bert_model/vocab.txt" --handler "./handler.py"
   

4. Prepare Model Store:

   mkdir model_store && mv bert.mar model_store
   

5. Start TorchServe:

   torchserve --start --model-store model_store --models bert=bert.mar
   

6. Query the Model:

   curl -X POST http://127.0.0.1:8080/predictions/bert -T predict.txt
   

   • Cloud GPUs: Consider using cloud services like AWS, GCP, or Azure to leverage GPU resources for faster model inference.

License

Information regarding the license for ESG-BERT is required. Users should verify the license details before deploying it in production environments.