temporal_tagger_ B E R T_tokenclassifier

Introduction

The Temporal Tagger BERT TokenClassifier is a token classification model for temporal tagging of plain text using the BERT language model. It was introduced in the paper "BERT got a Date: Introducing Transformers to Temporal Tagging." The model is designed to classify tokens into several categories related to time and dates.

Architecture

The model utilizes BERT, a transformer model pretrained on a large corpus of English data in a self-supervised fashion. It performs token classification to categorize tokens into classes such as:

• O: Outside of a tag
• I-TIME/B-TIME: Inside/Beginning of a time tag
• I-DATE/B-DATE: Inside/Beginning of a date tag
• I-DURATION/B-DURATION: Inside/Beginning of a duration tag
• I-SET/B-SET: Inside/Beginning of a set tag

Training

Training Data

The model was trained using three data sources: Tempeval-3, Wikiwars, and Tweets datasets. For the correct data versions, refer to the associated GitHub repository.

Training Procedure

The model was fine-tuned from publicly available checkpoints on Hugging Face (bert-base-uncased) with a batch size of 34, a learning rate of 5e-05 using the Adam optimizer, and linear weight decay. Fine-tuning was performed with 5 different random seeds, and this model version used seed=4. The training utilized 2 NVIDIA A100 GPUs, each with 40GB of memory.

Guide: Running Locally

1. Installation: Ensure you have the transformers library installed.
2. Loading the Model:

   from transformers import AutoTokenizer, BertForTokenClassification
   
   tokenizer = AutoTokenizer.from_pretrained("satyaalmasian/temporal_tagger_BERT_tokenclassifier", use_fast=False)
   model = BertForTokenClassification.from_pretrained("satyaalmasian/temporal_tagger_BERT_tokenclassifier")
   

3. Inference:

   processed_text = tokenizer(input_text, return_tensors="pt")
   result = model(**processed_text)
   classification = result[0]
   

4. Post-Processing: Use functions from the provided GitHub repository for post-processing.

Cloud GPUs

For efficient model training and inference, consider using cloud GPU services such as AWS EC2 P3 instances, GCP instances with NVIDIA GPUs, or Azure NC series.

License

The model's license details can be found in the associated GitHub repository. Please refer to it for information on usage rights and limitations.