klue roberta small 3i4k intent classification

Introduction
The KLUE-ROBERTA-SMALL-3I4K-INTENT-CLASSIFICATION model is a text classification model specifically designed for intent classification tasks. It was developed by Jaehyeong at Bespin Global and is part of the Hugging Face Model Hub. The model is fine-tuned using the KLUE benchmark's Roberta-Small as its base and is intended for classifying Korean text.

Architecture
The model is based on the RoBERTa architecture, a transformer-based model optimized for natural language processing tasks. It is specifically fine-tuned for intent classification, utilizing the Korean language dataset 3i4k, which includes various categories such as statements, questions, commands, and rhetorical forms.

Training

• Pretrained Model: KLUE Roberta-Small
• Fine-tuning Dataset: 3i4k
  • Training Set: 46,863 examples
  • Validation Set: 8,271 examples
  • Test Set: 6,121 examples
• Label Categories: fragment, statement, question, command, rhetorical question, rhetorical command, intonation-dependent utterance
• Training Parameters:
  • Epochs: 3 (early stopped, originally set to 10)
  • Batch size: 32
  • Optimizer: Adam with a learning rate of 5e-05

Guide: Running Locally
To run this model locally, follow these steps:

1. Install the necessary libraries:

   pip install transformers torch
   

2. Load the model and tokenizer:

   from transformers import RobertaTokenizerFast, RobertaForSequenceClassification, TextClassificationPipeline
   
   HUGGINGFACE_MODEL_PATH = "bespin-global/klue-roberta-small-3i4k-intent-classification"
   loaded_tokenizer = RobertaTokenizerFast.from_pretrained(HUGGINGFACE_MODEL_PATH)
   loaded_model = RobertaForSequenceClassification.from_pretrained(HUGGINGFACE_MODEL_PATH)
   
   text_classifier = TextClassificationPipeline(
       tokenizer=loaded_tokenizer, 
       model=loaded_model, 
       return_all_scores=True
   )
   

3. Make predictions:

   text = "your text"
   preds_list = text_classifier(text)
   best_pred = preds_list[0]
   print(f"Label of Best Intentation: {best_pred['label']}")
   print(f"Score of Best Intentation: {best_pred['score']}")
   

4. Cloud GPUs: For faster processing, consider using cloud-based GPU services like AWS EC2, Google Cloud, or Azure.

License
The model is licensed under the Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0). This means you can use, distribute, and build upon the work non-commercially, as long as you credit the original creator.