distilbert base turkish cased allnli_tr

Introduction

The distilbert-base-turkish-cased-allnli_tr model is a fine-tuned version of the dbmdz/distilbert-base-turkish-cased model, specifically tailored for zero-shot classification tasks in Turkish. This model is designed to classify text into categories without explicit training on those categories, making it versatile for various applications.

Architecture

The model is based on the DistilBERT architecture, a lighter and faster variant of BERT that maintains a balance between performance and computational efficiency. The distilbert-base-turkish-cased serves as the base model, ensuring compatibility with Turkish text through its cased nature.

Training

Training involved fine-tuning the base model with the following hyperparameters:

• Learning Rate: 2e-05
• Train Batch Size: 32
• Eval Batch Size: 32
• Seed: 42
• Optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• Learning Rate Scheduler: Linear
• Number of Epochs: 3

The model achieved an evaluation loss of 0.6481 and an accuracy of 0.7381 on the evaluation dataset.

Guide: Running Locally

To run this model locally, follow these steps:

1. Setup Environment: Ensure Python and PyTorch are installed. You can also use virtualenv to create a dedicated environment.
2. Install Required Libraries: Use pip to install the required packages:

   pip install transformers torch
   

3. Download the Model: Use Hugging Face's Transformers library to load the model:

   from transformers import pipeline
   
   classifier = pipeline("zero-shot-classification", model="emrecan/distilbert-base-turkish-cased-allnli_tr")
   

4. Run Inference: Use the classifier to predict labels for the input text:

   result = classifier("Dolar yükselmeye devam ediyor.", candidate_labels=["ekonomi", "siyaset", "spor"])
   print(result)
   

For optimal performance, especially with larger datasets, consider using cloud GPUs from providers like AWS, GCP, or Azure.

License

This model is licensed under the Apache-2.0 License, allowing for both personal and commercial use with compliance to the license terms.