Crypto B E R T

Introduction

CryptoBERT is a fine-tuned version of the ProsusAI/finbert model designed to perform sentiment analysis on cryptocurrency-related texts. It predicts whether the sentiment of a given text is positive or negative, achieving a loss of 0.3823 on the evaluation set.

Architecture

CryptoBERT is based on the BERT architecture, specifically fine-tuned from the ProsusAI/finbert model, which is pre-trained for analyzing financial text sentiment. The fine-tuning process adapts it to the cryptocurrency domain.

Training

The model was trained using the following hyperparameters:

• Learning rate: 5e-05
• Train batch size: 16
• Eval batch size: 8
• Seed: 42
• Optimizer: Adam with betas=(0.9,0.999), epsilon=1e-08
• Learning rate scheduler: Linear
• Number of epochs: 10

Training results showed a progressive decrease in training loss across epochs, with a final loss of 0.3823.

Guide: Running Locally

To run CryptoBERT locally, follow these steps:

1. Install Dependencies: Ensure you have Python installed, then install the necessary libraries:

   pip install transformers torch
   

2. Load the Model and Tokenizer:

   from transformers import BertTokenizer, BertForSequenceClassification, pipeline
   
   tokenizer = BertTokenizer.from_pretrained("kk08/CryptoBERT")
   model = BertForSequenceClassification.from_pretrained("kk08/CryptoBERT")
   classifier = pipeline("sentiment-analysis", model=model, tokenizer=tokenizer)
   

3. Perform Sentiment Analysis:

   text = "Bitcoin (BTC) touches $29k, Ethereum (ETH) Set To Explode, RenQ Finance (RENQ) Crosses Massive Milestone"
   result = classifier(text)
   print(result)
   

For optimal performance, especially with large datasets or high-throughput requirements, consider using cloud GPUs such as AWS EC2 instances or Google Cloud Compute Engine with GPU support.

License

The model and its code are provided under the Apache 2.0 License, allowing for both personal and commercial use, modification, and distribution with proper attribution.