camembert sentiment allocine

Introduction

The CAMEMBERT-SENTIMENT-ALLOCINE model is a fine-tuned version of camembert-base, designed for sentiment analysis using the Allocine dataset. It is intended for text classification tasks in the French language and is compatible with TensorFlow.

Architecture

The model architecture is based on the CamemBERT transformer, a variant of the RoBERTa model, specifically adapted for the French language. For this particular version, the encoder weights have been frozen, and only the classifier head is trainable. This approach simplifies the fine-tuning process by limiting the training to the classifier head.

Training

The model was fine-tuned using the TensorFlow version of camembert-base, with the encoder component set to non-trainable. The training was completed in one epoch for testing purposes. The following hyperparameters were used:

• Optimizer: Adam
  • Learning Rate: PolynomialDecay with an initial rate of 5e-05, decaying over 15,000 steps.
  • Decay: 0.0
  • Beta Values: 0.9 and 0.999
  • Epsilon: 1e-07
  • Amsgrad: False
• Precision: float32
• Epochs: 1

The model achieved an accuracy of 0.918 on the test set. The framework versions used are Transformers 4.22.2, TensorFlow 2.8.2, Datasets 2.5.2, and Tokenizers 0.12.1.

Guide: Running Locally

1. Setup Environment: Ensure Python and pip are installed. Use a virtual environment for isolation.

   python -m venv env
   source env/bin/activate
   

2. Install Dependencies: Install the necessary libraries.

   pip install transformers tensorflow datasets
   

3. Download Model: Use the Hugging Face Transformers library to load the model.

   from transformers import TFAutoModelForSequenceClassification, AutoTokenizer
   
   model = TFAutoModelForSequenceClassification.from_pretrained("alosof/camembert-sentiment-allocine")
   tokenizer = AutoTokenizer.from_pretrained("camembert-base")
   

4. Inference: Tokenize your input text and perform inference.

   text = "Votre avis ici"
   inputs = tokenizer(text, return_tensors="tf")
   outputs = model(inputs)
   

5. Cloud GPUs: For faster processing, consider using cloud services like AWS, GCP, or Azure that offer GPU instances.

License

The CAMEMBERT-SENTIMENT-ALLOCINE model is released under the MIT License, allowing for wide usage and modification.