distilgpt2-finetuned-amazon-reviews

Introduction

distilgpt2-finetuned-amazon-reviews is a text generation model based on the DistilGPT2 architecture. It has been fine-tuned for tasks related to generating text, specifically causal language modeling, although further details on the intended applications and limitations are currently not provided.

Architecture

The model relies on the DistilGPT2 architecture, which is a distilled version of the larger GPT-2 model. This architecture is optimized for efficiency and speed, making it suitable for deployment in environments where computational resources may be limited.

Training

Training Procedure

The training of the distilgpt2-finetuned-amazon-reviews model involved the following hyperparameters:

• Learning Rate: 2e-05
• Training Batch Size: 8
• Evaluation Batch Size: 8
• Seed: 42
• Optimizer: Adam with betas (0.9, 0.999) and epsilon 1e-08
• Learning Rate Scheduler Type: Linear
• Number of Epochs: 3.0

Framework Versions

• Transformers: 4.8.2
• PyTorch: 1.9.0+cu102
• Datasets: 1.9.0
• Tokenizers: 0.10.3

Guide: Running Locally

1. Clone the Repository: Start by cloning the model repository to your local machine.

   git clone https://huggingface.co/defex/distilgpt2-finetuned-amazon-reviews
   cd distilgpt2-finetuned-amazon-reviews
   

2. Install Dependencies: Ensure you have all necessary packages and dependencies.

   pip install transformers torch datasets
   

3. Load the Model: Use the transformers library to load the model.

   from transformers import AutoModelForCausalLM, AutoTokenizer
   
   model = AutoModelForCausalLM.from_pretrained("distilgpt2-finetuned-amazon-reviews")
   tokenizer = AutoTokenizer.from_pretrained("distilgpt2-finetuned-amazon-reviews")
   

4. Run Inference: Generate text using the model.

   input_text = "This product is"
   input_ids = tokenizer.encode(input_text, return_tensors='pt')
   output = model.generate(input_ids)
   print(tokenizer.decode(output[0], skip_special_tokens=True))
   

Cloud GPUs

For enhanced performance, especially with large datasets or extended training, consider using cloud-based GPUs from providers such as AWS, Google Cloud, or Azure.

License

The licensing details for distilgpt2-finetuned-amazon-reviews are not specified in the provided information. Please check the model repository for specific licensing terms before use.