gpt2 base french LLM Model

Introduction

The GPT-2 Base French model is designed for text generation tasks in the French language. It is based on the GPT-2 architecture and has been fine-tuned on specific French datasets to enhance its performance in generating coherent French text.

Architecture

The model follows the GPT-2 base architecture but with a modified vocabulary size. The tokenizer employed is a Byte Pair Encoding (BPE) tokenizer, featuring a vocabulary size of 50,000 tokens.

Training

The model was trained using:

wiki40b/fr: A French Wikipedia dataset.
CC-100/fr: A subset of monolingual datasets derived from web crawl data.

These datasets were selected to ensure a diverse range of textual data for training, offering the model a broad understanding of the French language and its nuances.

Guide: Running Locally

To run the GPT-2 Base French model locally, follow these steps:

Install Prerequisites: Ensure you have the transformers library version 4.19.2 installed.
```
pip install transformers==4.19.2
```

Run the Model: Use the following Python code snippet to generate French text:

from transformers import pipeline

generator = pipeline('text-generation', model='ClassCat/gpt2-base-french')
print(generator("Je vais à la", max_length=50, num_return_sequences=5))

Consider Cloud GPUs: For better performance and faster inference, consider using cloud-based GPU services such as AWS, Google Cloud, or Azure.

License

This model is released under the CC BY-SA 4.0 license, allowing for sharing and adaptation with attribution.

More Related APIs in Text Generation