Ara B A R T

Introduction

AraBART is an Arabic language model based on the BART architecture. It is designed for tasks such as abstractive summarization, fill-mask, and feature extraction. AraBART is notable for being the first Arabic model with both its encoder and decoder pretrained end-to-end.

Architecture

AraBART is based on the BART-Base architecture, featuring:

• 6 encoder layers
• 6 decoder layers
• 768 hidden dimensions
• A total of 139 million parameters

Training

AraBART is pretrained to achieve high performance on multiple abstractive summarization datasets. It outperforms several strong baselines, including pretrained Arabic BERT-based models and multilingual models like mBART and mT5.

Guide: Running Locally

1. Setup Environment: Ensure you have Python and PyTorch installed.
2. Install Transformers Library: Run pip install transformers.
3. Download AraBART: Use the Hugging Face model hub to download AraBART.
4. Run Inference: Load the model and tokenizer from the transformers library and start making predictions.

For enhanced performance, consider using cloud GPUs, such as AWS EC2 with NVIDIA GPUs, Google Cloud GPUs, or Azure N-series VMs.

License

AraBART is distributed under the Apache-2.0 License.