Jasmine 350 M

Introduction

JASMINE is a collection of Arabic autoregressive Transformer language models designed for few-shot learning, developed by UBC-NLP. These models range from 300 million to 6.7 billion parameters and are pretrained on a 235 GB diverse text dataset. This repository accompanies the EMNLP2023 paper titled "JASMINE: Arabic GPT Models for Few-Shot Learning."

Architecture

JASMINE models utilize the GPT (Generative Pretrained Transformer) architecture, which is known for its autoregressive capabilities. The models are designed to handle a variety of tasks in Arabic text generation, leveraging the Transformer framework to enable few-shot learning capabilities.

Training

The training of JASMINE models was conducted on a vast and diverse dataset of Arabic text, totaling 235 GB in size. The models were refined using state-of-the-art techniques to handle few-shot learning scenarios, making them suitable for tasks that require minimal task-specific data.

Guide: Running Locally

To run the JASMINE models locally, follow these steps:

1. Environment Setup: Ensure you have Python and Pip installed. Create a virtual environment for the project.
2. Install Dependencies: Use the command pip install transformers torch to install the necessary libraries.
3. Model Download: Access and download the model from the Hugging Face model repository.
4. Inference: Load the model using the Transformers library and perform inference on your input data.

For optimal performance, especially with larger models, consider using cloud GPU services such as AWS EC2, Google Cloud Platform, or Azure.

License

JASMINE models and their accompanying resources are provided under terms that likely require citation in scientific publications. Users must refer to the specific license details provided in the repository for comprehensive information.