C L I P Vi T B 32 xlm roberta base laion5 B s13 B b90k

Introduction

The CLIP ViT-B/32 xlm roberta base model is a machine learning model designed for tasks such as zero-shot image classification and image-text retrieval. It is developed using the LAION-5B dataset and OpenCLIP framework and was trained on the stability.ai cluster.

Architecture

This model combines the CLIP ViT-B/32 architecture on the visual side and XLM-Roberta base architecture on the text side. It is initialized with pre-trained weights, optimized for multilingual capabilities.

Training

The training utilized the LAION-5B dataset with a batch size of 90k for a total of 13 billion samples. Evaluation was performed using the LAION CLIP Benchmark suite, employing datasets like VTAB+, COCO, and Flickr for testing. The model has shown competitive performance on several benchmarks, including ImageNet, MSCOCO, and Flickr30k.

Guide: Running Locally

1. Clone the Repository: Clone the OpenCLIP repository from GitHub.

   git clone https://github.com/mlfoundations/open_clip.git
   cd open_clip
   

2. Set Up Environment: Install the necessary packages.

   pip install -r requirements.txt
   

3. Download Model Weights: Use the Hugging Face model hub to download the model weights.

4. Run the Model: Execute the provided scripts to perform tasks like zero-shot classification.

Cloud GPUs: For resource-intensive tasks, using cloud GPU services like AWS EC2, Google Cloud, or Azure is recommended to leverage high-performance computing resources.

License

The model is licensed under the MIT license, which allows for flexibility in usage, modification, and distribution.