Robotics Diffusion Transformer: Maniskill Model

Introduction

The Robotics Diffusion Transformer (RDT) is designed to facilitate simulation benchmarks in robotics. This repository provides models and datasets used for training and evaluation within the RDT framework.

Architecture

The RDT leverages precomputed language embeddings that are crucial for optimizing performance in simulation tasks. These embeddings are stored in the lang_embeds directory.

Training

Models within this repository have their weights stored in respective directories. These models are pre-trained and ready to be integrated into the RDT simulation workflow.

Guide: Running Locally

1. Dataset Setup:

   • Download the dataset parts: demo_1k_part_*.
   • Reconstruct and extract the dataset:

     cat demo_1k_part_* > demo_1k.zip
     unzip demo_1k.zip
     

2. Model Initialization:

   • Load model weights from the specified directories.
   • Utilize the precomputed language embeddings for initial setup.

3. Hardware Recommendations:

   • For optimal performance, utilize cloud GPUs such as those provided by AWS, Google Cloud, or Azure.

License

The content of this repository, including models and datasets, is subject to terms and conditions specified in the respective license agreement. Ensure compliance when using or distributing components from this repository.