tqc Panda Reach v1

Introduction

The TQC (Twin Delayed DDPG with Critic) agent is a reinforcement learning model trained to play PandaReach-v1 using the Stable Baselines3 library. This model utilizes the RL Zoo, a framework designed for training reinforcement learning agents with optimized hyperparameters and pre-trained agents.

Architecture

The TQC agent employs the Stable Baselines3 library and the RL Zoo for training. It operates within the PandaReach-v1 environment, a part of the Panda Gym suite, designed for reinforcement learning tasks. The model is structured to utilize a MultiInputPolicy with a network architecture of two layers of 64 neurons each and one critic.

Training

Training is performed using the RL Zoo framework. The process involves executing a Python script to train the TQC agent in the PandaReach-v1 environment. The training script uses specific hyperparameters for optimal performance, including a batch size of 256, a buffer size of 1,000,000, and a learning rate of 0.001. The training involves 20,000 timesteps, with normalization applied to observations but not rewards. The model uses a HerReplayBuffer with online sampling and a future goal selection strategy.

Guide: Running Locally

1. Clone the RL Zoo and Stable Baselines3 Repositories:

   • RL Zoo: https://github.com/DLR-RM/rl-baselines3-zoo
   • Stable Baselines3: https://github.com/DLR-RM/stable-baselines3

2. Download and Save the Model:

   python -m rl_zoo3.load_from_hub --algo tqc --env PandaReach-v1 -orga sb3 -f logs/
   

3. Run the Model:

   python enjoy.py --algo tqc --env PandaReach-v1 -f logs/
   

4. Training Locally:

   python train.py --algo tqc --env PandaReach-v1 -f logs/
   

5. Upload Model and Generate Video:

   python -m rl_zoo3.push_to_hub --algo tqc --env PandaReach-v1 -f logs/ -orga sb3
   

For optimal performance, consider using cloud GPUs to handle the computational demands of training and running the model.

License

The project is open-source and can be used and modified under the licenses associated with the Stable Baselines3 and RL Zoo libraries. For more detailed licensing information, refer to the respective repositories on GitHub.