M L Agents Snowball Fight 1vs1

Introduction

"Snowball Fight 1vs1" is a multi-agent environment developed using the Unity ML-Agents Toolkit, where two agents compete in a snowball fight. The environment is designed to facilitate reinforcement learning studies and can be accessed online via Hugging Face Spaces.

Architecture

The environment consists of two agents in a 1vs1 snowball fight game. The observation space includes ray-casts and vector observations to detect opponents, walls, snowballs, and more. The action space is discrete, allowing for movement and shooting actions. The reward function is designed to incentivize hitting opponents while avoiding being hit.

Observation Space

• Ray-casts:

  • 10 forward ray-casts for opponents.
  • 10 forward ray-casts for walls, shelter, and frontier.
  • 10 forward ray-casts for snowballs.
  • 3 backward ray-casts for walls and shelter.

• Vector Observations:

  • Bool canShoot: available every 2 seconds.
  • Float currentHealth: normalized [0, 1].
  • Vector3 for vertical and horizontal speed.
  • Vector3 for "home" position.

Action Space

• Vector Action Space:
  Four branched actions for movement, rotation, and shooting.

Agent Reward Function

• Points for shooting the opponent.
• Time penalty reduced when hitting the opponent.
• Penalty when hit by a snowball.

Training

The model was trained for 5,100,000 steps, achieving a final ELO score of 1766.452. The training process utilizes the PPO algorithm with specific hyperparameters such as batch size, buffer size, learning rate, and more as outlined in the config file.

Guide: Running Locally

To run the environment locally, follow these steps:

1. Clone the Repository:

   git clone https://huggingface.co/ThomasSimonini/ML-Agents-SnowballFight-1vs1
   

2. Set Up Unity Project:

   • Open Unity Hub and create a new 3D project.
   • Copy manifest.json and package.lock.json from the cloned project folder to Your Unity Project\Packages.

3. Import Unity Package:

   • Drop the SnowballFight-1vs1 Unity package into your Unity project.

4. View Trained Agents:

   • Open Assets\1vs1\Scenes\1vs1_v2_Training.
   • Place the ONNX model into BlueAgent and PurpleAgent Model slots.

5. Training:

   • To retrain, use the same scene Assets\1vs1\Scenes\1vs1_v2_Training.

Suggested Cloud GPUs

Consider using cloud GPU services such as AWS, Google Cloud, or Azure to handle computationally intensive tasks.

License

The project is licensed under the Apache 2.0 License.