Rocket League RL Bot
An offline Rocket League bot being trained with reinforcement learning, in progress: custom reward shaping and training loop in PyTorch, using a mocked Unreal Engine environment for sample-efficient training.
Reinforcement learning only really clicks when you watch an agent learn something you didn’t explicitly teach it. So I’m training an offline Rocket League bot with RLGym and PyTorch, and doing it the hard way: my own reward shaping and my own training loop.
The setup
- Environment: RLGym exposes Rocket League as a Gym-style RL environment: car and ball state in, controller actions out.
- Sample efficiency: training leverages a mocked Unreal Engine environment, so the agent can collect experience much faster than the real game loop would allow.
- Learning: a custom training loop in PyTorch, built to make experimenting with reward functions fast.
Reward shaping is the whole game
Left with only “score a goal,” an agent almost never stumbles into reward. The real work is designing rewards that guide without dictating: incentives for moving toward the ball and making contact, shaped so they can be annealed away as the policy improves. Getting that curriculum right is the core of the project.
Status
WIP. The environment and training loop are running; the current phase is iterating on reward design and watching what the agent discovers. It’s already been the best hands-on lesson in how sensitive RL is to reward and hyperparameter choices, lessons that map directly onto controls work on the car.