SpeedRunners RL
Deep RL agents playing SpeedRunners via direct game injection
Overview
Most reinforcement learning research happens in sanitized sandboxes, simulators built specifically for AI. Real-world software isn’t so accommodating. This project bridges that gap by injecting a custom AI interface directly into SpeedRunners, a fast-paced multiplayer platformer. It uses C++ DLL injection and shared memory to turn a closed-source commercial game into a high-performance Gym environment, enabling agents to train using state-of-the-art algorithms like Rainbow IQN.
What is SpeedRunners?
SpeedRunners is a competitive side-scrolling platformer where 4 players race to outrun each other. The core mechanic is unique: the camera follows the leader, and anyone who falls off-screen is eliminated.
To survive, players must master a complex physics system involving grappling hooks, wall jumps, and boost management. It’s not just a race; it’s a battle. You can use items like missiles and crates to trip up opponents, and the screen shrinks over time to force a confrontation.
- Elimination Mechanics: Falling behind the camera means instant death.
- Physics-Based Movement: Momentum is key. Grappling hooks allow for high-speed swings and shortcuts.
Why This Matters for AI
SpeedRunners presents a perfect storm of challenges for an AI agent:
- Adversarial Real-Time Physics: Unlike turn-based games, decisions must be made in milliseconds (16ms per frame at 60 FPS).
- Continuous Action Space: While inputs are digital, the timing and duration of button presses create a effectively continuous control problem.
- Partially Observable: You can’t see the whole map. You only see what’s on screen, requiring the agent to memorize level layouts and anticipate opponent moves.
- No API: The game has no built-in AI interface. One had to be built from scratch by reverse-engineering the game’s memory.
The Engineering Challenge
Because SpeedRunners is a compiled commercial game, it doesn’t have a Python API. To train an agent, a bridge was required:
- Reverse Engineering: Cheat Engine and IDA Pro were used to find the memory addresses for player position, velocity, and game state.
- DLL Injection: A custom C++ library was written that gets injected into the running game process. This allows the system to read memory directly and overwrite input commands.
- High-Speed Inter-Process Communication (IPC): The C++ hook talks to the Python training script via named pipes, streaming state data at 60Hz with less than 10ms of latency.
Learn More
To dive deeper into the code, architecture, and implementation details, check out the project on GitHub: