Vision-Only Racing Driver

Driving from pixels alone is a classic control problem: the agent sees only a rendered chase-camera view and must infer the road, its speed and its position on the track. Here the game is Third Rally, a 3D browser arcade racer with a HUD, minimap and AI rivals, run headless in Chromium. Each race is three laps on a procedurally generated road circuit. The controller never sees game state. It must turn 640x360 screenshots into steering, throttle, drift and boost decisions that keep the car on track and finish quickly, measured against a driver with privileged game access. Perception, control and any training loop are the agent's responsibility, on four CPUs with no GPU.

The agent delivers a Python controller. Each race starts a fresh process that reads one JSON line per step (base64 PNG screenshot, step number, 0.1-second time step) and replies with keys to hold: accelerate, brake/reverse, steer left/right, drift and boost. Each reply advances six 60 Hz physics steps and must arrive within 30 seconds. A valid race follows the route for three laps without restarting, within 3,000 actions. The deployed driver may use only screenshots and its own action history. Learned weights must be saved in NumPy formats.

The offline sandbox (4 CPUs, 8 GB, no GPU, 8 hours) connects to the game on four public circuits, with CPU PyTorch, Gymnasium, Stable-Baselines3, NumPy and Pillow installed. A drive command runs the controller on a circuit and reports progress, completion, crashes, off-track time and latency. A train command runs a training script against a Gymnasium-style environment with up to 64 processes; the environment exposes progress and crash signals for learning, though the deployed driver does not receive them. The game files are never in the workspace, and the evaluation circuits are held out.

The submission races 24 held-out episodes: 12 unseen circuits from the same generator, each from two starting placements, under the same race and reply limits as practice. Each valid completion earns min(1, oracle time / controller time), where the oracle has privileged game access and times are simulated seconds excluding the countdown. Unfinished or invalid races earn zero, and the score is the mean over all 24 races.