Hyperbolic distance helps agents navigate maze routes

In a maze, a goal can be nearby as the crow flies but distant along a route around walls. Euclidean agents can learn this route structure, but may need much deeper networks. Could changing the critic's distance measure help an agent represent routes more directly?

Hyperbolic CRL gives goal-reaching agents a critic whose distance measure better represents branching routes around maze walls.

Can a goal-reaching agent represent how far a target is along a route when walls make straight-line distance misleading? The authors introduce Hyperbolic CRL, which retains contrastive reinforcement learning but gives its critic a distance measure with a bias toward tree-like structure.

In online tests across 10 environments, depth-16 hyperbolic agents improved over Euclidean agents on many tasks, especially two difficult Humanoid mazes.

This supports the idea that the geometry can help goal-reaching where walls bend routes, while the task-dependent curvature shows that one setting does not fit every environment.

The results cover the tested environments and settings. The best curvature varies by task, and the critic's symmetric distance cannot represent one-way transitions.