ASTEROID teaches partial-view agents to explore from experience

When an expert can see information that a learner cannot, copying the expert’s actions may not teach the learner how to uncover that information. Can a student learn to explore from its own interaction history while still benefiting from expert guidance? ASTEROID investigates this question for decision-making tasks with partial observations.

ASTEROID trains students on their own interaction histories, using a privileged expert’s action labels to learn exploration under partial observability.

ASTEROID addresses a specific question: can a student with partial observations learn to find hidden task information from an expert that already knows the hidden state?

The method has the student generate interaction histories and uses the expert’s privileged action labels to train the student on those contexts.

On discrete exploration tasks, ASTEROID achieved higher final performance than the evaluated baselines under the same test-time interaction budget. In simulated manipulation with limited sensing, it was about 100 times more efficient in environment steps than reinforcement-learning methods with privileged supervision. These results support learning exploration from student-generated histories in the tested settings.

The theoretical match to posterior sampling holds only under assumptions about expert labels, history coverage and exact learning, and only for histories covered by the collected data. The hardware result covers one cube-picking task and 10 trials.