Interactive tools help agents check web accessibility more reliably

In plain words

The research

What the evidence shows

• The checking gap

  InterA11y is a web accessibility evaluation system that helps language-model agents assess 10 selected Web Content Accessibility Guidelines (WCAG) criteria. Existing checkers often inspect static page information, although some criteria concern behavior that appears only during interaction.

  Their results can also vary across language models. The paper asks whether rubric-guided agents equipped to gather interactive evidence can assess these criteria more reliably.

• Rubrics guide evidence

  The system uses criterion-specific rubrics to break accessibility rules into testable aspects, with guidance on the evidence and decision method for each one. Browser tools inspect page information and rendered appearance, measure properties, operate controls, and monitor responses.

  The pipeline gathers initial evidence, identifies candidate elements, applies conclusive deterministic checks, then lets the agent gather and interpret additional evidence before returning a verdict for each candidate.

• Two components add value

  The authors first compared system versions on 682 curated conformance cases covering 72 rules across 10 WCAG criteria. Adding InterA11y rubrics to GenA11y increased mean recall, the share of labeled errors detected, from 31.0% to 64.2%.

  Adding interactive tools to the rubric condition reduced mean false discovery rate, the share of flagged errors that were false positives, from 26.7% to 9.4%. The same language models and configurations were used across these ablation conditions.

• Evidence from webpages

  The authors then tested checkers on 197 elements labeled by accessibility experts and authors from sampled webpages saved as offline copies. InterA11y detected a larger share of labeled errors than the other tested checkers.

  Its false discovery rate was statistically similar to axe and a rule ensemble. This comparison suggests the system can find more labeled errors while maintaining a similar false-alarm rate to those two checkers in this evaluation.

• Tools also help other agents

  In follow-up tests on the expert-labeled elements, giving InterA11y tools to AgentLab and Browser Use raised each agent’s mean recall and lowered its mean false discovery rate. This supports reuse of the tools with different web agents under the tested conditions.

  The evidence remains bounded: the curated cases cover selected criteria, and the webpage evaluation used offline copies that excluded elements requiring dynamic server resources.