ai agents
Research on ai agents: explanations, evidence, limitations and connections between papers.
14 results
MagicBeanStalk builds training problems from hard seeds
MagicBeanStalk builds model-calibrated math training problems, improving average test accuracy in reported evaluations.
What Evidence Supports an AI Model Choice?
A home assistant can use a different model for each kind of request, but a choice that looks sound in a benchmark may fail when it reaches a real device. Image tests face a related problem: missing conditions can make the wrong model look preferable. Across four studies, evidence sits at different distances from the decision. Test sets and live trials examine outcomes; statistics from pretrained models help rank options earlier. Their useful answer is bounded by the conditions and behavior each study actually checked.
TacitVLA trains on reasoning so actions need not wait for its generated text
TacitVLA separates reasoning training from action generation, matching explicit reasoning across four manipulation and driving tests.
Checking each action can stop agents’ unauthorized acts
In mock-tool episodes, checking each proposed action reduced violations while matched tests tracked whether authorized work continued.
Sauron checks evidence before judging agent tasks
Sauron checks task states and action records, improving rubric judgments in tests of simulated and recorded agent tasks.
A harness-editing method improves transfer to unseen tasks
A method regulates agent harness edits with a frozen model and improves average scores on unseen workspace tasks.
Written skills improve agent procedures without changing weights
SkillOpt improves frozen agents through validated text edits, with gains across tested models, tasks, and execution modes.
A human-trained user simulator improves agent training
MIMESIS models varied human behavior; agents trained with it score better than GPT-5.5-trained agents across nine simulated users.
A graph result powers faster structured language decoding
A GPU decoding engine recomputes format masks and speeds six JSON extractions on one tested laptop.
CanonQ trains language models to use one shared low-bit codec
CanonQ improves measured accuracy and text prediction for a tested language model under joint low-bit compression.
Home automation can use different models for different calls
A live test of per-call model choices matched all-hosted human judgments on 39 of 43 counted cases, at lower hosted spend.
Visible labels can mislead language model judges
Tests find that visible system labels can distort language model judgments of entity alignment decisions in the studied conditions.
RT-SAFE reveals safety failures hidden by navigation success
RT-SAFE tests urban-agent safety during decisions; hard-route runs exposed risks hidden by successful arrivals in a changing world.
Why language models can overstate their progress
A statistical model and loop tests show why reused examples can make language-model progress scores too optimistic.