AI & Agents
Research on AI & Agents: explanations, evidence, limitations and connections between papers.
47 results
A testbed measures coding-agent gains on science models
Agent4Sci measures coding-agent gains on scientific ML tasks, while audits and separate test scoring guard against evaluation gaming.
Chat refusals often fail to prevent unsafe tool calls
A matched-request study finds chat refusals often fail to prevent unsafe tool calls across five tested language models.
World models can miss events in frames that look accurate
A transition-level diagnostic finds reward and ending events often drive return error in reliable Atari world-model comparisons.
A benchmark measures when reassurance leaves advice goals unmet
A diagnostic test finds that conversational models vary in leaving practical goals unmet across authored advice scenarios.
Aligning context fields improved industrial app ranking
EASformer aligns app histories with candidate context, improving offline ranking and eCPM in two tested online scenarios.
Agent S4 evolves computer-use tools from task experience
Agent S4 evolves reusable computer-use tools from task attempts, improving results in tests of general and scientific software tasks.
Score-guided sampling improves sparse attention
Stratified Sparse Attention uses score-guided sampling to improve sparse attention results in selected long-context language-model tests.
Inflated tool listings sometimes mislead agent buyers
A simulated tool-market study finds inflated seller listings can lower buyer task success in severe cases.
WildAWM creates computer worlds for cross-interface agent training
WildAWM creates cross-interface computer tasks; trained Qwen models improve on average across eight external task tests.
Tandem speeds hybrid phone-agent model serving
Tandem adapts hybrid model serving to changing phone-agent requests, reducing average step latency on a Snapdragon 8 Elite trace.
A price sweep reveals how seller cues sway AI shopping
Price-equivalent auditing measures how much seller cues offset product prices, revealing differences hidden by equal-price choices.
A model can see a failed step yet trust the agent’s record
EviGate checks agent decision states and improved task success across three tested settings with calibration from each matching domain.
How simulator report delivery changes agents’ answers
On verified PHREEQC questions, report retrieval lowered accuracy for two GPT agents; naming the report improved evidence use.
How shared memory can lead agents into the wrong account
A shared-memory audit separates credential exposure from account misuse and finds owner wording reduced misuse in controlled tests.
Place each trace where it can inform the next decision
A robot may need to choose the next object to move, an application judge may need to see what happens after a click, and a tool-using agent may need review before changing a record. Each can produce a trace, but the trace helps a different decision. The studies suggest placing evidence where that decision occurs. One robot method feeds reasoning into control; another moves text generation off the immediate action path. Application interaction informs a judge, while a grounded tool-use draft reaches a reviewer before a change. Their reported task success and timing results belong to their separate tests.
A causal method traces ad engagement to chosen labels
D-VAE estimates how marketing labels affect engagement through generated ads, with a small RED user experiment showing higher engagement for its recommendations.
Separate GPU paths speed prompt and token processing
FuTransformer tailors Llama-style inference to Hopper GPUs; tests show faster paths with a measurable INT4 text-prediction cost.
A graph monitor transfers hallucination detection across language models
LEMON uses attention graphs to detect hallucinated tokens on unseen models, outperforming probability scores in tested settings.
Canary contracts reveal where safety evidence gets lost
Canary contracts compare repaired framework paths to show when safety evidence reaches an enforcement decision.
Dia calls other models only when reasoning answers disagree
Dia uses answer disagreements to trigger independent solving, outperforming full chain-of-thought in most tested model and test settings.
Arcade lets agents request tools from a changing catalog
Arcade’s tool recommendation system ranked complete tool sets more often than BM25 in a same-harness test of tool queries.
WebCoder trains one agent to handle changing frontend projects
WebCoder trains a frontend agent to meet changing requirements, with higher scores than its base model on two evaluated task suites.
A customer-delivery benchmark finds agents struggle to build effective systems
FDE-Gym finds that AI agents struggle to discover customer constraints and turn gathered facts into effective delivery systems.
Attention statistics can be extracted without storing full matrices
A Triton backend extracts attention summaries without full matrices; added features improved detection in some tested dataset–proxy pairs.