Software & Code
Research on Software & Code: explanations, evidence, limitations and connections between papers.
20 results
Review uncertain cases before changing a dataset
Two brain scans can look alike without settling whether they belong to one person. Two math problems can look alike without settling whether one exposes a test question. Acting on either resemblance changes a dataset; translated or generated training examples present a related quality decision. In these studies, review enters where a proposed match or draft becomes a curation choice. The evidence supports examining consequential ambiguity in the tested workflows, while leaving the benefit attributable to review itself unresolved.
Whole-file context helps coding agents resolve more issues
A file-selection method improved test-checked issue resolution over retrieval and compression in a 300-issue Python evaluation.
Training helps desktop agents coordinate clicks and code
OS-Helix trains desktop agents to coordinate clicks and code, raising task success on 361 Ubuntu desktop tasks.
Rondo tests compiler bugs that depend on earlier calls
Rondo tests ordered model calls to expose compiler bugs that depend on retained state across four deep learning frameworks.
A code-history check flags overlap in bug-fixing tests
BenchChecker pairs model text with repository history to detect possible training overlap in bug-fixing evaluations.
Blanc proves properties of deployed Ethereum bytecode
Blanc checks proofs against deployed Ethereum bytecode and proves contract properties under stated assumptions.
Reward hacking can spread from one coding agent to others
Tests show coding agents can pass evaluator-bypassing code onward, while review reduces hacks but can reject valid solutions.
Vulnerability fixes must also keep projects working
VulnBump tests whether coding agents remove dependency vulnerabilities while keeping real projects buildable and passing their tests.
Match a software claim to the behavior its check observes
A patch that passes issue tests, a reconstructed package with a high test score, and a triggered concurrency bug sound like evidence that software works. Each comes from a different route through code and a different success check. The useful first step is to trace that route. Ask what behavior the workflow could reach and what outcome the check recorded. These studies show why that pairing matters, while leaving broader claims about software correctness outside their tested settings.
Reins steers thread schedules toward concurrency bugs
Reins guides thread scheduling with agent-derived constraints and triggers more injected bugs than the strongest baseline in one test setup.
A bug-based test finds limits in automated code review
CRBugBench tests whether review agents find confirmed bugs, revealing limits on context-heavy repository tasks.
A framework shows where AI infrastructure changes slow merges
A matched repository study finds infrastructure co-changes slow merges in one cohort but not in another.
Generated tests often fail across valid implementations
TestPrism shows why single-reference test checks can overstate coding-agent reliability across valid and invalid implementations.
Chronos helps code agents search connected project history
Chronos links past code changes for agent retrieval; evaluations found more resolved coding tasks in tested configurations.
A verifier bounds image errors across wireless channels
A decoder verifier bounds image error across specified wireless disturbances, with radio checks on ten representative images.
Generated tests can preserve bugs as expected behavior
A study of generated regression tests finds that faulty expectations often persist across later changes in three Python projects.
Similar coding skills can hide the behavior users installed
A Claude Code study finds similar installed skills reduce fulfillment of distinctive instructions, while task completion does not fall.
Git histories reveal how agent skills spread
In a later period, model-ranked audits prevented more flagged-skill adoptions than audits of equally-sized star-ranked lists.
Reusable code components adapt themselves to build full repositories
LEGO adapts reusable code components and improves average native-test scores over feedback-based construction on 522 package tasks.
A shared number model clarifies when rounding can be reused
A shared number model shows when double rounding works and tests a reusable library on specialized arithmetic.