How coding-agent context changes server work
The Physical Cost of Context: Quality-Constrained Resource Accounting for Coding Agents · arXiv:2610.34763
In plain words
Coding agents send information about a software project to a model each time they ask it to act. A shorter message may still take more work if editing earlier messages prevents the server from reusing information it already processed.
The paper introduces a method that checks task quality and counts both model calls and input the server must process again.
In separate localization tests, giving the agent retrieved code early passed the quality check and used fewer calls and less measured GPU energy than searching from the issue alone.
The research
When coding agents repeatedly send project context, does trimming their prompts reliably reduce the work a server performs? The answer may depend on whether earlier input can be reused and whether a policy changes how many requests the agent makes.
The authors introduce a quality-gated accounting method that separates prompt size from server work, then test how retrieval, history edits and caching change that work.
The authors compare Grep, which starts localization from an issue, with Eager, which supplies ten retrieved code spans before the first turn; both keep full history.
On 450 Python issues, Eager’s Recall@5 was 0.478 versus Grep’s 0.470, and its paired quality difference cleared the five-point non-inferiority gate.
In a separate, cache-disabled resource cohort, Eager used fewer calls and less GPU-energy proxy per attempted task.
Together, these results support early evidence delivery as a way to reduce measured work in this setup, while the accounting shows why prompt-token totals alone cannot establish savings.
The quality and detailed resource results for localization come from separate cohorts, and the patch study used a fixed 32-call budget. Tests covered four fixed policies on one device, serving one request at a time; they do not establish results for batching, cloud GPUs, or other policy designs.
What the evidence shows
- Ask what work is saved
The paper introduces a quality-gated resource-accounting protocol for comparing context policies used by coding agents. The question is whether shorter prompts reliably mean less work when agents make repeated requests and servers can reuse earlier input.
The protocol checks task quality before comparing resource costs, then measures model calls, uncached input processing, cache occupancy, time, and a GPU-energy proxy.
- Check the quality gate
In localization, Grep starts from the issue alone, while Eager supplies ten retrieved code spans before the agent’s first turn; both retain full history. Success means the target function or class appears among the five returned code blocks.
Across 450 Python issues from distinct repositories, Eager’s score was 0.478 versus 0.470 for Grep. Its paired quality difference passed the authors’ five-point non-inferiority gate.
- Measure the localization costs
The authors measured resources on a separate 100-task localization cohort, with prefix caching disabled and failed tasks still charged. There, Eager used fewer successful server calls and less GPU-energy proxy per attempted task than Grep.
This supports the interpretation that, in this comparison, supplying evidence early reduced work by avoiding some search calls; it does not combine quality and cost measurements from the same tasks.
- Test reuse and history edits
Two further comparisons show why prompt length is an incomplete guide. With Eager held fixed, enabling prefix caching left logical prompt volume statistically similar while sharply reducing uncached prefill.
In a different patch-generation setup, Mask-5 shortened prompts but failed the quality gate and sharply increased uncached prefill and measured GPU energy. The results show that reuse and agent behavior can change the relationship between prompt size and executed work.
Deep
- Original paper authors
- Zhongming Yu, Yitong Ma, Hengjia Yu, Boqin Yuan, Xiangye Lin
- Submitted to arXiv
- License
- arXiv non-exclusive