#evaluation
Research tagged evaluation: explanations, evidence, limitations and connections between papers.
11 results
AI weather forecasts retain skill beyond local heat records
A 2022 comparison finds AI weather systems retain heat forecast skill beyond local records, with less consistent results for wind.
A shared HTTP guard makes Java model calls more consistent
A shared Java HTTP guard made three client paths behave alike in scripted provider-failure tests, with a slow-stream timeout trade-off.
Training-time averaging leads across most MEG tasks
A MEG study finds training-time averaging leads in six of seven tested task and group-size conditions, with an imagined-digit exception.
Future-frame clues help neural codecs compress video
GFSC adds limited future-frame context to video codecs, reducing bitrate at comparable picture quality in tested models and datasets.
Legal search improves when chunks preserve section structure
StrucChunk preserves legal section structure; retrieval tests show gains on CrPC and a mixed result against hierarchy-aware chunks in Thai law.
Inherited software work exposes a gap in coding agents
SWE-Residual finds lower coding-agent success on inherited work than matched clean starts across repository and machine tasks.
When search results arrive in a different order, agents seek different evidence
Confluent Evidence Acquisition keeps fixed-plan search outcomes stable across delivery orders under stated retrieval and response assumptions.
Token choice can change measured hallucination scores
A POPE audit finds that tokenizer-aware yes/no extraction changes measured scores, while fixed token IDs fail on some models.
REX moves hazardous knowledge into removable experts
REX moves hazardous knowledge into removable experts; after training, it scored lowest among usable methods in six tested settings.
PACE-Bench checks whether coding agents deliver full requests
PACE-Bench evaluates full PRD delivery in existing repositories; the top tested configuration completed 6 of 43 cases.
A camera system shares edge work and adapts models centrally
YUTELA shares camera processing at the edge and supports central updates, but accuracy gains remain unmeasured.