Review uncertain cases before changing a dataset

Two brain scans can look alike without settling whether they belong to one person. Two math problems can look alike without settling whether one exposes a test question. Acting on either resemblance changes a dataset; translated or generated training examples present a related quality decision.

In these studies, review enters where a proposed match or draft becomes a curation choice. The evidence supports examining consequential ambiguity in the tested workflows, while leaving the benefit attributable to review itself unresolved.

Imagine curating a pooled collection of brain scans. Two images arrive under different participant identifiers and appear similar.

Joining their records might reconcile repeat scans of one person, but a resemblance alone does not settle identity. A curator of math training data faces a parallel choice: a problem resembling a benchmark question might be a copy or a distinct problem with changed numbers. What evidence is enough to change the dataset?

The brain-scan problem arises because identifiers assigned within separate studies can refer to the same person once those studies are pooled. An image-based search can surface records worth checking, but the repository ultimately needs a decision about which records belong together.

The question is consequential: joining records changes the repository’s account of its participants. The MRI study therefore places a review step between a suggested match and a reconciled identity.

Other curation choices have different consequences. A math-data curator may delete a training example after a similarity search flags it.

A team making multilingual instruction examples may keep a translated or generated response after checking its language and fit to the instruction. Across these cases, the shared question is where judgment belongs when an automated step has produced a plausible proposal but has not settled whether to accept it.