In plain words
Language models can identify a phrase in a passage but leave it unclear which occurrence they mean, or change the wording so it cannot be found. This paper introduces a training-free method that makes a model copy the passage and put a label around each phrase it selects.
Across six datasets, the method scored higher than tagging with the same prompts but no copying constraint on five datasets, and lower on one. Copying was exact whenever generation finished.
The research
When a language model identifies a phrase in a passage, how can we tell exactly which occurrence it means? A generated phrase may be changed or repeated in the input, making it hard to locate.
The paper asks whether a model can mark spans directly in a copy of the text so their positions are fixed as it generates them.
Constrained tagging makes a language model copy the input and mark spans inline, fixing their character locations as the labels are generated.
The method constrains a model to copy the input and add valid inline labels, so each span’s position is fixed as it is generated. The authors compared it with tagging that used the same prompts and markup without the constraint, measuring hard character-overlap F1, which requires the predicted label to match.
At batch size 1, the three-seed mean was higher with constrained tagging on five of six datasets and lower on Toxic Spans. This supports a benefit on most tested datasets, while the copy guarantee itself depends on generation finishing.
The method fixes where completed spans are located, but it does not ensure the model selects the right spans or boundaries. Results are based on the tested models, datasets and prompts; small score differences were not tested for statistical significance.
Analysis
Deep
- Original paper authors
- Radek Štulc, Jan Drchal
- Submitted to arXiv
- License
- CC BY 4.0