ChatGPT help appeared in few research tasks, concentrated among workers
In plain words
Researchers wanted to know how often people use ChatGPT to help answer online research tasks. Earlier estimates relied on what people reported or on automated detection, so the authors linked donated ChatGPT histories to task records on Prolific.
They found ChatGPT help in about 1% of eligible task windows from 408 weekly ChatGPT users. Use was concentrated among a small group, so this rate describes those observed records, not every worker or other AI services.
The research
If participants use chatbots to supply answers, online research may measure model output as if it came from people. But estimates of how often this happens have varied and relied on self-report or automated detection.
How often did visible ChatGPT help actually appear in research tasks, and what did its distribution and payment records show?
The researchers linked donated ChatGPT histories to Prolific tasks and classified visible exchanges to estimate how often AI helped shape responses.
The researchers linked donated ChatGPT histories to Prolific task records and had a classifier assess visible exchanges in context. A task window counted as assisted if any exchange supplied content that could answer or materially contribute to a response.
They classified 1.00% of 712,930 eligible task windows as assisted. That directly observed, task-weighted rate supports the conclusion that assistance was uncommon in this cohort’s recorded activity.
These findings describe eligible tasks from a selected group of weekly ChatGPT users on Prolific. The researchers could not observe other AI services or verify whether ChatGPT output entered final responses, so the observed rate may not represent all workers or all AI assistance.
Analysis
Deep
- License
- CC BY 4.0