Duke Scout logs show how clinicians used an EHR tool
Real-World Adoption, Use, and Governance of an LLM-Based Electronic Health Record Search and Synthesis Platform · arXiv:2611.08672
In plain words
Scout is a tool inside Duke Health's electronic health record. It answers care practitioners' questions about one patient's record and links its claims to source data. Researchers looked at logs to see what people asked it to do.
Across 80,465 logged tasks, most were classified as intended for direct patient care, and summarizing records was the most common requested operation. Many saved prompts were used, but only two had completed health-system approval by the study's end. The logs show what people requested, not whether the answers helped patients.
The research
For a health system weighing what broad access means, raw usage totals can hide whether a tool is used for patient care and whether reusable tasks are ready to spread.
This study makes those questions visible in one deployment, while showing that frequent use alone does not mean a workflow has completed system review.
Scout is a Duke EHR platform, and this study uses its task logs to map requested work and track how saved tasks reached governance review.
The researchers analyzed eligible Scout task logs from September 2025 to August 2026, classifying prompts by intended use and requested operation. Across 80,465 tasks, 92.5% were classified as direct clinical care and 40.4% requested summarization or synthesis.
The distributions describe what users asked for, while saved-task counts show how few reusable workflows had completed system approval.
The logs record requested tasks, not how well Scout performed them or whether they improved care. This was one health system, with only 29 days of follow-up after system-wide access; the prompt classification framework's overall accuracy was not measured.
What the evidence shows
- What the logs can show
Scout is a Duke Health electronic health record platform that answers a care practitioner's question about one patient's record with claims linked to source data. The study asks what happened as use grew: which work people requested, and whether saved tasks reached wider review.
Researchers examined task logs, so the evidence describes recorded use rather than a new test of answer quality or patient outcomes.
- Care dominated requests
Researchers classified 92.5% of eligible task events as primarily for direct clinical care. That makes patient care the dominant intended use in the observed requests, but says nothing about answer quality or clinical effect.
- Summaries led operations
The researchers also classified what operation users requested. Summarization or synthesis was the most frequent, at 40.4% of eligible task events.
Together, the two distributions show that the logs capture both why Scout was used and what users asked it to do; neither percentage measures how well Scout completed the task.
- Use and approval diverged
Saved prompts were used repeatedly, but their reach did not mean they had cleared broader review. By the study's end, 752 SmartTasks had been invoked 15,322 times, while two had completed health-system approval.
The study therefore documents substantial recorded use alongside limited completed approval, not a validated link between approval status and task quality.
- The evidence has boundaries
These observations help characterize one deployment and the work requested within it. The log study did not reassess task performance or measure clinical outcomes, and post-launch follow-up after system-wide access lasted 29 days.
Its prompt classification framework also lacked an estimate of overall accuracy, which limits how firmly its categories can be treated as measured prevalence.
Deep
- Original paper authors
- Jiunn-Tyng Yeh, Michael Gao, Marshall Nichols, Anika Mohan, Tareq Aljurf
- Submitted to arXiv
- License
- Not specified by source