A hospital AI platform runs with its tools inside a closed network
The LLENS Platform for On-Premises Large Language Model Agents in a Japanese University Hospital: Development and Evaluation Study · arXiv:2610.32151
In plain words
Hospitals may need patient information to stay inside their own computer network. The authors built LLENS, a clinical support AI platform that runs there and lets a model read documents, work with files, and return results.
On 180 Japanese questions written by the development team, an automated grader found higher average answer accuracy when the model could interact with documents and tools than when it received passages from one fixed search. This was a technical test, not a study of patient care.
The research
When patient information must stay inside a hospital network, how can a model do more than chat or retrieve a few passages, including reading materials, working with files, and returning useful artifacts?
The authors address this technical question by building and evaluating an in-network platform that combines a model with document and computing tools.
The authors built LLENS, an in-network clinical support AI platform, and compared its interactive document access with a single fixed search.
The authors compared interactive document access with a single fixed search on 180 Japanese hospital-document questions written by the development team. The model answered each question three times; an automated grader matched final answers to references, and the authors averaged accuracy across runs and questions. Interactive access averaged 97.2%, versus 81.3% for fixed search.
The result shows higher accuracy for the tested interactive setup across this question set. A separate exploratory analysis found the largest gap on questions requiring filtering or counting across documents, consistent with the challenge of answering from one search’s passages.
The comparison bundled tools, orientation text, and skills, so it does not identify which component accounts for the difference.
These tests support technical feasibility on one host, not clinical effectiveness.
The document questions were written by the development team, the comparison used fixed retrieval rather than other search approaches, and the demonstrations used synthetic data; the study did not evaluate staff use or patient outcomes.
What the evidence shows
- Work inside a closed network
LLENS (Large Language Enhanced Nexus System) is a clinical support AI agent platform built to run inside Hokkaido University Hospital’s closed network.
The question behind the work is how a hospital can support tasks involving documents, scans, calculations, and returned files when both the model and the tools that process information must stay in the network.
- Compare two access setups
The authors tested document access with 180 Japanese questions across six domains, written by the development team. Each question was run three times.
One setup gave the model the ten passages from a single fixed search; the other let it use document tools and a sandbox, with orientation text and seven domain skills. The model, instructions, decoding settings, questions, and document collection were held constant, so this compared the full access setups rather than one isolated tool.
- The gap centers on filtering
An automated grader matched answers to references. Interactive access averaged 97.2% accuracy across questions, compared with 81.3% for fixed search.
The difference was especially pronounced on 30 exploratory filtering and counting questions: averages were 87.8% with interactive access and 16.7% with fixed search. This subset suggests that gathering information across documents was difficult for one fixed search pass in these tested questions.
- Controls narrow the claim
A control comparison helps qualify the overall result. Mean accuracy was 97.2% with both document tools and the sandbox, and 97.0% with the sandbox alone; that 0.2-point difference was not statistically significant.
So the higher score than fixed search does not establish that combining both tool types improves overall accuracy over sandbox access alone.
- Scenarios show technical reach
The authors also checked whether expected tools were triggered in 13 scenario requests, repeating each four times with synthetic materials. Expected components appeared in all runs for 12 scenarios, with inconsistent checking and artifact return in the falls-data scenario.
Five separate synthetic demonstrations showed varied multistep tasks, each run once. These technical exercises do not measure answer reliability in clinical work or effects on staff or patients.
Deep
- Original paper authors
- Ken Enda, Jien Saito, Masami Mukai, Feng Han, Emir Turkes
- Submitted to arXiv
- License
- CC BY 4.0