WorldFoundry makes world-model testing easier to run and audit
WorldFoundry: Towards Unified Infrastructure for World Intelligence · arXiv:2611.10771
In plain words
The paper asks how researchers can compare systems that generate or interact with changing worlds when they use different formats and testing setups. It introduces WorldFoundry, open-source tools for finding, running, and evaluating video, 3D, and action-based systems.
The paper also defines a world model as a system that keeps track of its environment, changes its predictions when actions occur, and updates from new observations.
Every integrated runner completed a documented workflow, showing it could run in the tested setup, not that every model meets this definition.
The research
For judging whether a system models a changing world, visual output alone can mislead: the key question is whether actions and new observations shape later predictions.
A shared functional definition and auditable runs make that distinction easier to examine across unlike systems, while keeping execution success separate from evidence of capability.
WorldFoundry pairs a functional definition of world models with open infrastructure for running and evaluating unlike systems under their native workflows.
The framework describes systems by state inference, prediction under changes, and responses to actions, then WorldFoundry connects discovery, execution, stored outputs, and evaluation through modality-specific adapters.
Its catalog lists 270 model entries, 74 test entries, and 62 metrics; every integrated runner completed a documented end-to-end reference workflow.
That record makes workflows auditable across different systems, but records readiness rather than capability.
The runner checks cover documented workflows under tested configurations. They do not show that each integrated system maintains a changing internal state or qualifies as a world model.
What the evidence shows
- A shared way to compare systems
WorldFoundry is open-source infrastructure for running and evaluating video, 3D, 4D, embodied-action, and interactive systems. The paper frames the problem as more than a shortage of models: incompatible representations, runtimes, interfaces, and evaluation procedures make it hard to combine capabilities, reproduce results, and measure progress.
Its proposal addresses both what counts as a world model and how to organize evidence about such systems.
- Persistence is the key test
The framework describes three functions: infer environmental state from observations, predict how it changes under dynamics or interventions, and produce futures shaped by actions. The deciding criterion is persistence across that loop: an intervention must affect later predictions, and new observations must revise the state. This makes plausible-looking output insufficient on its own as evidence of world modeling.
- Infrastructure spans many systems
WorldFoundry puts this framing into infrastructure that preserves systems’ native execution and evaluation procedures while connecting discovery, execution, artifact storage, and evaluation. Its catalog reports 270 model entries.
That scope shows the infrastructure organizes a varied set of systems; it does not establish that those models share the same capability or meet the paper’s definition.
- Execution is not capability
Every integrated runner completed a documented end-to-end reference workflow, giving an auditable record of readiness in the tested configuration. The paper keeps this operational check separate from validated performance.
That distinction supports a bounded conclusion: WorldFoundry helps make workflows executable and claims traceable, while the runner checks alone do not establish that the integrated models maintain persistent state.
Deep
- Original paper authors
- Juanxi Tian, Jiadong Hou, Baiang Li, Xin Lin, Zhengbo Jiao
- Submitted to arXiv
- License
- CC BY 4.0