One table benchmark reveals shared strengths and trade-offs
One Table, Many Queries: Benchmarking Generalist Tabular Models Across Generation, Imputation, and Prediction · arXiv:2611.13001
In plain words
A table can be used to make new rows, fill in missing values, or predict a target. The One Table Benchmark (OTBench) is a shared test suite that compares methods across all three jobs on aligned datasets, while spelling out what information each job can use.
The results show some shared strengths: methods good at recovering values or relationships can also rank well at prediction.
But matching a table’s overall distribution can favor different methods, and adding a task’s own training objective helped more consistently than adding another task’s objective in the tested systems.
The research
A table method can recover values or predict well without reproducing the table’s full distribution. That distinction matters when judging whether one model can cover several jobs: strength on a predictive measure may carry across tasks, while distributional fidelity can point to a different choice.
OTBench aligns three table tasks to compare shared capabilities and training objectives, finding that each task’s own objective has the strongest pooled gains.
In a matched study, five model backbones were trained with all seven nonempty combinations of generation, imputation, and prediction objectives, then evaluated on all three tasks. Adding the recipient task’s own objective improved every displayed capability axis on average and outperformed auxiliary additions.
This supports a direct role for task-specific training, while the selective rank agreement points to narrower shared capabilities.
The rank comparisons describe which methods tend to perform well together, not whether one task’s training causes gains in another. The controlled study covered five backbones and five datasets, with only one regression dataset; some comparisons also changed the inference rule used by a system.
What the evidence shows
- Compare three table tasks
The One Table Benchmark (OTBench) is a shared evaluation suite for generating table rows, filling missing values, and predicting a target. These tasks are often tested separately, so their scores do not show which abilities overlap or whether training for one helps another.
OTBench aligns datasets and specifies what information each task may use, making those relationships easier to compare without treating the tasks as identical.
- Shared strengths are selective
The comparison asks whether methods that rank well on imputation also rank well on prediction. Their alignment is selective: value recovery and relationships among variables can carry useful information across tasks, but distribution measures are less consistent. This means some capabilities overlap, while no single measure captures every task relationship.
- The task’s own objective leads
A separate matched study trains five model backbones with different combinations of the three task objectives. Across the 19 displayed capability axes, including a task’s own training objective had a positive pooled effect and exceeded either auxiliary-objective effect.
This is the clearest result about training changes in the study. For imputation and prediction in some systems, however, adding the objective also changed the system’s inference rule, so those comparisons combine training and inference choices.
- Transfer depends on the measure
Auxiliary benefits were less uniform. Adding prediction training to generation raised the pooled rank for how well synthetic-data-trained predictors approached a real-data-trained reference, while the global-utility rank barely changed in the opposite direction.
The prediction-related result varied across backbones, and its pooled interval included zero. The findings therefore support possible cross-task benefits, but not a dependable gain for every model or capability.
Deep
- Original paper authors
- Yuanrui Wang, Kaifang Li, Xuanyue Li, Ziyun Li, Yan Lu
- Submitted to arXiv
- License
- CC BY 4.0