Triage matches video attention work to each tile
Triage: Skip, Skim, or Refine Attention Tiles in Video Diffusion Transformers · arXiv:2610.35706
In plain words
Video diffusion models use attention to connect many visual tokens, and that work can be expensive. Triage is a training-free method that decides tile by tile whether to skip a calculation, do a quick 4-bit version, or add extra correction for better accuracy.
In tests on four video models across five settings, one tested Triage setting per case ran faster than the evaluated alternatives and produced videos with higher PSNR (peak signal-to-noise ratio) than the most faithful alternative.
PSNR compares each generated video with the dense-generated reference; higher values mean a closer match. These results apply to the tested models and Blackwell GPUs.
The research
Video diffusion transformers spend substantial computation on attention across many visual tokens. Existing approaches can skip tiles, use lower precision, or combine both, but do not adapt the precision of each retained tile.
The question is whether attention can save work while keeping more accurate computation for tiles where approximation would matter most.
Triage routes each video-attention tile to skip, low-bit computation, or correction when its estimated error reduction justifies the added work.
The authors ask whether video attention can use less computation without losing fidelity where it matters. Triage answers by estimating each tile’s error under different compute levels, then spending extra work only when the predicted error reduction warrants it.
Across five settings, researchers found a tested Triage price that was faster than each evaluated training-free baseline and had higher PSNR (peak signal-to-noise ratio) than the highest-PSNR baseline against dense-generated video.
PSNR compares each generated video with the dense reference; higher values mean a closer match. The result supports this adaptive allocation on the tested models and hardware.
The router’s error estimates rely on simplifying assumptions; they are neither exact errors nor certified upper bounds. The end-to-end results cover five settings on single Blackwell GPUs, with some steps and non-attention work left dense.
What the evidence shows
- A tile-by-tile choice
Triage is a training-free attention method for video diffusion transformers. It addresses a practical tension inside attention: skipping computation saves work but can lose information, while using low precision everywhere can leave errors in important tiles.
The authors ask whether the computation can instead be matched to the likely importance of each tile.
- Different rows need different choices
To test why one fixed allocation may fall short, the researchers compared sparse, low-bit, and combined attention at an equal modeled compute cost. They measured relative L2 error, the relative size of the difference between approximate and dense attention row outputs, on captured calls from two video models.
Sparse attention worked best on rows with concentrated attention mass, the combined method on intermediate rows, and low-bit attention on rows with spread-out mass. Different rows therefore favor different allocations.
- Route by estimated error
Triage’s router estimates each tile’s mass and the error under skipping or low-bit computation from block statistics. It assigns skip, skim, or refine according to whether the estimated error reduction justifies extra compute.
A fused kernel then executes the choices together. The method’s estimates make routing inexpensive, but they approximate error rather than measuring or certifying it exactly.
- Speed and fidelity in tests
In five end-to-end settings, the researchers compared request latency with dense FlashAttention-2 and video fidelity with dense-generated video. Each setting had a Triage price that was faster than every evaluated training-free baseline and had higher PSNR (peak signal-to-noise ratio) than the baseline with the highest PSNR.
PSNR compares generated video with dense-generated video; higher values mean a closer match. In a separate replay of captured calls from two models, adding router-selected refinement lowered the lowest relative L2 error, supporting the role of that tier in the tested calls.
Deep
- Original paper authors
- Yeonjae Kim, Omin Kwon, Seong Hoon Seo, Jae W. Lee
- Submitted to arXiv
- License
- arXiv non-exclusive