A learned noise policy steers fast generators without changing them

Few-step generators can produce outputs with only a few model runs, but steering them toward a desired quality is difficult when reward scores provide no usable gradients.

The paper asks whether those scores can guide generation without changing the generator or repeatedly searching for a better input for each new output.

Noise Policy Optimization trains a reusable input-noise transformation around a frozen generator, steering outputs with rewards that have no usable gradients.

The paper asks how to steer few-step generators when reward scores have no usable gradients, while retaining the efficiency of producing outputs in only a few model runs. Noise Policy Optimization learns a reusable transformation of input noise and leaves the generator frozen.

Against direct fine-tuning, it improved rewards while better preserving measured image, text and protein qualities in the tested tasks. Against per-sample noise search, it matched or exceeded rewards at lower inference cost, supporting the case for learning the search once and reusing it.

The evidence comes from the tested image, language and protein tasks and their specific generators and reward measures. NPO also adds inference cost: in the reported configurations, it took about 2.2 to 2.6 times the cost of running the frozen generator alone.