In plain words
Text-to-image generators can make unsafe images from prompts that get past safeguards. Checking only the finished image means the generator has already done all its work.
This paper introduces a method that watches signals the generator produces while making an image. It stops generations whose signals look risky and lets others continue.
In tests on five generators, it detected unsafe prompts better than six comparison detectors in most tested settings, at a matched false-alarm rate.
The research
When a text-to-image generator’s safeguards miss a risky prompt, checking the finished image means the full generation cost has already been incurred.
It remains unclear whether the generator reveals enough about risk while creating an image to stop a flagged request sooner, without stopping every benign one.
CSTI reads early internal signals from a text-to-image generator to flag risky prompts and stop their generation before synthesis finishes.
The authors test whether signals inside a text-to-image generator can reveal risky prompts before the image is finished. Their method reads paired responses the generator already computes, makes one stopping decision after three early updates, and leaves accepted generations on their original path.
Across five generators and 40 risk-prompt settings, CSTI had the highest rate of catching unsafe inputs in 37 settings among six detectors, when compared at the same 5% empirical false-positive rate. This supports early interception within the tested collections.
The comparison covers five generators and the tested prompt collections; it does not establish performance on prompts or settings outside them. On challenging benign prompts that resembled unsafe concepts, false interceptions remained a limitation, particularly for SD3.5.
Analysis
Deep
- Original paper authors
- Yanchun Li, Yujin Wu, Gangkai Shen, Ming Yang, Li Zeng
- Submitted to arXiv
- License
- CC BY 4.0