Attractive faces shape image rankings and scores

When people rank generated faces, do their choices reflect image quality or aesthetic preference, or do attractive faces gain an advantage even when the comparison is between real photographs? That question matters to how we interpret human rankings and the image scores trained to predict them.

The authors audit attractiveness bias across human rankings, image scores, and generated faces, then adjust preference scores to reduce their link to attractiveness.

To test whether attractiveness affects preferences even when image sources are held more comparable, researchers examined the two real photographs in each four-face ranking. Participants favored the real face they rated as more attractive under both instruction conditions, with a larger difference under aesthetic-preference wording.

The pattern also appeared in a constructed leaderboard simulation and in several preference-model scores. In one test, a post-hoc adjustment reduced the correlation between predicted attractiveness and HPSv3 scores from 0.33 to 0.09 on Stable Diffusion 3.5 images, while making only minimal score changes. The findings support a bias across these tested settings and a score-level mitigation.

These results point to an attractiveness-related bias across several study settings, but they do not establish its size in real-world leaderboards.

The user ratings, simulated matches, and edited images each have limitations, and the score adjustment changes scores rather than the underlying preference data.