Chiosa
AI Security & Safety

A query-only monitor finds coordinated accounts despite prompt padding

2 min ·

En termes simples

A provider may see queries from many accounts but lack information linking those accounts. If a campaign spreads its queries out, each account can look ordinary, while legitimate users may also ask similar questions.

The paper introduces a monitor that compares account query patterns with patterns in earlier benign traffic and looks for groups moving together.

In tests on consumer-chat logs, its revised account-selection method still detected some campaigns after natural prompts were mixed into each account, though the tested campaigns made up unusually large shares of the monitored groups.

La recherche

If a provider cannot link accounts through metadata, can it recognize a campaign from queries alone when each account looks ordinary?

The problem is that legitimate users also coordinate around shared topics, trends, scripts, and templates, so a group’s unusualness must be judged against real benign activity.

The authors compare account query patterns with benign history and introduce a selection method that detects padded campaigns in later test weeks.

The authors compare account query patterns with earlier benign traffic, then test whether a revised way of selecting coordinated accounts can still find campaigns that mix natural prompts into every account.

On 17 later, untouched WildChat weeks, hollowed selection detected half-padded campaigns in multiple tested settings, while the two-scale detector detected none.

This supports the method’s continued detection in those settings after the pipeline was frozen, though false alarms and detection rates varied by setting.

These results come from consumer-chat logs, where hashed IPs only approximate accounts and low-volume accounts were excluded. Campaigns made up 14–50% of tested windows, far above production shares; the study does not establish performance on API traffic or at realistic production scale.

Analyse

Détails

Article suivantNOMOS rend les politiques écrites plus sûres pour les agents qui utilisent des outils