Separate camera statistics help cooperative detection under uneven corruption
Agent-Wise Test-Time Normalization for Camera-Based Cooperative Perception with Corrupted Collaborators · arXiv:2610.39505
In plain words
When vehicles share camera views to detect objects, a partner’s foggy or blurry camera can make the combined result worse than using the clean vehicle’s camera alone.
The paper introduces AgentNorm, which adjusts each partner’s camera features using that partner’s own current view, while leaving the clean vehicle’s settings as they were. In tests, it improved detection over pooling information from all vehicles when partners had fog or motion blur. It did not reliably help when images lost content to occlusion.
The research
When vehicles combine camera views, what happens if one partner’s camera is corrupted but the ego vehicle’s camera remains clear?
In the paper’s tests, fog or motion blur on a collaborator could make cooperative detection worse than using the ego alone, raising the question of whether the views can be combined without changing the clean vehicle’s own evidence.
AgentNorm normalizes each collaborator’s camera features separately while keeping the clean ego on its original statistics, improving detection under uneven shifts.
The authors compared the clean ego’s detection using its original statistics with detection after pooling statistics from a corrupted collaborator. They measured only the ego’s own features, discarding collaborators before fusion.
Pooling lowered the clean ego’s [email protected] by 3.58 to 6.68 points, showing how a partner’s camera shift can alter the ego’s evidence.
They then tested AgentNorm against pooled normalization on four camera models. Under severity-5 fog and motion blur on collaborators, AgentNorm scored higher in all eight model-by-corruption comparisons.
This supports using separate statistics when agents’ shifts differ, while the shared-corruption control and occlusion results bound that conclusion.
The results cover camera-based models with batch normalization under the tested corruptions and datasets. AgentNorm does not reliably help with occlusion, which removes image content, and the real-world dataset check used a model with low absolute accuracy.
What the evidence shows
- When cooperation can hurt
AgentNorm is a training-free test-time normalization method for camera-based cooperative perception. On OPV2V, a simulated vehicle-camera test set, the authors found that severity-5 fog or motion blur on collaborators could make AttFuse’s detection score lower than using the clean ego vehicle alone.
[email protected] measures detection quality by how well predicted objects overlap labeled objects at a 0.5 threshold. The comparison establishes why corrupted collaborators pose a problem for cooperation.
- A shift in collaborator features
The authors measured collaborator features before fusion across four camera corruptions. As corruption severity increased, the features moved farther from statistics measured on clean training data and had lower local variation.
These mean measurements covered 3,445 collaborator instances and describe feature statistics, not detection accuracy. The authors interpret the changes as the kind of shift normalization can address, suggesting that each collaborator’s own observation contains information about its altered features.
- Pooling can change the ego
A separate AttFuse comparison isolated the effect on the clean ego. The authors normalized features with statistics pooled across the clean ego and a severity-5 corrupted collaborator, then discarded collaborator features and measured detection from the ego’s own features.
Its [email protected] fell by 3.58 to 6.68 points, depending on corruption, compared with using the model’s original statistics. This shows how pooling can affect the clean agent before fusion.
- Separate statistics help under shifts
AgentNorm uses each collaborator’s current observation to calculate its own normalization statistics and keeps the ego on the model’s original statistics. Across four OPV2V camera checkpoints, it scored higher than pooled normalization for all eight severity-5 fog and motion-blur combinations.
The gains ranged from 5.53 to 12.94 [email protected] points across those combinations. The result supports this approach when agents experience different camera shifts, while the shared-corruption control and occlusion results limit the claim.
Deep
- Original paper authors
- Yu Wang, Shaohua Wang, Yicheng Li, Kecheng Sun
- Submitted to arXiv
- License
- CC BY 4.0