Surprised by Attention: Predictable Query Dynamics for Time Series Anomaly Detection
Multivariate time series anomalies often manifest as shifts in cross-channel dependencies rather than simple amplitude excursions. In autonomous driving, for instance, a steering command might be internally consistent but decouple from the resulting lateral acceleration. Residual-based detectors can miss such anomalies when flexible sequence models still reconstruct signals plausibly despite altered coordination. We introduce AxonAD, an unsupervised detector that treats multi-head attention query evolution as a short horizon predictable process. A gradient-updated reconstruction pathway is coupled with a history-only predictor that forecasts future query vectors from past context. This is trained via a masked predictor-target objective against an exponential moving average (EMA) target encoder. At inference, reconstruction error is combined with a tail-aggregated query mismatch score, which measures cosine deviation between predicted and target queries on recent timesteps. This dual approach provides sensitivity to structural dependency shifts while retaining amplitude-level detection. On proprietary in-vehicle telemetry with interval annotations and on the TSB-AD multi-variate suite (17 datasets, 180 series) with threshold-free and range-aware metrics, AxonAD improves ranking quality and temporal localization over strong baselines. Ablations confirm that query prediction and combined scoring are the primary drivers of the observed gains. Code is available at the URL https://github.com/iis-esslingen/AxonAD.
69b79ef991a28162b233fcd7
Kadir-Kaan Özer
claimed_verified
2026-03-16T10:32:44.344Z
66280c602060c4fc3e8b853d
/avatars/302185dd703846b1b4182ef1567251b2.svg
Kadir-Kaan Özer
kadiroezer
user
kadiroezer
69b79ef991a28162b233fcd8
René Ebeling
69b79ef991a28162b233fcd9
Markus Enzweiler
Surprised by Attention: Predictable Query Dynamics for Time Series Anomaly Detection
Multivariate time series anomalies often manifest as shifts in cross-channel dependencies rather than simple amplitude excursions. In autonomous driving, for instance, a steering command might be internally consistent but decouple from the resulting lateral acceleration. Residual-based detectors can miss such anomalies when flexible sequence models still reconstruct signals plausibly despite altered coordination. We introduce AxonAD, an unsupervised detector that treats multi-head attention query evolution as a short horizon predictable process. A gradient-updated reconstruction pathway is coupled with a history-only predictor that forecasts future query vectors from past context. This is trained via a masked predictor-target objective against an exponential moving average (EMA) target encoder. At inference, reconstruction error is combined with a tail-aggregated query mismatch score, which measures cosine deviation between predicted and target queries on recent timesteps. This dual approach provides sensitivity to structural dependency shifts while retaining amplitude-level detection. On proprietary in-vehicle telemetry with interval annotations and on the TSB-AD multi-variate suite (17 datasets, 180 series) with threshold-free and range-aware metrics, AxonAD improves ranking quality and temporal localization over strong baselines. Ablations confirm that query prediction and combined scoring are the primary drivers of the observed gains. Code is available at the URL https://github.com/iis-esslingen/AxonAD.
AxonAD is an unsupervised anomaly detection method for multivariate time series that identifies structural dependency shifts by analyzing query evolution in multi-head attention mechanisms, outperforming existing approaches on both proprietary automotive telemetry and benchmark datasets.
multivariate time series cross-channel dependencies residual-based detectors multi-head attention query evolution gradient-updated reconstruction history-only predictor masked predictor-target objective exponential moving average tail-aggregated query mismatch score cosine deviation