NAF: Zero-Shot Feature Upsampling via Neighborhood Attention Filtering
Vision Foundation Models (VFMs) extract spatially downsampled representations, posing challenges for pixel-level tasks. Existing upsampling approaches face a fundamental trade-off: classical filters are fast and broadly applicable but rely on fixed forms, while modern upsamplers achieve superior accuracy through learnable, VFM-specific forms at the cost of retraining for each VFM. We introduce Neighborhood Attention Filtering (NAF), which bridges this gap by learning adaptive spatial-and-content weights through Cross-Scale Neighborhood Attention and Rotary Position Embeddings (RoPE), guided solely by the high-resolution input image. NAF operates zero-shot: it upsamples features from any VFM without retraining, making it the first VFM-agnostic architecture to outperform VFM-specific upsamplers and achieve state-of-the-art performance across multiple downstream tasks. It maintains high efficiency, scaling to 2K feature maps and reconstructing intermediate-resolution maps at 18 FPS. Beyond feature upsampling, NAF demonstrates strong performance on image restoration, highlighting its versatility. Code and checkpoints are available at https://github.com/valeoai/NAF.
6927339d243b2216fb75cca7
Loick Chambon
claimed_verified
2025-11-27T09:59:23.747Z
646b8882e96a751c5252bf18
/avatars/14bbe1d53ce00e530b29fa3c6cac7a6b.svg
Loick 
LChambon
user
LChambon
6927339d243b2216fb75cca8
Paul Couairon
6927339d243b2216fb75cca9
Eloi Zablocki
6927339d243b2216fb75ccaa
Alexandre Boulch
6927339d243b2216fb75ccab
Nicolas Thome
6927339d243b2216fb75ccac
Matthieu Cord
NAF: Zero-Shot Feature Upsampling via Neighborhood Attention Filtering
Vision Foundation Models (VFMs) extract spatially downsampled representations, posing challenges for pixel-level tasks. Existing upsampling approaches face a fundamental trade-off: classical filters are fast and broadly applicable but rely on fixed forms, while modern upsamplers achieve superior accuracy through learnable, VFM-specific forms at the cost of retraining for each VFM. We introduce Neighborhood Attention Filtering (NAF), which bridges this gap by learning adaptive spatial-and-content weights through Cross-Scale Neighborhood Attention and Rotary Position Embeddings (RoPE), guided solely by the high-resolution input image. NAF operates zero-shot: it upsamples features from any VFM without retraining, making it the first VFM-agnostic architecture to outperform VFM-specific upsamplers and achieve state-of-the-art performance across multiple downstream tasks. It maintains high efficiency, scaling to 2K feature maps and reconstructing intermediate-resolution maps at 18 FPS. Beyond feature upsampling, NAF demonstrates strong performance on image restoration, highlighting its versatility. Code and checkpoints are available at https://github.com/valeoai/NAF.
Neighborhood Attention Filtering (NAF) upsamples features from Vision Foundation Models without retraining, achieving state-of-the-art performance across tasks with high efficiency.
Neighborhood Attention Filtering (NAF) Vision Foundation Models (VFMs) spatially downsampled representations upsampling approaches classical filters learnable upsamplers Cross-Scale Neighborhood Attention Rotary Position Embeddings (RoPE) zero-shot upsampling feature upsampling image restoration