TALON: Test-time Adaptive Learning for On-the-Fly Category Discovery
On-the-fly category discovery (OCD) aims to recognize known categories while simultaneously discovering novel ones from an unlabeled online stream, using a model trained only on labeled data. Existing approaches freeze the feature extractor trained offline and employ a hash-based framework that quantizes features into binary codes as class prototypes. However, discovering novel categories with a fixed knowledge base is counterintuitive, as the learning potential of incoming data is entirely neglected. In addition, feature quantization introduces information loss, diminishes representational expressiveness, and amplifies intra-class variance. It often results in category explosion, where a single class is fragmented into multiple pseudo-classes. To overcome these limitations, we propose a test-time adaptation framework that enables learning through discovery. It incorporates two complementary strategies: a semantic-aware prototype update and a stable test-time encoder update. The former dynamically refines class prototypes to enhance classification, whereas the latter integrates new information directly into the parameter space. Together, these components allow the model to continuously expand its knowledge base with newly encountered samples. Furthermore, we introduce a margin-aware logit calibration in the offline stage to enlarge inter-class margins and improve intra-class compactness, thereby reserving embedding space for future class discovery. Experiments on standard OCD benchmarks demonstrate that our method substantially outperforms existing hash-based state-of-the-art approaches, yielding notable improvements in novel-class accuracy and effectively mitigating category explosion. The code is publicly available at blue{https://github.com/ynanwu/TALON}.
69b159a71f7902d47f6dfc53
Yanan Wu
69b159a71f7902d47f6dfc54
Yuhan Yan
claimed_verified
2026-03-11T12:52:15.225Z
69b1597e15ed2b14d9c2ba13
/avatars/1a95aba1e8d01b0da17fa1dce2b39b00.svg
Yuhan Yan
conrain
user
conrain
69b159a71f7902d47f6dfc55
Tailai Chen
69b159a71f7902d47f6dfc56
Zhixiang Chi
69b159a71f7902d47f6dfc57
ZiZhang Wu
69b159a71f7902d47f6dfc58
Yi Jin
69b159a71f7902d47f6dfc59
Yang Wang
69b159a71f7902d47f6dfc5a
Zhenbo Li
TALON: Test-time Adaptive Learning for On-the-Fly Category Discovery
On-the-fly category discovery (OCD) aims to recognize known categories while simultaneously discovering novel ones from an unlabeled online stream, using a model trained only on labeled data. Existing approaches freeze the feature extractor trained offline and employ a hash-based framework that quantizes features into binary codes as class prototypes. However, discovering novel categories with a fixed knowledge base is counterintuitive, as the learning potential of incoming data is entirely neglected. In addition, feature quantization introduces information loss, diminishes representational expressiveness, and amplifies intra-class variance. It often results in category explosion, where a single class is fragmented into multiple pseudo-classes. To overcome these limitations, we propose a test-time adaptation framework that enables learning through discovery. It incorporates two complementary strategies: a semantic-aware prototype update and a stable test-time encoder update. The former dynamically refines class prototypes to enhance classification, whereas the latter integrates new information directly into the parameter space. Together, these components allow the model to continuously expand its knowledge base with newly encountered samples. Furthermore, we introduce a margin-aware logit calibration in the offline stage to enlarge inter-class margins and improve intra-class compactness, thereby reserving embedding space for future class discovery. Experiments on standard OCD benchmarks demonstrate that our method substantially outperforms existing hash-based state-of-the-art approaches, yielding notable improvements in novel-class accuracy and effectively mitigating category explosion. The code is publicly available at blue{https://github.com/ynanwu/TALON}.
A test-time adaptation framework for on-the-fly category discovery that dynamically updates prototypes and encoder parameters while calibrating logits to improve novel class recognition and prevent category explosion.
on-the-fly category discovery hash-based framework feature quantization category explosion test-time adaptation semantic-aware prototype update stable test-time encoder update margin-aware logit calibration intra-class variance inter-class margins