HyTRec: A Hybrid Temporal-Aware Attention Architecture for Long Behavior Sequential Recommendation
Modeling long sequences of user behaviors has emerged as a critical frontier in generative recommendation. However, existing solutions face a dilemma: linear attention mechanisms achieve efficiency at the cost of retrieval precision due to limited state capacity, while softmax attention suffers from prohibitive computational overhead. To address this challenge, we propose HyTRec, a model featuring a Hybrid Attention architecture that explicitly decouples long-term stable preferences from short-term intent spikes. By assigning massive historical sequences to a linear attention branch and reserving a specialized softmax attention branch for recent interactions, our approach restores precise retrieval capabilities within industrial-scale contexts involving ten thousand interactions. To mitigate the lag in capturing rapid interest drifts within the linear layers, we furthermore design Temporal-Aware Delta Network (TADN) to dynamically upweight fresh behavioral signals while effectively suppressing historical noise. Empirical results on industrial-scale datasets confirm the superiority that our model maintains linear inference speed and outperforms strong baselines, notably delivering over 8% improvement in Hit Rate for users with ultra-long sequences with great efficiency.
699d32bf4e37ec6dfa1bc690
Lei Xin
699d32bf4e37ec6dfa1bc691
Yuhao Zheng
699d32bf4e37ec6dfa1bc692
Ke Cheng
699d32bf4e37ec6dfa1bc693
Changjiang Jiang
claimed_verified
2026-02-26T10:03:02.605Z
652fc2605615e57807e3db19
https://cdn-avatars.huggingface.co/v1/production/uploads/652fc2605615e57807e3db19/kbRcpR0YFQnU3IlziqCHf.png
Changjiang Jiang
arnodjiang
user
arnodjiang
699d32bf4e37ec6dfa1bc694
Zifan Zhang
699d32bf4e37ec6dfa1bc695
Fanhu Zeng
claimed_verified
2026-03-16T10:45:29.189Z
667b7388580a979b703c3c2b
/avatars/15d42bf75f88f1e2ac112c98cb53bad6.svg
Fanhu Zeng
AuroraZengfh
user
AuroraZengfh
HyTRec: A Hybrid Temporal-Aware Attention Architecture for Long Behavior Sequential Recommendation
Modeling long sequences of user behaviors has emerged as a critical frontier in generative recommendation. However, existing solutions face a dilemma: linear attention mechanisms achieve efficiency at the cost of retrieval precision due to limited state capacity, while softmax attention suffers from prohibitive computational overhead. To address this challenge, we propose HyTRec, a model featuring a Hybrid Attention architecture that explicitly decouples long-term stable preferences from short-term intent spikes. By assigning massive historical sequences to a linear attention branch and reserving a specialized softmax attention branch for recent interactions, our approach restores precise retrieval capabilities within industrial-scale contexts involving ten thousand interactions. To mitigate the lag in capturing rapid interest drifts within the linear layers, we furthermore design Temporal-Aware Delta Network (TADN) to dynamically upweight fresh behavioral signals while effectively suppressing historical noise. Empirical results on industrial-scale datasets confirm the superiority that our model maintains linear inference speed and outperforms strong baselines, notably delivering over 8% improvement in Hit Rate for users with ultra-long sequences with great efficiency.
HyTRec addresses the challenge of modeling long user behavior sequences by combining linear and softmax attention mechanisms with a temporal-aware delta network to balance efficiency and retrieval precision.
Hybrid Attention linear attention softmax attention long-term stable preferences short-term intent spikes temporal-aware delta network TADN Hit Rate