Query as Anchor: Scenario-Adaptive User Representation via Large Language Model
Industrial-scale user representation learning requires balancing robust universality with acute task-sensitivity. However, existing paradigms primarily yield static, task-agnostic embeddings that struggle to reconcile the divergent requirements of downstream scenarios within unified vector spaces. Furthermore, heterogeneous multi-source data introduces inherent noise and modality conflicts, degrading representation. We propose Query-as-Anchor, a framework shifting user modeling from static encoding to dynamic, query-aware synthesis. To empower Large Language Models (LLMs) with deep user understanding, we first construct UserU, an industrial-scale pre-training dataset that aligns multi-modal behavioral sequences with user understanding semantics, and our Q-Anchor Embedding architecture integrates hierarchical coarse-to-fine encoders into dual-tower LLMs via joint contrastive-autoregressive optimization for query-aware user representation. To bridge the gap between general pre-training and specialized business logic, we further introduce Cluster-based Soft Prompt Tuning to enforce discriminative latent structures, effectively aligning model attention with scenario-specific modalities. For deployment, anchoring queries at sequence termini enables KV-cache-accelerated inference with negligible incremental latency. Evaluations on 10 Alipay industrial benchmarks show consistent SOTA performance, strong scalability, and efficient deployment. Large-scale online A/B testing in Alipay's production system across two real-world scenarios further validates its practical effectiveness. Our code is prepared for public release and will be available at: https://github.com/JhCircle/Q-Anchor.
6993e4ac50fb2c0be4783d48
Jiahao Yuan
claimed_verified
2026-02-17T15:54:02.776Z
67acb56faee29e2e5c1c41ec
/avatars/b47094266fe70d10786da845ae2ade2c.svg
Jiahao Yuan
Jhcircle
user
Jhcircle
6993e4ac50fb2c0be4783d49
Yike Xu
6993e4ac50fb2c0be4783d4a
Jinyong Wen
6993e4ac50fb2c0be4783d4b
Baokun Wang
6993e4ac50fb2c0be4783d4c
Ziyi Gao
6993e4ac50fb2c0be4783d4d
Xiaotong Lin
6993e4ac50fb2c0be4783d4e
Yun Liu
6993e4ac50fb2c0be4783d4f
Xing Fu
6993e4ac50fb2c0be4783d50
Yu Cheng
6993e4ac50fb2c0be4783d51
Yongchao Liu
6993e4ac50fb2c0be4783d52
Weiqiang Wang
6993e4ac50fb2c0be4783d53
Zhongle Xie
Query as Anchor: Scenario-Adaptive User Representation via Large Language Model
Industrial-scale user representation learning requires balancing robust universality with acute task-sensitivity. However, existing paradigms primarily yield static, task-agnostic embeddings that struggle to reconcile the divergent requirements of downstream scenarios within unified vector spaces. Furthermore, heterogeneous multi-source data introduces inherent noise and modality conflicts, degrading representation. We propose Query-as-Anchor, a framework shifting user modeling from static encoding to dynamic, query-aware synthesis. To empower Large Language Models (LLMs) with deep user understanding, we first construct UserU, an industrial-scale pre-training dataset that aligns multi-modal behavioral sequences with user understanding semantics, and our Q-Anchor Embedding architecture integrates hierarchical coarse-to-fine encoders into dual-tower LLMs via joint contrastive-autoregressive optimization for query-aware user representation. To bridge the gap between general pre-training and specialized business logic, we further introduce Cluster-based Soft Prompt Tuning to enforce discriminative latent structures, effectively aligning model attention with scenario-specific modalities. For deployment, anchoring queries at sequence termini enables KV-cache-accelerated inference with negligible incremental latency. Evaluations on 10 Alipay industrial benchmarks show consistent SOTA performance, strong scalability, and efficient deployment. Large-scale online A/B testing in Alipay's production system across two real-world scenarios further validates its practical effectiveness. Our code is prepared for public release and will be available at: https://github.com/JhCircle/Q-Anchor.
A novel framework called Query-as-Anchor is introduced that transforms user modeling from static encoding to dynamic, query-aware synthesis using large language models with specialized architectures and training methods.
UserU Q-Anchor Embedding dual-tower LLMs joint contrastive-autoregressive optimization Cluster-based Soft Prompt Tuning KV-cache-accelerated inference