HyperVL: An Efficient and Dynamic Multimodal Large Language Model for Edge Devices
Current multimodal large lanauge models possess strong perceptual and reasoning capabilities, however high computational and memory requirements make them difficult to deploy directly on on-device environments. While small-parameter models are progressively endowed with strong general capabilities, standard Vision Transformer (ViT) encoders remain a critical bottleneck, suffering from excessive latency and memory consumption when processing high-resolution inputs.To address these challenges, we introduce HyperVL, an efficient multimodal large language model tailored for on-device inference. HyperVL adopts an image-tiling strategy to cap peak memory usage and incorporates two novel techniques: (1) a Visual Resolution Compressor (VRC) that adaptively predicts optimal encoding resolutions to eliminate redundant computation, and (2) Dual Consistency Learning (DCL), which aligns multi-scale ViT encoders within a unified framework, enabling dynamic switching between visual branches under a shared LLM. Extensive experiments demonstrate that HyperVL achieves state-of-the-art performance among models of comparable size across multiple benchmarks. Furthermore, it significantly significantly reduces latency and power consumption on real mobile devices, demonstrating its practicality for on-device multimodal inference.
6942650d5d5b2dc1052749c8
HyperAI Team
6942650d5d5b2dc1052749c9
Yuchen Liu
6942650d5d5b2dc1052749ca
Kaiyang Han
6942650d5d5b2dc1052749cb
Zhiqiang Xia
6942650d5d5b2dc1052749cc
Yuhang Dong
6942650d5d5b2dc1052749cd
Chen Song
claimed_verified
2025-12-19T09:07:32.108Z
648c314cfee82becc992852a
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/qN0C-ZWyNzHwiE8FPTn2g.jpeg
Chen Song
Kujou-Sara
user
Kujou-Sara
6942650d5d5b2dc1052749ce
Kangyu Tang
6942650d5d5b2dc1052749cf
Jiaming Xu
6942650d5d5b2dc1052749d0
Xiushi Feng
6942650d5d5b2dc1052749d1
WenXuan Yu
6942650d5d5b2dc1052749d2
Li Peng
6942650d5d5b2dc1052749d3
Mingyang Wang
6942650d5d5b2dc1052749d4
Kai Wang
6942650d5d5b2dc1052749d5
Changpeng Yang
6942650d5d5b2dc1052749d6
Yang Li
6942650d5d5b2dc1052749d7
Haoyu Lu
6942650d5d5b2dc1052749d8
Hao Wang
6942650d5d5b2dc1052749d9
Bingna Xu
6942650d5d5b2dc1052749da
Guangyao Liu
6942650d5d5b2dc1052749db
Long Huang
6942650d5d5b2dc1052749dc
Kaibin Guo
6942650d5d5b2dc1052749dd
Jinyang Wu
6942650d5d5b2dc1052749de
Dan Wu
6942650d5d5b2dc1052749df
Hongzhen Wang
6942650d5d5b2dc1052749e0
Peng Zhou
6942650d5d5b2dc1052749e1
Shuai Nie
6942650d5d5b2dc1052749e2
Shande Wang
6942650d5d5b2dc1052749e3
Runyu Shi
6942650d5d5b2dc1052749e4
Ying Huang
HyperVL: An Efficient and Dynamic Multimodal Large Language Model for Edge Devices
Current multimodal large lanauge models possess strong perceptual and reasoning capabilities, however high computational and memory requirements make them difficult to deploy directly on on-device environments. While small-parameter models are progressively endowed with strong general capabilities, standard Vision Transformer (ViT) encoders remain a critical bottleneck, suffering from excessive latency and memory consumption when processing high-resolution inputs.To address these challenges, we introduce HyperVL, an efficient multimodal large language model tailored for on-device inference. HyperVL adopts an image-tiling strategy to cap peak memory usage and incorporates two novel techniques: (1) a Visual Resolution Compressor (VRC) that adaptively predicts optimal encoding resolutions to eliminate redundant computation, and (2) Dual Consistency Learning (DCL), which aligns multi-scale ViT encoders within a unified framework, enabling dynamic switching between visual branches under a shared LLM. Extensive experiments demonstrate that HyperVL achieves state-of-the-art performance among models of comparable size across multiple benchmarks. Furthermore, it significantly significantly reduces latency and power consumption on real mobile devices, demonstrating its practicality for on-device multimodal inference.
HyperVL, an efficient multimodal large language model for on-device inference, uses image tiling, Visual Resolution Compressor, and Dual Consistency Learning to reduce memory usage, latency, and power consumption while maintaining performance.
multimodal large language models Vision Transformer (ViT) image-tiling strategy Visual Resolution Compressor Dual Consistency Learning on-device inference latency power consumption benchmarks