Counteracting Matthew Effect in Self-Improvement of LVLMs through
  Head-Tail Re-balancing
Self-improvement has emerged as a mainstream paradigm for advancing the
reasoning capabilities of large vision-language models (LVLMs), where models
explore and learn from successful trajectories iteratively. However, we
identify a critical issue during this process: the model excels at generating
high-quality trajectories for simple queries (i.e., head data) but struggles
with more complex ones (i.e., tail data). This leads to an imbalanced
optimization that drives the model to prioritize simple reasoning skills, while
hindering its ability to tackle more complex reasoning tasks. Over iterations,
this imbalance becomes increasingly pronounced--a dynamic we term the "Matthew
effect"--which ultimately hinders further model improvement and leads to
performance bottlenecks. To counteract this challenge, we introduce four
efficient strategies from two perspectives: distribution-reshaping and
trajectory-resampling, to achieve head-tail re-balancing during the
exploration-and-learning self-improvement process. Extensive experiments on
Qwen2-VL-7B-Instruct and InternVL2.5-4B models across visual reasoning tasks
demonstrate that our methods consistently improve visual reasoning
capabilities, outperforming vanilla self-improvement by 3.86 points on average.
6904182a2c556835fa67f02d
Xin Guo
6904182a2c556835fa67f02e
Zhiheng Xi
6904182a2c556835fa67f02f
Yiwen Ding
6904182a2c556835fa67f030
Yitao Zhai
6904182a2c556835fa67f031
Xiaowei Shi
6904182a2c556835fa67f032
Xunliang Cai
6904182a2c556835fa67f033
Tao Gui
6904182a2c556835fa67f034
Qi Zhang
6904182a2c556835fa67f035
Xuanjing Huang
Counteracting Matthew Effect in Self-Improvement of LVLMs through
  Head-Tail Re-balancing
Self-improvement has emerged as a mainstream paradigm for advancing the
reasoning capabilities of large vision-language models (LVLMs), where models
explore and learn from successful trajectories iteratively. However, we
identify a critical issue during this process: the model excels at generating
high-quality trajectories for simple queries (i.e., head data) but struggles
with more complex ones (i.e., tail data). This leads to an imbalanced
optimization that drives the model to prioritize simple reasoning skills, while
hindering its ability to tackle more complex reasoning tasks. Over iterations,
this imbalance becomes increasingly pronounced--a dynamic we term the "Matthew
effect"--which ultimately hinders further model improvement and leads to
performance bottlenecks. To counteract this challenge, we introduce four
efficient strategies from two perspectives: distribution-reshaping and
trajectory-resampling, to achieve head-tail re-balancing during the
exploration-and-learning self-improvement process. Extensive experiments on
Qwen2-VL-7B-Instruct and InternVL2.5-4B models across visual reasoning tasks
demonstrate that our methods consistently improve visual reasoning
capabilities, outperforming vanilla self-improvement by 3.86 points on average.
Strategies for head-tail re-balancing during self-improvement enhance visual reasoning capabilities in large vision-language models.
self-improvement large vision-language models LVLMs trajectories head data tail data imbalanced optimization Matthew effect distribution-reshaping trajectory-resampling visual reasoning tasks