Is There a Better Source Distribution than Gaussian? Exploring Source Distributions for Image Flow Matching
Flow matching has emerged as a powerful generative modeling approach with flexible choices of source distribution. While Gaussian distributions are commonly used, the potential for better alternatives in high-dimensional data generation remains largely unexplored. In this paper, we propose a novel 2D simulation that captures high-dimensional geometric properties in an interpretable 2D setting, enabling us to analyze the learning dynamics of flow matching during training. Based on this analysis, we derive several key insights about flow matching behavior: (1) density approximation can paradoxically degrade performance due to mode discrepancy, (2) directional alignment suffers from path entanglement when overly concentrated, (3) Gaussian's omnidirectional coverage ensures robust learning, and (4) norm misalignment incurs substantial learning costs. Building on these insights, we propose a practical framework that combines norm-aligned training with directionally-pruned sampling. This approach maintains the robust omnidirectional supervision essential for stable flow learning, while eliminating initializations in data-sparse regions during inference. Importantly, our pruning strategy can be applied to any flow matching model trained with a Gaussian source, providing immediate performance gains without the need for retraining. Empirical evaluations demonstrate consistent improvements in both generation quality and sampling efficiency. Our findings provide practical insights and guidelines for source distribution design and introduce a readily applicable technique for improving existing flow matching models. Our code is available at https://github.com/kwanseokk/SourceFM.
694ac2a9d8d7445f420c1732
Junho Lee
claimed_verified
2025-12-29T14:25:02.932Z
64a7c64d01646254507a644c
/avatars/79ae2bfc5eb9eaeca53e5d2c9cc10831.svg
junho lee
isno0907
user
isno0907
694ac2a9d8d7445f420c1733
Kwanseok Kim
claimed_verified
2025-12-25T20:46:52.534Z
6902f4ecd642fc67f25f41d4
/avatars/c5237e662ac5e83b7a2ca2dacf95c49d.svg
Kwanseok Kim
Kwanseok-k
user
Kwanseok-k
694ac2a9d8d7445f420c1734
Joonseok Lee
Is There a Better Source Distribution than Gaussian? Exploring Source Distributions for Image Flow Matching
Flow matching has emerged as a powerful generative modeling approach with flexible choices of source distribution. While Gaussian distributions are commonly used, the potential for better alternatives in high-dimensional data generation remains largely unexplored. In this paper, we propose a novel 2D simulation that captures high-dimensional geometric properties in an interpretable 2D setting, enabling us to analyze the learning dynamics of flow matching during training. Based on this analysis, we derive several key insights about flow matching behavior: (1) density approximation can paradoxically degrade performance due to mode discrepancy, (2) directional alignment suffers from path entanglement when overly concentrated, (3) Gaussian's omnidirectional coverage ensures robust learning, and (4) norm misalignment incurs substantial learning costs. Building on these insights, we propose a practical framework that combines norm-aligned training with directionally-pruned sampling. This approach maintains the robust omnidirectional supervision essential for stable flow learning, while eliminating initializations in data-sparse regions during inference. Importantly, our pruning strategy can be applied to any flow matching model trained with a Gaussian source, providing immediate performance gains without the need for retraining. Empirical evaluations demonstrate consistent improvements in both generation quality and sampling efficiency. Our findings provide practical insights and guidelines for source distribution design and introduce a readily applicable technique for improving existing flow matching models. Our code is available at https://github.com/kwanseokk/SourceFM.
A novel framework for flow matching combines norm-aligned training with directionally-pruned sampling to improve generation quality and sampling efficiency by addressing mode discrepancy and path entanglement.
flow matching source distribution Gaussian distributions density approximation directional alignment path entanglement norm misalignment norm-aligned training directionally-pruned sampling