Discriminately Treating Motion Components Evolves Joint Depth and
  Ego-Motion Learning
Unsupervised learning of depth and ego-motion, two fundamental 3D perception
tasks, has made significant strides in recent years. However, most methods
treat ego-motion as an auxiliary task, either mixing all motion types or
excluding depth-independent rotational motions in supervision. Such designs
limit the incorporation of strong geometric constraints, reducing reliability
and robustness under diverse conditions. This study introduces a discriminative
treatment of motion components, leveraging the geometric regularities of their
respective rigid flows to benefit both depth and ego-motion estimation. Given
consecutive video frames, network outputs first align the optical axes and
imaging planes of the source and target cameras. Optical flows between frames
are transformed through these alignments, and deviations are quantified to
impose geometric constraints individually on each ego-motion component,
enabling more targeted refinement. These alignments further reformulate the
joint learning process into coaxial and coplanar forms, where depth and each
translation component can be mutually derived through closed-form geometric
relationships, introducing complementary constraints that improve depth
robustness. DiMoDE, a general depth and ego-motion joint learning framework
incorporating these designs, achieves state-of-the-art performance on multiple
public datasets and a newly collected diverse real-world dataset, particularly
under challenging conditions. Our source code will be publicly available at
mias.group/DiMoDE upon publication.
690ab99cd70e173c84528f9c
Mengtan Zhang
690ab99cd70e173c84528f9d
Zizhan Guo
690ab99cd70e173c84528f9e
Hongbo Zhao
690ab99cd70e173c84528f9f
Yi Feng
690ab99cd70e173c84528fa0
Zuyi Xiong
690ab99cd70e173c84528fa1
Yue Wang
690ab99cd70e173c84528fa2
Shaoyi Du
690ab99cd70e173c84528fa3
Hanli Wang
690ab99cd70e173c84528fa4
Rui Fan
Discriminately Treating Motion Components Evolves Joint Depth and
  Ego-Motion Learning
Unsupervised learning of depth and ego-motion, two fundamental 3D perception
tasks, has made significant strides in recent years. However, most methods
treat ego-motion as an auxiliary task, either mixing all motion types or
excluding depth-independent rotational motions in supervision. Such designs
limit the incorporation of strong geometric constraints, reducing reliability
and robustness under diverse conditions. This study introduces a discriminative
treatment of motion components, leveraging the geometric regularities of their
respective rigid flows to benefit both depth and ego-motion estimation. Given
consecutive video frames, network outputs first align the optical axes and
imaging planes of the source and target cameras. Optical flows between frames
are transformed through these alignments, and deviations are quantified to
impose geometric constraints individually on each ego-motion component,
enabling more targeted refinement. These alignments further reformulate the
joint learning process into coaxial and coplanar forms, where depth and each
translation component can be mutually derived through closed-form geometric
relationships, introducing complementary constraints that improve depth
robustness. DiMoDE, a general depth and ego-motion joint learning framework
incorporating these designs, achieves state-of-the-art performance on multiple
public datasets and a newly collected diverse real-world dataset, particularly
under challenging conditions. Our source code will be publicly available at
mias.group/DiMoDE upon publication.
A discriminative approach to depth and ego-motion estimation leverages geometric constraints to improve performance and robustness in 3D perception tasks.
unsupervised learning depth ego-motion 3D perception optical axes imaging planes optical flows geometric constraints coaxial coplanar closed-form geometric relationships DiMoDE