EgoEdit: Dataset, Real-Time Streaming Model, and Benchmark for Egocentric Video Editing
We study instruction-guided editing of egocentric videos for interactive AR applications. While recent AI video editors perform well on third-person footage, egocentric views present unique challenges - including rapid egomotion and frequent hand-object interactions - that create a significant domain gap. Moreover, existing offline editing pipelines suffer from high latency, limiting real-time interaction. To address these issues, we present a complete ecosystem for egocentric video editing. First, we construct EgoEditData, a carefully designed and manually curated dataset specifically designed for egocentric editing scenarios, featuring rich hand-object interactions, while explicitly preserving hands. Second, we develop EgoEdit, an instruction-following egocentric video editor that supports real-time streaming inference on a single GPU. Finally, we introduce EgoEditBench, an evaluation suite targeting instruction faithfulness, hand and interaction preservation, and temporal stability under egomotion. Across both egocentric and general editing tasks, EgoEdit produces temporally stable, instruction-faithful results with interactive latency. It achieves clear gains on egocentric editing benchmarks-where existing methods struggle-while maintaining performance comparable to the strongest baselines on general editing tasks. EgoEditData and EgoEditBench will be made public for the research community. See our website at https://snap-research.github.io/EgoEdit
69379c1c19d912300c34a2bc
Runjia Li
admin_assigned
2025-12-09T11:13:56.638Z
638e29cf319f9c746b87ad4b
https://cdn-avatars.huggingface.co/v1/production/uploads/638e29cf319f9c746b87ad4b/YjKrcY-NyvETSWrRnC0Tx.jpeg
Runjia Li
liguang0115
user
liguang0115
69379c1c19d912300c34a2bd
Moayed Haji-Ali
69379c1c19d912300c34a2be
Ashkan Mirzaei
69379c1c19d912300c34a2bf
Chaoyang Wang
69379c1c19d912300c34a2c0
Arpit Sahni
69379c1c19d912300c34a2c1
Ivan Skorokhodov
69379c1c19d912300c34a2c2
Aliaksandr Siarohin
69379c1c19d912300c34a2c3
Tomas Jakab
69379c1c19d912300c34a2c4
Junlin Han
69379c1c19d912300c34a2c5
Sergey Tulyakov
69379c1c19d912300c34a2c6
Philip Torr
69379c1c19d912300c34a2c7
Willi Menapace
EgoEdit: Dataset, Real-Time Streaming Model, and Benchmark for Egocentric Video Editing
We study instruction-guided editing of egocentric videos for interactive AR applications. While recent AI video editors perform well on third-person footage, egocentric views present unique challenges - including rapid egomotion and frequent hand-object interactions - that create a significant domain gap. Moreover, existing offline editing pipelines suffer from high latency, limiting real-time interaction. To address these issues, we present a complete ecosystem for egocentric video editing. First, we construct EgoEditData, a carefully designed and manually curated dataset specifically designed for egocentric editing scenarios, featuring rich hand-object interactions, while explicitly preserving hands. Second, we develop EgoEdit, an instruction-following egocentric video editor that supports real-time streaming inference on a single GPU. Finally, we introduce EgoEditBench, an evaluation suite targeting instruction faithfulness, hand and interaction preservation, and temporal stability under egomotion. Across both egocentric and general editing tasks, EgoEdit produces temporally stable, instruction-faithful results with interactive latency. It achieves clear gains on egocentric editing benchmarks-where existing methods struggle-while maintaining performance comparable to the strongest baselines on general editing tasks. EgoEditData and EgoEditBench will be made public for the research community. See our website at https://snap-research.github.io/EgoEdit
EgoEdit is a real-time, instruction-following egocentric video editor that addresses challenges in handling egomotion and hand-object interactions, outperforming existing methods on egocentric editing tasks.
EgoEditData EgoEdit EgoEditBench egocentric video editing instruction-following real-time streaming inference hand-object interactions egomotion instruction faithfulness hand preservation interaction preservation temporal stability