Region-Constraint In-Context Generation for Instructional Video Editing
The In-context generation paradigm recently has demonstrated strong power in instructional image editing with both data efficiency and synthesis quality. Nevertheless, shaping such in-context learning for instruction-based video editing is not trivial. Without specifying editing regions, the results can suffer from the problem of inaccurate editing regions and the token interference between editing and non-editing areas during denoising. To address these, we present ReCo, a new instructional video editing paradigm that novelly delves into constraint modeling between editing and non-editing regions during in-context generation. Technically, ReCo width-wise concatenates source and target video for joint denoising. To calibrate video diffusion learning, ReCo capitalizes on two regularization terms, i.e., latent and attention regularization, conducting on one-step backward denoised latents and attention maps, respectively. The former increases the latent discrepancy of the editing region between source and target videos while reducing that of non-editing areas, emphasizing the modification on editing area and alleviating outside unexpected content generation. The latter suppresses the attention of tokens in the editing region to the tokens in counterpart of the source video, thereby mitigating their interference during novel object generation in target video. Furthermore, we propose a large-scale, high-quality video editing dataset, i.e., ReCo-Data, comprising 500K instruction-video pairs to benefit model training. Extensive experiments conducted on four major instruction-based video editing tasks demonstrate the superiority of our proposal.
6948c8f334f46eaf46cbb325
Zhongwei Zhang
6948c8f334f46eaf46cbb326
Fuchen Long
claimed_verified
2025-12-31T20:58:00.554Z
6449f2dfeb7db8f70fb990f8
/avatars/72c02e754bb35dab05c4a5f1e69c95f1.svg
Fuchen
FireCRT
user
FireCRT
6948c8f334f46eaf46cbb327
Wei Li
6948c8f334f46eaf46cbb328
Zhaofan Qiu
6948c8f334f46eaf46cbb329
Wu Liu
6948c8f334f46eaf46cbb32a
Ting Yao
6948c8f334f46eaf46cbb32b
Tao Mei
Region-Constraint In-Context Generation for Instructional Video Editing
The In-context generation paradigm recently has demonstrated strong power in instructional image editing with both data efficiency and synthesis quality. Nevertheless, shaping such in-context learning for instruction-based video editing is not trivial. Without specifying editing regions, the results can suffer from the problem of inaccurate editing regions and the token interference between editing and non-editing areas during denoising. To address these, we present ReCo, a new instructional video editing paradigm that novelly delves into constraint modeling between editing and non-editing regions during in-context generation. Technically, ReCo width-wise concatenates source and target video for joint denoising. To calibrate video diffusion learning, ReCo capitalizes on two regularization terms, i.e., latent and attention regularization, conducting on one-step backward denoised latents and attention maps, respectively. The former increases the latent discrepancy of the editing region between source and target videos while reducing that of non-editing areas, emphasizing the modification on editing area and alleviating outside unexpected content generation. The latter suppresses the attention of tokens in the editing region to the tokens in counterpart of the source video, thereby mitigating their interference during novel object generation in target video. Furthermore, we propose a large-scale, high-quality video editing dataset, i.e., ReCo-Data, comprising 500K instruction-video pairs to benefit model training. Extensive experiments conducted on four major instruction-based video editing tasks demonstrate the superiority of our proposal.
ReCo is a novel instructional video editing paradigm that enhances accuracy and reduces token interference by incorporating constraint modeling and regularization techniques during in-context generation.
in-context generation instructional video editing denoising ReCo constraint modeling latent regularization attention regularization backward denoised latents attention maps ReCo-Data video diffusion learning instruction-video pairs video editing tasks