Workflow: V2V ReTake - Recreate any section of any video
LTX-2.3 V2V ReTake - recreate any section of any video
This is based on the LTX website concept they call "Re-Take", where you can recreate a section within a video.
In other words both the start and end point for the re-take is within the input video, to essentially re-create/re-take/re-shoot part of the input video.
Works within reasonable limits (smaller edits not going too off rail hehe)
This version of this workflow is a basic one to play around with.
But I'll try update it soon where not the whole input video gets loaded, but rather just the section needed (can save a bit of memory on longer videos)
Using KJNodes LTXVAudioVideoMask to mask an area to recreate.
Feel free to try it out ;-)
https://huggingface.co/RuneXX/LTX-2.3-Workflows/
This workflow will be updated, will try make it work better, currently it doesnt really allow much changes.
Congratulations!!! This is just amazing.... it even works really well :)
Thanks very much for the workflow. This works perfectly for the audio. I can use any video with any actor/actress and it clones there voice perfectly. Why is this not possible with your Longer Video workflows? The voice carry over never matches and sounds terrible in comparison to this. I don't get it. Obviously this workflow shows it is possible to clone a voice accurately with LTX 2.3. Any ideas why it only works properly with this one? The length doesn't seem to matter here. I can make a 20 second recut for a video with this one and the voice is a perfect match from start to finish.
. Why is this not possible with your Longer Video workflows? The voice carry over never matches
In the Long Video workflow multiple videos are created in a loop, or in groups (depending on what workflow you used).
So in a way its multiple independent video parts created. And in the end, all these videos are glued together as one long video.
It does have some reference input before creating a new video part, about 73 frames (but you can set this longer in the workflow settings at bottom). If there is no audio in those last 73 frames (voice audio), the next video part has no idea what was created before and will create an entirely new voice audio. Its a limitation the way it is currently
So the work-around would be to add custom audio input ... for example a song. As used in the examples.
But it could of course be one long monolog input audio as well
Now, that being said, luckily we have a new lora out, called ID-Lora, that should be able to use 5 second reference audio in every video part in the long video.
By this the voice should be consistent (for one character). Will update a long ID-Lora workflow soon.
LTX can only create video with same voice audio, if it knows what voice audio to use ;-) either by custom audio input, or by having enough reference frames that include voice audio when extending or editing
Thank you very much for the detailed explanation. Your very helpful. Also, thanks again for this recut workflow. I'm having a blast messing around with it.
By the way, this is better then any voice cloning model I've ever used. I've tried many of the recent ones. This surpasses them all by a long shot. In my opinion, there all crap compared to this.
LTX can be quite surprising sometimes.. . Its a bit of a "jack of all trades" .. ;-)
I had a surprised moment when i just input a drum solo audio clip and a static image of a drummer, and it pretty much nailed all the drum hits (kind of, at least good enough for a regular non drummer viewer)
And for voice cloning its surprisingly good. It can even dub to other languages (made a workflow for that for LTX-2.0, will update it to LTX-2.3).