Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 4th December 2018, 16:07   #1  |  Link
abolibibelot
Registered User
 
Join Date: Jun 2015
Location: France
Posts: 29
Issue with "morph" + questions about frame interpolation and other subjects

Hi,

I'm trying to improve a movie I made about my deceased grandmother; I want this to be finished by the end of December. I shot the AVCHD 1280x720 source footage with two Panasonic compact cameras : a ZS3 shooting in 25 FPS in dec. 2013, with a strong backlight issue (mostly shot in front of a large window) ; a ZS7 shooting in 29.97 FPS in dec. 2015, that camera had a malfunctioning optical stabilization system, causing spontaneous intermittent vertical jerkiness (with the picture shaking up and down by about 5 pixels, 5 times per second, at random intervals...).

I tried to improve the picture as much as possible, using :
– the stabilization feature from Magix Video Deluxe (NLE software used to edit this movie), followed by a combination of FrameSurgeon and Morph (Avisynth functions) to interpolate the remaining blurry frames caused by the jerkiness on the 2015 videos (see this thread – Sawbones was a huge help, considering that I had to manually check about 35min. of footage, amounting to about 50.000 frames) ;
– a combination of AutoAdjust and HDRAGC to get something watchable out of the dark shadows on the 2013 videos (see this thread).

The first thing I want to do is interpolate the bad frames on the source footage rather than the edited / rendered movie, since some of those bad / blurry frames are located at crossfade points and I couldn't correct them satisfyingly when I finalized the first version.
But when examining the scripts I made a few months ago, I found out that the Morph() function no longer works as expected : now if I use the command "Morph(137,139)" for instance, I get a blend of frame 137 and the last frame of the input movie, instead of a blend of 137 and 139. What could be the explanation ? I tried copying the Morph function inside a script, changing its name to Morph2 and calling Morph2(x,y), same result.
I haven't voluntarily changed anything in my Avisynth configuration since then, but I noticed that the modified date of most subdirectories in my Avisynth+ directory, 2018/10/26 22:52, is very close to the date of StaxRip subdirectories, so apparently something was changed in Avisynth when I first ran StaxRip. I couldn't say which changes were made though, but I could check the differences with a backup made before that date, if needed.

Another issue is that, previously, I ran the frame interpolation script on the already stabilized footage, at the final encoding stage. It would have been even more difficult and painstaking to spot damaged frames among the original jerky mess, and probably much less efficient too (potentially interpolating from misaligned frames). So now I have to stabilize the footage before interpolating. Are there Avisynth function(s)/plugin(s) that would be at least as good as the NLE feature, and produce a reliable result ? I have briefly tried Deshaker in VirtualDub, it seemed quite efficient, but quite complicated, I had already many complications to deal with and, since the Magix stabilizer was performing mostly fine (better in my experience than the third-party proDAD Mercalli plugin that it now also includes, and which is supposed to be a professional level tool), I figured that I wouldn't need anything else... The alternative would be to treat the whole source files with MVD's stabilizer, export them as lossless intermediates, then process them with Avisynth to interpolate the bad frames, then re-import them into MVD, but I'd prefer not to...

Then, would it be possible, and wise, to load the scripts (probably 5-6) to the NLE using AVFS (Avisynth Virtual File System), with my machine based on an Intel i7 6700K CPU with 16GB of RAM, or would it be better to first export the processed footage as intermediate lossless files ? The first option would be more convenient of course, but may be slower, and potentially less reliable (if there's an out-of-memory issue for instance the results can start to get wonky, and some Avisynth functions apparently don't produce the same output when processing footage in a linear way or when previewing a particular frame at a random spot). With the second option, I know what the processed footage will look like, there are less moving parts, but if I need to make a slight adjustement I have to start all over again and re-export the whole bunch of big files, instead of just reloading the virtual AVI files.

Apparently this depends partly on the source loading plugin used : for instance I've had very unreliable results in the past when using DirectShowSource. In this case the source files are .m2ts : which plugin is generally recommanded with that format, and why ? ffms2 / FFVideoSource, or LSMASH / LWLibavVideoSource ? or something else ?

On frame interpolation, again : with FrameSurgeon, I get ugly artifacts in case of fast motion (that's why I used Morph for some frames, as it usually performs better in those cases, even though it's much less convenient and gets unstable if called more than a few dozen times in the same script, whereas FrameSurgeon doesn't choke with more than 4800 frames to process). I have read (see here – it's in french but the screenshots alone illustrate the issue very well – and here) that this kind of artifacts were produced by MVTools2 when processing "HD" footage, and not at lower resolutions. Is it a known issue, and is it still the case ? (Those two threads are 5-6 years old...)

Then, with regards to the backlit footage, is there anything better than HDRAGC that I could try this time ? Apparently this plugin is quite old; yet this kind of function should be high in demand as this is a very common issue, but I found nothing equivalent for Avisynth. I have also tried the Gamma HDR function in Magix Video Deluxe, I have tried various processing methods in DaVinci Resolve (possibly not to the best of their abilities as I was just discovering this seemingly highly capable software), both of which produced worse results than HDRAGC.

And what is the wisest way of combining 29.97 FPS footage with 25 FPS footage in the same movie ? (In two distinct parts, not mixed together.) At first I tried exporting in 25 FPS, but it caused the Magix stabilizer to malfunction ; then (using the 2010 version of that software) I exported in 29.97 FPS and I had a desynchronization issue, solved by updating to the 2016 version. On VideoHelp I've been told that exporting in 59.94 FPS would result in a smoother playback, but Magix Video Deluxe doesn't allow that framerate, 29.97 or 30 are the maximum values available. Would it be worth anything to convert the framerate to 59.94 with an Avisynth function at the final x264 encoding stage ?

Thanks !

Last edited by abolibibelot; 5th December 2018 at 17:20.
abolibibelot is offline   Reply With Quote
Old 6th December 2018, 15:48   #2  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,053
Quote:
And what is the wisest way of combining 29.97 FPS footage with 25 FPS footage in the same movie ? (In two distinct parts, not mixed together.)
If you just asking about converting the 25.0 fps to 29.97, might try the stuff in below link, I'de love to hear if it works ok for you.

Quote:
Originally Posted by StainlessS View Post
It may be possible to up framerate to a non multiple of source whilst keeping all source frames verbatim, where eg source
is 24 FPS and required = 25 FPS, and using,
Src24.ChangeFPS(25) to produce a new clip at 25 FPS with an exact duplicate every 24 source frames and save new source as eg Src25.avi.
Process new source with eg src25.Duplicity2(ThG=0.1,DBase="D:\Src25_Det.DB"), to produce a DBase (perhaps with MaxDupLen=1, perhaps not).
Then use Src25.DropDeadGorgeous("D:\Src25_Det.DB",ScanAhead=12) to interpolate dupe frames achieving 25.0 and with all original
source frames. With a ScanAhead of eg 12, will locate dupes and then backwards scan up to 12 frames before the dupes looking for
frame of greatest movement, insert an interpolated frame prior to that greatest movement frame and shove all following frames along one
to remove the dupe. Might be worth someone experimenting.
Or, you could just use eg Duplicity2(Mode=2,ThG=0.1) to interpolate the exact dupes directly and without DropDeadGorgeous but that might
produce a more rhythmic stutter every 25 frames.
Quote:
Originally Posted by StainlessS View Post
I tried above converting [EDIT: true] 25.0 FPS to 30.0FPS, using ChangeFPS(30.0), and Duplicity2(ThG=0.1,MaxDuplen=1),
Then, used DropDeadGorgeous(ScanAhead=5), result produced is 30FPS with all original source frames, and with good fluid motion throughout.
ChangeFPS(30.0) inserts duplicate after every five source frames, and DropDeadGorgeous(ScanAhead=5) introduces
an interpolated frame within 5 frames prior to the duplicate (between frames of greatest motion) and pushes the following frames one position
along to squeeze out the duplicate.
Works pretty good.
EDIT: If source clip was badly decimated from eg 30FPS to 25 and with jerky motion due to decimation, above should produce pretty much the best possible result and repair/tween the jerk frames produced by the decimation.
EDIT: Should also perform well in cases where cadence changes, eg where source was decimated from 29.97 FPS to 25 FPS,
or where source was edited after decimation, neither one should be any problem.
EDIT: Where above ThG=0.1, use ThG=0.0.
For ChangeFPS(30.0) use ChangeFPS(30000,1001), ie 29.97
EDIT: Should be no change in duration or [EDIT: perceptible] audio sync.

https://forum.doom9.org/showthread.p...65#post1857865
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 6th December 2018 at 19:39.
StainlessS is offline   Reply With Quote
Old 7th December 2018, 00:21   #3  |  Link
abolibibelot
Registered User
 
Join Date: Jun 2015
Location: France
Posts: 29
So I solved the Morph issue by re-installing Avisynth+ r1576 (replacing AviSynthPlus-MT-r2728.exe included in StaxRip). Does it help to pinpoint what was causing it ?

Doing further testing, I find it surprising and puzzling that applying Morph after FrameSurgeon on the same frame (which is problematic for FrameSurgeon because of fast motion and produces those weird fuzzy edges) gives a slightly better result than applying Morph alone (and much better than FrameSurgeon alone for that particular frame). Normally, interpolating, say, frame 138 from the adjacent frames with Morph(137,139) should totally cancel the interpolation of the same (single) frame with FrameSurgeon(I1 138), right ? Or does Morph somehow include the actual frame that is being replaced in its calculations to create the interpolated frame ? Strangely, this happens only if “show=true” is used in the FrameSurgeon command, without it there is no difference (therefore it's moot since that switch is only useful for previewing and has to be set to “false” or removed for the actual rendering).
With Morpheus I don't see the same effect as described above, i.e., applying Morpheus after FrameSurgeon(show=true) produces the same result as Morpheus alone.
You can download PNG screenshots showing those findings on this thread where I asked roughly the same questions (over here, even if they get approved, the pictures will apparently be converted to heavily compressed JPG, as the limit for PNG is 200KB – which is really tiny nowadays...).

Quote:
If you just asking about converting the 25.0 fps to 29.97, might try the stuff in below link, I'de love to hear if it works ok for you.
Thus far, I just imported the source files as-is into the NLE, and exported as 29.97 FPS, which automatically converts the 25 FPS part, most likely by duplicating one frame every 5 (minus one every 1000, I suppose, to account for the 0.03 discrepancy ?). There's not much motion in the 25 FPS part, so it works like that (there are far more severe issues with levels / crushed shadows and blown highlights) ; but since I have to pre-process that footage anyway, if that aspect could be improved with a relatively simple Avisynth command (nothing is quite “simple” when it comes to Avisynth ! ), that would be welcome. Do you think that the above method (which I haven't tested yet) would significantly slow down the processing ?

Last edited by abolibibelot; 9th December 2018 at 01:36.
abolibibelot is offline   Reply With Quote
Old 7th December 2018, 00:47   #4  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,053
Using anything else before hand should make no difference unless Morph is doing something wrong.
(Or FrameSurgen is affecting frames it should not, I dont think so).
Substituting a random frame for the frame to be interpolated (or maybe from a few frames away) should similarly make no difference to output.

Quote:
significantly slow down the processing ?
Would require frame accurate lossless file, scan complete clip to create a DBase (Using Duplicity2), and then a further pass
to change the framerate. I think you said 35 mins clip somewhere, so I would not consider that to provide much of a time overhead.
I once spend 3 days rendering a movie, and another time did 2000+ single frame hand edits, using something similar to Morph(start,end).

EDIT:
Quote:
“show=true” is used in the FrameSurgeon command, without it there is no difference
Yeh well that is one difference and could affect vectors (although why it might make it better, ???).
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 7th December 2018 at 01:36.
StainlessS is offline   Reply With Quote
Old 9th December 2018, 01:22   #5  |  Link
abolibibelot
Registered User
 
Join Date: Jun 2015
Location: France
Posts: 29
Quote:
Yeh well that is one difference and could affect vectors (although why it might make it better, ???).
Well, I couldn't say for sure that it's indeed better (I would need a perfect shot as a comparison !), but, if you saw the screenshots, the front of the bike seems “more straight”, and the letters on the box are better shaped ; but there are extra artifacts on the bike's left handle ; hard to say with just one frame.

Quote:
I think you said 35 mins clip somewhere, so I would not consider that to provide much of a time overhead.
Indeed, but I'd like this to be completed by the 25th, and I'm progressing really slowly ! My main issue right now is to get the stabilization right before applying the interpolation filtering (most of the interpolation hard work is done, but I'll have to start all over again for about 2 min. worth of footage which were slowed down in the final rendered movie, which I used before as a source for Sawbones / FrameSurgeon, therefore the frame numbers are off for that part).
I've tried using Deshaker within an Avisynth script : it works, the problem is that it works only in RGB colorspace, and if I add a ConvertToRGB command before, then the interpolation commands no longer work.
“MSuper: Clip must be YUV or YUY2 or planar RGB/RGBA” (for the Morph function)
“FrameSurgeon: Id Interpolation commands Planar and YUY2 ONLY”
(In the first case I don't understand why it doesn't work, since RGB is apparently allowed. Or is “planar RGB” something else entirely ?)
Another problem is that Deshaker runs the second pass each time the script is reloaded, which takes way too long and is not practical at all.
So, is there a way to avoid a triple colorspace conversion (since I will have to export the pre-processed files as RGB to avoid color discrepancies once the lossless intermediates are imported into the NLE), and to avoid having to first export the stabilized footage before applying the interpolation treatment ?
I've tried DePanStabilize, which is a native Avisynth filter and works in YUV, but I have a hard time finding a comprehensive documentation.
- Can I get it to zoom in to remove the “dancing” black borders ?
- Can it store the stabilizing parameters in a log file, to speed up further processing and playback once the calculations have been done ?
- Another issue is that when a frame n is blurred, the stabilizer sometimes aligns the frame n+1 with the upper or lower edges of the blurred frame, whereas it should align it with the frame n-1 to avoid an unwanted sudden jerkiness. I managed to improve that behaviour with Deshaker by increasing the “Motion smoothness” values, so is there an equivalent parameter with DePanStabilize ? Or would it be better to apply the interpolation filters before stabilizing, so that the stabilizer can deal with “clean” frames ? But in this case, won't that make the interpolations worse, in those cases where the adjacent frames are misaligned ?

Last edited by abolibibelot; 9th December 2018 at 01:43.
abolibibelot is offline   Reply With Quote
Old 9th December 2018, 04:17   #6  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,064
If Deshaker works, and if you happen to own Vegas, I wrote a complete automation suite that does batch conversion of video within Vegas. You can choose to use lossless codec for feeding to Deshaker and for saving the result, thus eliminating any "roundtrip" conversion loss when deshaking.
johnmeyer is offline   Reply With Quote
Old 9th December 2018, 05:53   #7  |  Link
abolibibelot
Registered User
 
Join Date: Jun 2015
Location: France
Posts: 29
Quote:
If Deshaker works, and if you happen to own Vegas, I wrote a complete automation suite that does batch conversion of video within Vegas. You can choose to use lossless codec for feeding to Deshaker and for saving the result, thus eliminating any "roundtrip" conversion loss when deshaking.
Thanks for the suggestion, but in this case I already made all the editing work with another software, namely Magix Video Deluxe. As much as I'd like to try Vegas, which seems to get a lot of praise, it would be a chore to start all over again for this particular movie. But I'd still be interested, if you have a link.
The stabilizer included with Magix VD did a fine job as I said in the first post (and it does save the parameters, so once the actual processing is done the playback is fluid, even on my former machine based on a Dual Core E5200 from 2009 I could play stabilized 1280x720p AVCHD footage in real time). If you felt the need to make Vegas work with Deshaker, does that mean that it doesn't include a good stabilizer of its own ?
abolibibelot is offline   Reply With Quote
Old 9th December 2018, 07:54   #8  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,064
Vegas includes a limited version of the Mercalli stabilizer. If you want the best stabilization software, get the full version of Mercalli. It runs rings around everything else, but it is $$$ (€€€).

Here is a link to the post I made in the Vegas forum over a decade ago where I developed my scripts to automate deshaking from within Vegas. It ended up being a bit of a three-corner billiard shot because it involved both a script within Vegas that "talked" to a script inside of VirtualDub, with parameters being passed back and forth.

https://www.vegascreativesoftware.in...render--57366/
johnmeyer is offline   Reply With Quote
Reply

Tags
backlight, hdragc, interpolation, jerky video, stabilization

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 17:37.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.