Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 18th May 2020, 09:24   #241  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,823
Quote:
Originally Posted by manono View Post
Yes it does, but it's easy enough to use any deinterlacer you like. To use QTGMC for post-processing, for example, you do this:

tdeintted = QTGMC(FPSDivisor=2)
tfm(clip2=tdeintted).tdecimate()


That's from some included manual or other for TIVTC.
That's what I've been doing for the samples I've uploaded, aside from the VFR encodes, although I suspect this might work better as it'd ensure the frames being de-interlaced by QTGMC are the same frames TFM wants to repair.

tdeintted = tfm(pp=1).QTGMC(FPSDivisor=2)
tfm(clip2=tdeintted).tdecimate()


But even then, because the source clip contains lots of aliasing, depending how closely the de-interlaced clip matches the clip TFM wants to repair, replacing combed pixels with QTGMC pixels could possibly cause shimmering that wasn't there before.

For DS9, if you change TFM's own de-interlacing it looks like the result will be better. I haven't played with every imaginable combination of settings, but for the VFR encodes I uploaded yesterday, switching to pp=5 and not using a QTGMC deinterlaced clip has produced the best result so far, compared to the rest of the samples I've uploaded, and I played around quite a bit with TFM's settings for those encodes.
The VFR encodes are clean enough for repairing the CGI sections with QTGMC to be fairly optional (there's a tad less shimmering than when the source is being IVTC'd by my video card), and even though the CGI section is telecined, VFR rate appears to allow TFM to treat the fades from one shot to the next as purely interlaced (where hard telecine is overlaid on hard telecine with a different field order, I think), and those transitions are quite smooth. I guess my video card does the same, as the source looks smooth through those transitions too, without any combing.
The "problem" telecined CGI section is roughly from frame 570 to frame 3000, after decimation.

Quote:
# timecode format v1
Assume 29.970030
# TDecimate v1.0.3.1 by tritical
# Mode 5 - Auto-generated mkv timecodes file
0,971,23.976024
992,1831,23.976024
1837,1840,23.976024
1846,2585,23.976024
2591,2998,23.976024
For a VFR encode, if you use a de-interlaced clip for TFM you have to use it for the analysis pass too (otherwise TDecimate complains the CRC no longer matches the analysis clip), so if it's a QTGMC de-interlaced clip, it'd slow down a normally fast analysis pass quite considerably.

Last edited by hello_hello; 18th May 2020 at 13:48.
hello_hello is offline   Reply With Quote
Old 18th May 2020, 09:49   #242  |  Link
zapp7
Registered User
 
Join Date: May 2020
Location: Canada
Posts: 49
There's one other problem that I have only seen briefly mentioned, and that is the abysmal source footage quality in the early seasons of the show. Later season episodes, like Sacrifice of Angels, look pristine by comparison. I've tried various noise filters (as well as no noise filter) after performing the IVTC on the film sections and the result, when upscaled by Topaz, looks like garbage.

Here's a sample of IVTC'd source footage of the station. How can this possibly be upscaled to look okay?
sample
zapp7 is offline   Reply With Quote
Old 18th May 2020, 10:19   #243  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,823
Quote:
Originally Posted by zapp7 View Post
Here's a sample of IVTC'd source footage of the station. How can this possibly be upscaled to look okay?
sample
If they used the same shot, or a similar one, in another episode that's much better quality, you could always swap them out.
hello_hello is offline   Reply With Quote
Old 18th May 2020, 16:21   #244  |  Link
videoh
Useful n00b
 
Join Date: Jul 2014
Posts: 1,667
Quote:
Originally Posted by Stereodude View Post
There's 3:2 cadence judder and then there's bad 24p content. The two are being incorrectly conflated.
Quite right, Stereodude. Also, tearing and pulldown are being conflated. Actually, pure 3:2 pulldown can be easily removed so it is rather strange to call it an artifact.
videoh is offline   Reply With Quote
Old 18th May 2020, 17:46   #245  |  Link
Katie Boundary
Registered User
 
Katie Boundary's Avatar
 
Join Date: Jan 2015
Posts: 1,048
Quote:
Originally Posted by manono View Post
Yes it does, but it's easy enough to use any deinterlacer you like. To use QTGMC for post-processing, for example...
OR, you could just tweak TFM's settings. Setting cthresh and mthresh to 2 or less, and using clip2 to retrieve pixels from a temporally-aware bob-deinterlacer like Yadif, works miracles.

Quote:
Originally Posted by manono View Post
If you use gmail at all, or other Google products, you should have access to your own Google Drive. I forget how large the files you can up/download. 5 GB, maybe? I hardly use it and I'm sure others know more about it.
I do not use gmail because Google is evil.

Quote:
Originally Posted by zapp7 View Post
There's one other problem that I have only seen briefly mentioned, and that is the abysmal source footage quality in the early seasons of the show. Later season episodes, like Sacrifice of Angels, look pristine by comparison. I've tried various noise filters (as well as no noise filter) after performing the IVTC on the film sections and the result, when upscaled by Topaz, looks like garbage.

Here's a sample of IVTC'd source footage of the station. How can this possibly be upscaled to look okay?
TNLmeans or KNLmeans. Pure motherf***ing magic.

Quote:
Originally Posted by videoh View Post
Should I re-send my friend request that you ignored? I feel so bad about Boris kicking you off my web site. You can come back any time and I will personally protect you. Your knight in shining armor!
Well the only reason I was on your website in the first place was to ask that ONE question about 48 -> 44.1 khz conversion and what the differences were between the quality settings. It's not like I was going to stick around one way or another. I will, however, continue to hawk DGindex as the #1 best way to get video content from a VOB file into AVIsynth


Joel, meet the legendary Donald Graft, also known as Neuron2. Back in the old days, he created he original Decomb plugin, which was the preferred deinterlacing tool used in the older versions of READFAG, written by the equally legendary Justin Emerson, also known as ErMaC. He also forked DVD2AVI (another program endorsed by early versions of READFAG) to create DGindex. Decomb has since been made obsolete by Tritical's TIVTC, but DGindex is still in use.
__________________
I ask unusual questions but always give proper thanks to those who give correct and useful answers.

Last edited by Katie Boundary; 18th May 2020 at 18:03.
Katie Boundary is offline   Reply With Quote
Old 18th May 2020, 18:29   #246  |  Link
videoh
Useful n00b
 
Join Date: Jul 2014
Posts: 1,667
A little aside on tritical's stuff. It is heavily based on the Decomb design. He made it when I went AWOL on further development of Decomb due to my father's illness. tritical did a fantastic job on picking up the torch. I was actually the first person to develop IVTC based on field-matching plus decimation, although it wasn't my original idea. A poster at Avery Lee's VirtualDub forum first suggested field-matching plus decimation, but he was not a developer. I wish I could remember his name; sadly the forum is now off-line and I can't go look it up. If anybody remembers, please let me know.

Of course the inspiration for DGMPGDec (DGIndex plus DGDecode) was jackei's amazing DVD2AVI. Trying to remember who wrote the first Avisynth DLL working with DVD2AVI...our late and beloved trbarry? Help me out guys.

The good old days!

Last edited by videoh; 19th May 2020 at 04:10.
videoh is offline   Reply With Quote
Old 18th May 2020, 19:24   #247  |  Link
JoelHruska
Registered User
 
Join Date: May 2020
Posts: 77
Hidden dependency: Probably of little interest to anyone except myself (since it's part of one of my own workflows), but in the name of sharing information:

My repair scripts that utilize QTGMC will not run against output from DaVinci Resolve Studio. Once the output has passed through DaVinci Resolve Studio and been converted to 119.88 fps, QTGMC's progressive repair mode will fail to engage (at least, when used in Staxrip). Instead, the application will hang. It will continue to do this, even if the framerate on the DRS output is decimated back to 23.976 fps using ChangeFPS(24000,1001). I am now attempting it on footage that used TDecimate instead of ChangeFPS and will report if it works.

You can still engage QTGMC in a standard preset mode, i.e.,
Quote:
QTGMC("Medium")
I am not sure if it is something specific about the flags I use or just the attempt to engage the progressive repair mode, but it doesn't work.

Typically I have 1). processed the DVD footage in AviSynth, 2). changed the frame rate in DaVinci Resolve Studio, and then 3). upscaled the final product in Topaz VEAI. In this workflow I decided to test; 1). changing the frame rate, 2). upscaling the video, and 3). Attempting to process the upscaled footage the same way I typically do, just to see an apples-to-apples comparison of the final output. I did this to avoid throwing data away early in QTGMC before performing the upscale. It took eight days to upscale the video in this fashion, which is why I'm just now reporting on the results of a test that I started like four pages ago.

However, my Step #3 cannot be meaningfully applied in this workflow.

Not necessarily a problem or anything, just logging that it occurs.

Last edited by JoelHruska; 18th May 2020 at 19:43.
JoelHruska is offline   Reply With Quote
Old 18th May 2020, 19:28   #248  |  Link
JoelHruska
Registered User
 
Join Date: May 2020
Posts: 77
Zapp7,

Crapstation is a problem I haven't solved yet, either. Like you, I've been very unhappy to discover how bad the early footage is.

Last edited by JoelHruska; 18th May 2020 at 19:51.
JoelHruska is offline   Reply With Quote
Old 18th May 2020, 20:41   #249  |  Link
videoh
Useful n00b
 
Join Date: Jul 2014
Posts: 1,667
Quote:
Originally Posted by JoelHruska View Post
Probably of little interest to anyone except myself
Correct, nobody cares.
videoh is offline   Reply With Quote
Old 18th May 2020, 20:41   #250  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,823
Quote:
Originally Posted by Katie Boundary View Post
OR, you could just tweak TFM's settings. Setting cthresh and mthresh to 2 or less, and using clip2 to retrieve pixels from a temporally-aware bob-deinterlacer like Yadif, works miracles.
Maybe if Katie wasn't ignoring so many people, she'd be aware of the full conversations. The statement was "TFM's default de-interlacing sucks balls for the CGI sections. When I realised it didn't look anywhere near as good as my video card's de-interlacing, I switched it out for PP=5. Much better."

TFM is capable of six different types of deinterlacing.

Still, what do I know, so I ran another VFR encode using cthresh=2, mthresh=2 and Yadif as the de-interlaced clip.
Aside from the fact you've only got to look at the timecodes to see the low cthresh and mthresh settings cause TFM to think there's much more combing, and therefore to declare multiple slices of the film sections are interlaced instead of telecined, replacing the pixels detected as combed with Yadif pixels causes the same problems as TFMs default de-interlacing.
I could tell when TFM was going into interlaced mode for the Katie method, if for no other reason, because the stars would blink out.

cthresh=2, mthresh=2, Yadif de-interlacing


TFM pp=5


cthresh=2, mthresh=2, Yadif de-interlacing


TFM pp=5


manono,
I may have debunked my theory that this:
DeintClip = TFM(pp=1).Yadif()
Should be better than this:
DeintClip = Yadif()

When I compared the two using Katie's de-interlacing, it sometimes looked better, but other times it didn't, then for the last frame on a scene change where the next shot is mainly black space, I found this. I'm not sure I fully understand why, although it's an example of how much of a progressive frame can be unnecessarily de-interlaced with low cthresh and mthresh settings.


Last edited by hello_hello; 18th May 2020 at 23:41.
hello_hello is offline   Reply With Quote
Old 18th May 2020, 20:57   #251  |  Link
Katie Boundary
Registered User
 
Katie Boundary's Avatar
 
Join Date: Jan 2015
Posts: 1,048
Quote:
Originally Posted by videoh View Post
The good old days!
Haha yes. When Smartripper was new, monitors were fullscreen, ATI still made "All-in-Wonder" cards, and we all got our anime music videos from Kazaa and Morpheus because Youtube didn't exist yet. I wasn't into video editing back then, but my mentor (Fuzzy Chickens) was, so I have been instructed in the Old Ways

Quote:
Originally Posted by videoh View Post
Of course the inspiration for DGMPGDec (DGIndex plus DGDecode) was jackei's amazing DVD2AVI. Trying to remember who wrote the first Avisynth DLL working with DVD2AVI...our late and beloved trbarry? Help me out guys.
Mpeg2dec3.dll is credited to "MarcFD, Nic, trbarry, Sh0dan and others", but was based on mpeg2dec2, which is credited to Mr. Barry alone. Mpeg2dec2 was in turn based on mpeg2dec, credited to "Dividee and others"
__________________
I ask unusual questions but always give proper thanks to those who give correct and useful answers.

Last edited by Katie Boundary; 18th May 2020 at 21:17.
Katie Boundary is offline   Reply With Quote
Old 18th May 2020, 21:07   #252  |  Link
Katie Boundary
Registered User
 
Katie Boundary's Avatar
 
Join Date: Jan 2015
Posts: 1,048
h_h, there's something very wrong with your settings or your script. That looks straight YADIFed, with no TFM at all (or like TFM is set to one of its "dumb" deinterlacing modes, which results in the same thing).
__________________
I ask unusual questions but always give proper thanks to those who give correct and useful answers.
Katie Boundary is offline   Reply With Quote
Old 18th May 2020, 21:07   #253  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,823
Quote:
Originally Posted by JoelHruska View Post
Typically I have 1). processed the DVD footage in AviSynth, 2). changed the frame rate in DaVinci Resolve Studio, and then 3). upscaled the final product in Topaz VEAI. In this workflow I decided to test; 1). changing the frame rate, 2). upscaling the video, and 3). Attempting to process the upscaled footage the same way I typically do, just to see an apples-to-apples comparison of the final output. I did this to avoid throwing data away early in QTGMC before performing the upscale. It took eight days to upscale the video in this fashion, which is why I'm just now reporting on the results of a test that I started like four pages ago.
Wow, and that's easier than adding a few Trims to a script?
Why on earth are you outputting 120fps if you're just going to decimate it back to 23.976?
The whole idea of 120fps is it's a multiple of both 24 and 60, so even though there's lots of repeated frames the interlaced sections still play at their original speed, as do the film sections.

Thinking about it, TDecmiate(mode=7, rate = 23.976) mightn't be the best tool for the job. The output will have a constant frame rate, so to achieve 23.976fps you might end up with the interlaced and film sections effectively playing at the wrong speeds. If the Avisynth output was 23.976fps though, it should be fine.
Maybe I'm missing something, but I don't get it. If the output from Avisynth is 23.976 and you're decimating to 23.976, why the conversion to 120fps and back? Aren't you just creating a heap of duplicate frames that you have to process, only to throw them away later?

After trying some VFR encodes myself, using TFM for a VFR encode from Avisynth seems like the best method anyway, so I don't know why you wouldn't want to try it.

Last edited by hello_hello; 18th May 2020 at 21:10.
hello_hello is offline   Reply With Quote
Old 18th May 2020, 21:17   #254  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,823
Quote:
Originally Posted by Katie Boundary View Post
h_h, there's something very wrong with your settings or your script. That looks straight YADIFed, with no TFM at all (or like TFM is set to one of its "dumb" deinterlacing modes, which results in the same thing).
There was nothing wrong with my settings. I ran a VFR encode using the settings you recommended for TFM.

Quote:
Crop(8,0,-8,0, Align=true)
DeintClip = Yadif()
TFM(Input="TFMMetrics.txt", clip2=DeintClip, cthresh=2, mthresh=2, micmatching=0)
TDecimate(Mode=5, Hybrid=2, Input="TDecimateMetrics.txt", tfmIn="TFMMetrics.txt", mkvout="TimeCodes.txt")
Resize8(640,480)
Not every frame looks so bad, just the ones where TFM was taking a lot of pixels from Yadif.

The "good" screenshots used the following script:

Quote:
Crop(8,0,-8,0, Align=true)
TFM(Input="TFMMetrics.txt", PP=5, micmatching=0)
TDecimate(Mode=5, Hybrid=2, Input="TDecimateMetrics.txt", tfmIn="TFMMetrics.txt", mkvout="TimeCodes.txt")
Resize8(640,480)

Last edited by hello_hello; 18th May 2020 at 21:36.
hello_hello is offline   Reply With Quote
Old 18th May 2020, 21:25   #255  |  Link
Katie Boundary
Registered User
 
Katie Boundary's Avatar
 
Join Date: Jan 2015
Posts: 1,048
It shouldn't be taking ANY pixels from YADIF except in the parts that are interlaced, in frames that are interlaced. I've never seen results like that from my own scripts.
__________________
I ask unusual questions but always give proper thanks to those who give correct and useful answers.
Katie Boundary is offline   Reply With Quote
Old 18th May 2020, 21:39   #256  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,823
Your cthresh and mthresh settings are so low it thinks large chinks are interlaced, although I assume you actually mean combed.

I don't need to get into a debate about it. It's pointless anyway. I can still remember when you argued the evils of IVTC with TFM, or deinterlacing with NNEDI3 and Yadif.

I've run quite a few test encodes using the dual TFM method you love, a heavily modified version of it using two QTGMC de-interlaced clips to make it not suck, and encodes using a single instance of TFM with QTGMC de-interlacing that was virtually as good (for when a constant 23.976 frame rate was required), but after trying VFR encoding when poisondeathray linked to a sample of a purely interlaced CGI section, the above VFR output using PP=5 has easily produced the best result.
I've linked to sample encodes in quite a few posts if you want to look for yourself, assuming you can play MKVs like the rest of us now, but please don't just tell me I'm wrong or I've used the wrong settings if you think you can do better, because it's easy to upload a sample of your own to prove it.

There's links for the sample source files I used earlier in the thread somewhere.
Edit: I think this was the first one:
https://forum.doom9.org/showthread.p...94#post1906994
Second one here:
https://forum.doom9.org/showthread.p...67#post1912067

I joined them like this for encoding:

Quote:
A = DGDecode_mpeg2source("D:\VTS_02_1_sample.d2v")
B = DGDecode_mpeg2source("D:\space battle.d2v")
A + B
VFR samples here:
https://forum.doom9.org/showthread.p...88#post1912388
CFR sample here:
https://forum.doom9.org/showthread.p...41#post1911841

Last edited by hello_hello; 18th May 2020 at 23:22.
hello_hello is offline   Reply With Quote
Old 18th May 2020, 22:08   #257  |  Link
Katie Boundary
Registered User
 
Katie Boundary's Avatar
 
Join Date: Jan 2015
Posts: 1,048
I never argued that those were "evil". Lies like that are why you're on my ignore list.

There is another argument to be made in favor of pp=5, however. My current deinterlacing algorithm heavily favors field-accuracy, the representation of the original fields and frames as accurately as possible in the newly deinterlaced frames, and the minimization of blended frames and residual interlacing. Why? because the obsessive-compulsive demons who whisper in my ear demanded it. It can't fully eliminate visible interlacing without a high false positive rate, which it conceals pretty well with temporally-aware bobbers (Yadifmod2). But there are types of content where that's not the right tradeoff to make, and blending is better than an autistically literal representation of the original content. The most obvious is the aforementioned retrograde field behavior, but fades, credit animations, and a few other effects are harmed less by blending than the rest of the frame is by bobbing. There are also times when the last field of one clip and the first field of the following clip will be interlaced together and encoded as a single frame, which causes their chroma planes to merge; in such cases, each field benefits from being matched and blended with the nearest field from its own clip rather than bobbed. This doesn't mean that cthresh and mthresh should be set higher, however. You can't do that without unacceptable amounts of visible interlacing getting through.
__________________
I ask unusual questions but always give proper thanks to those who give correct and useful answers.

Last edited by Katie Boundary; 18th May 2020 at 22:28.
Katie Boundary is offline   Reply With Quote
Old 18th May 2020, 22:30   #258  |  Link
JoelHruska
Registered User
 
Join Date: May 2020
Posts: 77
Quote:
Wow, and that's easier than adding a few Trims to a script?
Testbeds do 100% of the work. I set up the encode, walk away, come back 4-8 hours (or 4-8 days) later and check the output. The Trim method is something I need to learn from scratch, using tools I have not found easy to work with. I've been experimenting with the code you've posted and trying to figure out how this different approach to editing works, while continuing some evaluation on my own previously-developed workflow.

Quote:
Why on earth are you outputting 120fps if you're just going to decimate it back to 23.976?
To see what the outcome would be. Exactly the same reason that I have two encodes running downstairs since last night, converting the 8-day Emissary encode I did -- one with ChangeFPS(24000,1001) and one with the TDecimate command you suggested, to see which handles the content better, and how smooth the output is.

I wanted to compare the outputs against what native 23.976 fps video looked like, versus video run through progressive repair mode in QTGMC, etc, etc.

Quote:
Maybe I'm missing something, but I don't get it. If the output from Avisynth is 23.976 and you're decimating to 23.976, why the conversion to 120fps and back? Aren't you just creating a heap of duplicate frames that you have to process, only to throw them away later?
Well, keep in mind I reversed the workflow in this manner to test the impact of preserving detail for longer. I wanted to compare the visual impact of running QTGMC later in the workflow, which meant keeping the workflow identical except for running QTGMC later in the process. Unfortunately, this proved impossible, because my script won't execute after running DaVinci Studio Resolve.

EDIT: Using TDecimate instead of ChangeFPS doesn't change whether or not my repair script will run after DaVinci RS. It still will not. 2

I will not be retaining this workflow. I'm not spending 4 days of upscaling per episode.

But as for why I'd try it in the first place? Because I am new at this, and do not know what will work and what will not, and have been willing to burn CPU cycles on things I was pretty sure wouldn't work to see what they could teach me. As one simple example: I noticed using QTGMC in default mode would create an interpolated frame in-between each "regular" frame. After I noticed this, I ran multiple tests to look at the difference between SelectOdd and SelectEven. At one point, I double-ran QTGMC and then tried "SelectOdd,SelectOdd," "SelectOdd,SelectEven", "SelectEven, SelectEven", etc. Why? Because I wanted to understand what kind of output would be produced.

All of them were garbage, but some of them were more garbage than others.

"Test it and see what it looks like," has been my guiding philosophy on this endeavor.

Last edited by JoelHruska; 19th May 2020 at 00:59.
JoelHruska is offline   Reply With Quote
Old 18th May 2020, 23:04   #259  |  Link
wonkey_monkey
Formerly davidh*****
 
wonkey_monkey's Avatar
 
Join Date: Jan 2004
Posts: 2,493
Quote:
Originally Posted by videoh View Post
Correct, nobody cares.
Why didn't you just not say anything?
__________________
My AviSynth filters / I'm the Doctor
wonkey_monkey is offline   Reply With Quote
Old 18th May 2020, 23:07   #260  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,823
Quote:
Originally Posted by Katie Boundary View Post
I never argued that those were "evil". Lies like that are why you're on my ignore list.
I'm on your ignore list because you're a child.

Obviously I'm not on it now though, so you're lying about that, but given you're claiming I'm on it, you couldn't have read my posts, which means you couldn't have replied, and therefore I don't have to bother reading it.

I said you "argued the evils". How many times have you accused others of not being able to understand plain English?

Quote:
Originally Posted by Katie Boundary View Post
So, with almost no noticeable difference between AVIsynth's built-in Bob() filter and an actual proper field match, why should I believe that there's a huge difference between Bob() and more computationally intensive, easier-to-screw-up bob-deinterlacers like Yadif and NNEDI?

Last edited by hello_hello; 18th May 2020 at 23:47.
hello_hello is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 20:06.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.