Deinterlacing advice

[flossy_cake]

Hello, I am interested in deinterlacing the Bluray release of classic comedy show Fawlty Towers. The Bluray is 1080i at mostly 1:1 cadence for the indoor shots, and rarely has some outdoor shots at 2:2.

My goal is to convert it to 1080p50 files, preserving the full framerate of the 1:1 cadence sections. I am not bothered if the 2:2 cadence sections aren't weaved to true 1080p frames since they are rare.

In researching I came across this ranking site for deinterlacers: https://videoprocessing.ai/benchmarks/deinterlacer.html

Highlighted in red are the ones with seemingly acceptable frame rates, otherwise it seems I would have to leave my PC on for several days at 100% CPU load which is not practical for me.



So far these are my findings of the deinterlacers highlighted in red:

Bob
I didn't find this acceptable since it is too flickery and wastes vertical resolution. It does preserve the full framerate of 1:1 cadence though, so I would still prefer it to any half framerate deinterlacer.

Vapoursynth EEDI3, Vapoursynth TDeintMod
I haven't tried these as I'm not familiar with Vapoursynth yet. Can I use them through ffmpeg? I'm new to all this and my understanding is limited.

Weston 3-field, Yadif, Bob-weave
I was able to test these as they are included with ffmpeg.

Weston 3-field: seems to be identical to bob? I can't see a significant improvement vs bob. I'm not sure if this is due to incorrect implementation in ffmpeg as there are articles saying it was developed by a BBC engineer so it really should be better than bob.

Yadif: seems decent but fails to resolve high contrast 1px patterns in vertical direction, and has some wrong colour pixels on certain patterns. It is also very old.

Bob-weave: seems decent. Resolves high contrast 1px vertical patterns albeit with some random flickering/glitches. But apart from that it seems to fit the bill.

.zip file with some short video clips comparing the above 3
https://drive.google.com/file/d/1aL_...ew?usp=sharing

_____________________________________________________

Before I commit to using ffmpeg's bob-weave deinterlacer, are there any others that might suit my requirements?

For example, my Nvidia GPU has its own DXVA2 deinterlacing which is ok -- is it possible to somehow use that deinterlacer during the transcoding process?

Thanks

[Emulgator]

QTGMC ?
But thanks for the MSU comparison, MFDIN-L seems the way to go if AI can be used.
Open Source Soon it says.

[Katie Boundary]

Fawly Towers was made in the freaking SEVENTIES, dude. 1080i didn't exist back then. Who the hell authored those Blu-Rays?

Actually, it doesn't matter. Shows were shot and edited 100% on film back then so there's little to no chance of any orphaned fields. Just run it through tfm(mode=0,pp=0,slow=2,mchroma=false,micmatching=0) in AVIsynth and you should be good to go.

[flossy_cake]

Originally Posted by hello_hello

I'd be interested to see a sample of the Bluray version of Faulty Towers.

Here is a transition from outdoor 2:2 to indoor 1:1
https://drive.google.com/file/d/1k8M...ew?usp=sharing

Thank you for the QTGMC sample, it looks really good. I'm not sure why it's bobbing the 1px pattern -- most deinterlacers on TV's and PVRs weave that section, but they have a lot more aliasing on moving parts of image.

Originally Posted by Katie Boundary

Fawly Towers was made in the freaking SEVENTIES, dude. 1080i didn't exist back then. Who the hell authored those Blu-Rays?

Actually, it doesn't matter. Shows were shot and edited 100% on film back then so there's little to no chance of any orphaned fields. Just run it through tfm(mode=0,pp=0,slow=2,mchroma=false,micmatching=0) in AVIsynth and you should be good to go.

The indoor scenes were shot on 1970s video cameras at interlaced 1:1 cadence, the outdoor scenes look like film 2:2 cadence.

They still shouldn't have authored it at 1080i as scaling from interlaced to interlaced requires deinterlacing otherwise you get mice teeth. So they should have authored it at 1080p50 and avoided a second round of deinterlacing at the user's end. Maybe they used 1080i for compatibility reasons I don't know.

[Katie Boundary]

Originally Posted by flossy_cake

The indoor scenes were shot on 1970s video cameras

Jesus, the first season was made in literally the same year in which the first Betamax devices were released in Japan. How cutting-edge they must have thought they were being

I think any reasonable solution to this issue would require ripping the episodes from DVD, not Blu-Ray.

[hello_hello]

Originally Posted by Katie Boundary

Jesus, the first season was made in literally the same year in which the first Betamax devices were released in Japan. How cutting-edge they must have thought they were being

Video tape has been used since the late 50's.
https://en.wikipedia.org/wiki/Videotape#Broadcast_video

Originally Posted by Katie Boundary

I think any reasonable solution to this issue would require ripping the episodes from DVD, not Blu-Ray.

The Bluray version looks better.

[flossy_cake]

Originally Posted by hello_hello

The Bluray version looks better.

It's a bit contentious actually! On other forums some are saying the "remastered DVD" is still better as it doesn't have DNR and sharpening of the BD version (plus the aspect isn't squished, but they didn't notice that).

Personally I am satisfied with the BD version but only because we can unsquish the aspect. If all I had was a BD player without fine-tuning of the aspect then I probably wouldn't find it acceptable.

Is your DVD the original or remastered?

[hello_hello]

I'd be interested to see a sample of the Bluray version of Faulty Towers. The DVD version isn't great quality, but if it's been interpolated to 1080i from PAL it might be hard to de-interlace and require a better/slower de-interlacer.

QTGMC would require a crash course in Avisynth (or VapourSynth) and it is slow, but there's a fair difference compared to the samples you uploaded.

Avisynth script:

DGDecode_mpeg2source("D:\1 - original interlaced.d2v")
QTGMC()

QTGMC 1.mkv

The test pattern thingy at the beginning and end isn't exactly a real-world example of interlaced video, but because there's no movement, in a perfect world you'd treat it as progressive and just duplicate each frame.

DGDecode_mpeg2source("D:\1 - original interlaced.d2v")
Trim(0, 81).ChangeFPS(50) ++ \
Trim(82, 647).QTGMC() ++ \
Trim(648, 0).ChangeFPS(50)

QTGMC 2.mkv

[flossy_cake]

Originally Posted by hello_hello

The test pattern thingy at the beginning and end isn't exactly a real-world example of interlaced video, but because there's no movement, in a perfect world you'd treat it as progressive and just duplicate each frame.

I'd really like to get into writing my own deinterlacer as I have all these ideas in my head which I'd like to try out and see if they're any good in practice.

One idea was to simply weave fields and compare that frame with previous 1 or 2 frames and if the pixels are the same (or similar within some threshold) then there was no change in those pixels and they can stay weaved. By this rule the 1px pattern should get weaved. BWDIF has some random flickering on the 1px pattern -- perhaps its algorithm sees some slight change in those pixels, maybe just compression noise?

I'd also like to have cadence detection working in the background to detect the overall cadence, and if it detects 2:2 it switches to weaving full frames.

But then there is the difficulty of handling a scene with mixed cadences. In that case all I can think of is to just treat it as 1:1 cadence probably. Unless maybe I could split the raster up into zones and perform cadence detection within each zone. But I'm not good enough with code and math/geometry to know how to implement something like this.

[flossy_cake]

Sorry to ramble on a bit, but I've got another issue I'm working on which is kind of related to deinterlacing.

I've come across some TV shows that were shot at 24p, then interlaced to 30i (3:2 cadence, aka "hard telecine", aka "24p as 60i"). Then this 30i was deinterlaced to 30p at half framerate. The problem is that it ends up as something like a 1:1:1:2 cadence. The interesting part is that the 24p frames still exist inside the 30p stream, so in theory it is possible to pull them back out again and create a true 24p stream from it. Has anyone ever made a script that can do something like this? I've seen there is a function called decimate() -- should I look at trying to use that or will I run into issues?

edit: clip https://drive.google.com/file/d/1t03...ew?usp=sharing

edit: oh wow it seems MadVR can do this in realtime by tagging the file with [deint=film]. In its debug screen (ctrl+j) it reports that it detected a 4:2:2:2 cadence in the above clip. Result is true 24p @ 1:1 cadence outputting as 24hz over HDMI -- amazing. edit: but it only seems to work if HDMI is outputting 24hz. With 60hz output it's still displayed as 1:1:1:2 (or 4:2:2:2 as MadVR calls it) even though stepping through each frame with ctrl+arrows shows a 1:1 cadence. edit: after further scrutiny @ 24hz output, the cadence detection is not getting it quite right after certain scene changes where it may stutter for a bit before correcting itself so it's not quite a solution unfortunately. Whatever the solution, I think it would need a very high quality cadence detection (with "bad edit" detection or whatever it's called when the cadence changes on a scene splice) and perhaps end up as a 60p stream to hide them well enough to not cause big stutters.

[hello_hello]

Originally Posted by flossy_cake

Sorry to ramble on a bit, but I've got another issue I'm working on which is kind of related to deinterlacing.

I've come across some TV shows that were shot at 24p, then interlaced to 30i (3:2 cadence, aka "hard telecine", aka "24p as 60i"). Then this 30i was deinterlaced to 30p at half framerate. The problem is that it ends up as something like a 1:1:1:2 cadence. The interesting part is that the 24p frames still exist inside the 30p stream, so in theory it is possible to pull them back out again and create a true 24p stream from it. Has anyone ever made a script that can do something like this? I've seen there is a function called decimate() -- should I look at trying to use that or will I run into issues?

edit: clip https://drive.google.com/file/d/1t03...ew?usp=sharing

edit: oh wow it seems MadVR can do this in realtime by tagging the file with [deint=film]. In its debug screen (ctrl+j) it reports that it detected a 4:2:2:2 cadence in the above clip. Result is true 24p @ 1:1 cadence outputting as 24hz over HDMI -- amazing. edit: but it only seems to work if HDMI is outputting 24hz. With 60hz output it's still displayed as 1:1:1:2 (or 4:2:2:2 as MadVR calls it) even though stepping through each frame with ctrl+arrows shows a 1:1 cadence. edit: after further scrutiny @ 24hz output, the cadence detection is not getting it quite right after certain scene changes where it may stutter for a bit before correcting itself so it's not quite a solution unfortunately. Whatever the solution, I think it would need a very high quality cadence detection (with "bad edit" detection or whatever it's called when the cadence changes on a scene splice) and perhaps end up as a 60p stream to hide them well enough to not cause big stutters.

3:2 pulldown duplicates two fields every eight, so eight fields are spread out over ten fields. The field matching part of inverse telecine usually throws one of the duplicate fields away while the other duplicates are both matched. The end result is 5 progressive frames, one of which is a duplicate. When it's removed you're back to 24p. For the TIVTC Avisynth plugin, TFM does the field matching and/or de-interlacing, and TDecimate removes the duplicates. It can also convert interlaced sections to 24p, or 24p sections to 30p. It also has a 2 pass mode for variable frame rate encoding.

For typical NTSC inverse telecine the defaults usually work fine, so all you need in a script is

TFM()
TDecimate()

If the field matching is already done and there's a duplicate frame in every five, then you only need

TDecimate()

Decimate() does the same thing.

[hello_hello]

Originally Posted by flossy_cake

I'd really like to get into writing my own deinterlacer as I have all these ideas in my head which I'd like to try out and see if they're any good in practice.

One idea was to simply weave fields and compare that frame with previous 1 or 2 frames and if the pixels are the same (or similar within some threshold) then there was no change in those pixels and they can stay weaved. By this rule the 1px pattern should get weaved. BWDIF has some random flickering on the 1px pattern -- perhaps its algorithm sees some slight change in those pixels, maybe just compression noise?

I'd also like to have cadence detection working in the background to detect the overall cadence, and if it detects 2:2 it switches to weaving full frames.

But then there is the difficulty of handling a scene with mixed cadences. In that case all I can think of is to just treat it as 1:1 cadence probably. Unless maybe I could split the raster up into zones and perform cadence detection within each zone. But I'm not good enough with code and math/geometry to know how to implement something like this.

For de-interlacing to double frame rate (50p for PAL) bob de-interlacers usually extract each field, which is every second scan line, then interpolate the missing scan lines for a full frame. So 50i becomes 50p.
For that 1x pattern, every second scan line is black and the others are white, so the black scan lines are interpolated to a full frame by creating additional black scan lines, and the white scan lines are interpolated with additional white scan lines. The result is one frame of pure black and the next is pure white and so on... As the black and white scan lines are offset for each half, they alternate black and white.
That's definitely not a real world de-interlacing job though, but how BWDIF avoids it in double frame rate mode I don't know. Hopefully someone who does will come along.

When de-interlacing to the original frame rate, some de-interlacers weave the fields and check for combing, and only repair the pixels where combing is found. For TFM(), which can field match and de-interlace, the default is to check for combed pixels and only repair those. The rest are left alone. You can even take the pixels to be used to repair combing from an externally de-interlaced clip if you wish. An Avisynth script could look something like this:

Some Telecined Or Interlaced Video
A = last
B = A.QTGMC(FPSDivisor=2)
C = A.TFM(Clip2=B)
return C

[hello_hello]

flossy_cake,
Are you familiar with the resizing mess for DVDs?
The short story is DVDs can have a generic aspect ratio, which makes a 4:3 DVD exactly 4:3, or an ITU aspect ratio which follows the standard for digitising video, and as a result the aspect ratio could be ~1.36 instead of 1.33 (before any cropping). I'm pretty sure I went with an ITU aspect ratio for the DVD version, and the first thing I noticed when comparing it to the Bluray was the difference in aspect ratio, which looks to be exactly the difference between a generic and ITU aspect ratio for DVDs. Here's a couple of screenshots. Have a look at the clock-face on the left. I'll leave you to decide in which screenshot it looks round, or for that matter, if it's even supposed to be round.

https://i.ibb.co/tJXf43p/DVD-Encode.png
https://i.ibb.co/61qLbTy/Bluray.png

By the way, the DVD encode was actually resized to 640x480 after cropping and de-interlacing, and resized to full-screen for the screenshot. The Bluray version definitely looks better but based on the amount of detail you're not going to lose anything by resizing down to 720p. Anyway....

Here's a couple of sample encodes for you. To help you possibly be indecisive, I stretched the Bluray to match my DVD encode (you can decide if it's correct) and cropped a few pixels top and bottom for exactly 4:3, because I'm OCD about 4:3 being the minimum aspect ratio. That's just me though....

Based on that sample BWDIF seems to do a pretty good job. There's a tad less shimmering with QTGMC but whether that's worth the speed reduction is up to you. Probably not.

FFVideoSource("D:\ft.mkv", cachefile="D:\ft.mkv.ffindex")
Crop(266,0,-266,0)
Trim(0,152).ChangeFPS(50) ++ \
Trim(153,0).BWDIF(field=-2).AssumeFPS(50)
CropResize(1440,1080, 0,0,-2,0, InDAR=1419.55/1080.0)

ft BWDIF.mkv

FFVideoSource("D:\ft.mkv", cachefile="D:\ft.mkv.ffindex")
Crop(266,0,-266,0)
Trim(0,152).ChangeFPS(50) ++ \
Trim(153,0).QTGMC().AssumeFPS(50)
CropResize(1440,1080, 0,0,-2,0, InDAR=1419.55/1080.0)

ft QTGMC.mkv

I'll have to dig out the original DVDs at some stage to make sure I didn't do anything odd to the aspect ratio.

Edit: I just noticed the very first frame in the video section is combed in both samples (check her left foot). That was my fault. The Trims in the script were off by a frame. It should've been:

FFVideoSource("D:\ft.mkv", cachefile="D:\ft.mkv.ffindex")
Crop(266,0,-266,0)
Trim(0,151).ChangeFPS(50) ++ \
Trim(152,0).QTGMC().AssumeFPS(50)
CropResize(1440,1080, 0,0,-2,0, InDAR=1419.55/1080.0)

[flossy_cake]

Originally Posted by hello_hello

I'm pretty sure I went with an ITU aspect ratio for the DVD version, and the first thing I noticed when comparing it to the Bluray was the difference in aspect ratio, which looks to be exactly the difference between a generic and ITU aspect ratio for DVDs. Here's a couple of screenshots. Have a look at the clock-face on the left. I'll leave you to decide in which screenshot it looks round, or for that matter, if it's even supposed to be round.

https://i.ibb.co/tJXf43p/DVD-Encode.png
https://i.ibb.co/61qLbTy/Bluray.png

I was looking into this and saw this comment mentioning Fawlty Towers was shot on these EMI 2001 cameras. Their spec sheet can be found here, but it doesn't mention aspect ratio, only that it uses the 625-line CCIR system (page 10). But the CCIR spec says the aspect is 4:3.

So the Blu-ray cover and camera spec are saying 4:3, however active pixel area is about 1.29:1, and this German release of the Bluray mentions 1.29:1 also.

What do you think -- does it look a little wide when stretched to 4:3? On some shots I think it does, but on others it doesn't really. I think the cameras might have some geometry distortion going on stretching it out at the edges, so when actors are in dead center of frame they look skinnier.

Considering the Bluray is a BBC release and BBC made the EMI 2001 camera, you'd think they would know what the aspect should be.

[flossy_cake]

Originally Posted by hello_hello

Are you familiar with the resizing mess for DVDs?
The short story is DVDs can have a generic aspect ratio, which makes a 4:3 DVD exactly 4:3, or an ITU aspect ratio which follows the standard for digitising video, and as a result the aspect ratio could be ~1.36 instead of 1.33 (before any cropping). I'm pretty sure I went with an ITU aspect ratio for the DVD version, and the first thing I noticed when comparing it to the Bluray was the difference in aspect ratio, which looks to be exactly the difference between a generic and ITU aspect ratio for DVDs. Here's a couple of screenshots. Have a look at the clock-face on the left. I'll leave you to decide in which screenshot it looks round, or for that matter, if it's even supposed to be round.

https://i.ibb.co/tJXf43p/DVD-Encode.png
https://i.ibb.co/61qLbTy/Bluray.png

Out of curiosity I bought the newest version of the DVD ("remastered" one, specifically this one which was referenced here as being superior to the BD).

Well, the DVD aspect is identical to the BD, except the BD is slightly zoomed in to remove a few pixels of blanking at the bottom edge present on the DVD.

Oddly though, the external shots are a lot worse on DVD in terms of colour and resolution, except for the intro sequence.

Edge enhancement seems present in the DVD too, but diminished probably as a result from the downscale from 1080 to 576. Noise reduction is not present in the DVD it seems.

Screenshots:

DVD: https://i.lensdump.com/i/RR6Hyr.png
BD: https://i2.lensdump.com/i/RR6Oe7.png

DVD: https://i3.lensdump.com/i/RR6S73.png
BD: https://i1.lensdump.com/i/RR6zRF.png

DVD: https://i2.lensdump.com/i/RR6eU0.png
BD: https://i.lensdump.com/i/RR6vID.png

Clips:

DVD: https://drive.google.com/file/d/1qcU...usp=share_link
BD: https://drive.google.com/file/d/1a-N...usp=share_link

DVD: https://drive.google.com/file/d/1pAw...usp=share_link
BD: https://drive.google.com/file/d/1tw9...usp=share_link

[poisondeathray]

An alternative to BWDIF is Tdeint as motion adaptive deinterlacer to weave progressive content sections . You can use edeint with a 2nd clip for the replacement pixels for either BWDIF or TDeint, such as QTGMC or NNEDI3, which will produce better results on the interlaced content sections, but slower. But it should be faster than pure QTGMC

The BW in BWDIF stands for Bob Weaver (so it weaves static content,in theory, similar to TDeint) - but you can see the distortions and glitches mentioned in the OP in that patterns when using BWDIF, or the text in the clip section. TDeint is much cleaner on that test pattern and diagonals (such as the eyeglasses) on the actual clip in terms of aliasing artifacts

tdeint(order=1, mode=1, edeint=nnedi3(field=3))
prefetch(something)

This will produce good results on a test pattern, but it will usually produce worse results on typical interlaced content compared to full QTGMC in terms of aliasing and flicker artifacts. On the other hand, the weaved progressive sections will have full resolution compared to QTGMC. Distributed interlaced content is usually low passed in a dumb manner (even static sections which shouldn't be low passed), so you often never have 1 pixel vertical resolution content. But TDeint is a good alternative, if you have full resolution progressive sections, where you don't want QTGMC smoothing details too much

The other option is VFR, where progressive sections are weaved and decimated, and interlaced sections are deinterlaced, everything runs at it's native content framerate. (25p at 25p, 29.97p at 29.97, 50p at 50p etc...) . I think ctools can do this, but I havent tested it fully yet. The original TIVTC VFR had issues with 50p/59.94p timecodes

[flossy_cake]

Thanks for the explanations and all the helpful code snippets, much appreciated.

And yes I did notice the squished aspect in Fawlty Towers! Also had a similar issue with the Family Guy DVD where some episodes are stretched to 1.36:1 and the character faces look a bit wide. The solution I'm currently using is to tag those files with [profile='aspect 1.36 to 1.33'] which tells MadVR to squish it back to 1.33. This way I don't have to re-encode anything, although MPC-HC does support opening .avs files and applying them in real time which is fantastic -- I'll try using TDecimate() on that Oz clip if I can get it working (currently it's giving me an error message saying Decimate function not found... probably need to update my Avisynth & dependencies).

For Fawlty Towers I measured a screenshot in Photoshop which came out to 1386x1080 of "active" pixels. 4:3 would be 1440x1080, so under the assumption the active pixels are meant to cover a 4:3 area, the MadVR "stretch factor" would be 1386/1440.

But of course this calculation could be wrong as I don't know the relationship between active pixels and the intended DAR with that 1970's equipment, eg. there could be some intentional "blanking" down the sides. But looking at it subjectively, the actor's faces look subjectively "better" to my eyes with the 1386/1440 correction. Although hello_hello matched it to the DVD with a correction factor of 1419.55/1440 so maybe that is the correct one. Interlace & non-square pixels -- what a disaster

[flossy_cake]

Originally Posted by flossy_cake

I'll try using TDecimate() on that Oz clip if I can get it working (currently it's giving me an error message saying Decimate function not found... probably need to update my Avisynth & dependencies).

Got it working!

But now I have a new problem

The show also contains some sections at true 30p 1:1 cadence, and TDecimate is now causing those sections to become stuttery.

I tried adding ChangeFPS(60) before/after the TDecimate() which according to MadVR's ctrl+j debug screen really did create a 60fps stream, but for some reason the 30fps 1:1 sections are stuttery. I would have thought TDecimate would have seen that there were no duplicates in that section and left them alone?

Clip: https://drive.google.com/file/d/10pq...ew?usp=sharing

The first 8 seconds are 1:1:1:2, followed by 5 seconds of 1:1. If you are using MPC-HC you can step through each frame with ctrl+arrows to count the cadence manually.

[flossy_cake]

Originally Posted by flossy_cake

The show also contains some sections at true 30p 1:1 cadence, and TDecimate is now causing those sections to become stuttery.

It seems I have 2 options:

Option 1. Use TDecimate(hybrid=1) to "blend decimation of 30p sections into 24p and leave 24p untouched". In practice this doesn't seem to work as I'm not observing any blended frames in the 30p section (cop walking up the stairs). Maybe I need to play with some other params like vidThresh.

Option 2. Try to somehow convert it to 60p so that the 24p sections end up at 3:2 and 30p sections at 2:2. This would be my preferred option but I can't get it to work for the 2:2 sections, only the 3:2 sections. I'm sure this is possible to achieve it's just a matter of finding the right settings.


edit: managed to force Option1 to work with a manual override to tell TDecimate the frames with the cop walking up the stairs to treat as video and blend to 24p:

TDecimate(mode=0,cycle=5,cycleR=1,hybrid=1,ovr="override.txt",display=true)

contents of override.txt:
275,417 v

It works and I can see the blending, but I'm not satisfied with the way it looks (too blurry). So my goal will be Option2.

[poisondeathray]

Originally Posted by flossy_cake

Option 2. Try to somehow convert it to 60p so that the 24p sections end up at 3:2 and 30p sections at 2:2. This would be my preferred option but I can't get it to work for the 2:2 sections, only the 3:2 sections. I'm sure this is possible to achieve it's just a matter of finding the right settings.

TFM().ChangeFPS(60000,1001)

This is encoded progressively, so you don't even need TFM(). Just ChangeFPS

But the duplicates in "1-1-1-2 to 1-1.mkv" are not "true" duplicates in the source 1st section - they are sort of blended encoding artifacts, eitherway you're just duplicating frames with ChangeFPS

It seems I have 2 options:

Option 3 is VFR. TIVTC 2pass VFR (TDecimate mode 4+5). Everything runs at their content frame rate, no "physical" duplicates (if you run a 60Hz display, your display will be adding effective duplicates), timestamps(timecodes) keep everything in sync