Deep Space Nine upscale project

[bladerunner1982]

I also had this upscale project in mind for the past three years. (taking long breaks in between )
From time to time I experimented a little bit with different scripts and
parameters using the first episode of season 6 (the first 20 minutes).
Here is the script I came so far using the NTSC DVDs:

myvideo = "ep1ntsc.d2v"
DGDecode_mpeg2source(myvideo, idct=5)

#deint = QTGMC().SelectEven()   #using QTGMC as deinterlacer optional
deint = tdeint()
tfm(d2v=myvideo,clip2=deint)
TDecimate(hybrid = 1)

QTGMC(InputType=1)   #using QTGMC on progressive frames again to clean up a little bit
#QTGMC( Preset="Slower", InputType=1 ) #optional

tweak(cont=1.3,sat=1.1)

ConvertToRGB24()
SuperResXBR(Passes=5, Str=1, Soft=0.0, XbrStr=2.7, XbrSharp=1.3, MatrixIn="PC601")
SuperResXBR(Passes=3, Str=1, Soft=0.0, XbrStr=2.7, XbrSharp=1.3, MatrixIn="PC601", fWidth=1440, fHeight=1080, fKernel="SSim")
ConvertToYV12()

addgrainc()
trim(0,26000)+trim(49114,50000)
AssumeFPS(25)

If someone is interested I can provide a link to the encoded result via pm.
Maybe it´s not a pro solution but for me it was ok.
I´m always open for suggestions.

[ryrynz]

Well the pro solution would be something similar to this that would be created with the help of people here. The project creator has far less knowledge of Avisynth filters. Wouldn't hurt to flick that across to him for starters. You could also maybe provide a, sample for us with this to evaluate and maybe improve upon?

[Stereodude]

Originally Posted by bladerunner1982

I also had this upscale project in mind for the past three years. (taking long breaks in between )
From time to time I experimented a little bit with different scripts and
parameters using the first episode of season 6 (the first 20 minutes).
Here is the script I came so far using the NTSC DVDs:

myvideo = "ep1ntsc.d2v"
DGDecode_mpeg2source(myvideo, idct=5)

#deint = QTGMC().SelectEven()   #using QTGMC as deinterlacer optional
deint = tdeint()
tfm(d2v=myvideo,clip2=deint)
TDecimate(hybrid = 1)

QTGMC(InputType=1)   #using QTGMC on progressive frames again to clean up a little bit
#QTGMC( Preset="Slower", InputType=1 ) #optional

tweak(cont=1.3,sat=1.1)

ConvertToRGB24()
SuperResXBR(Passes=5, Str=1, Soft=0.0, XbrStr=2.7, XbrSharp=1.3, MatrixIn="PC601")
SuperResXBR(Passes=3, Str=1, Soft=0.0, XbrStr=2.7, XbrSharp=1.3, MatrixIn="PC601", fWidth=1440, fHeight=1080, fKernel="SSim")
ConvertToYV12()

addgrainc()
trim(0,26000)+trim(49114,50000)
AssumeFPS(25)

If someone is interested I can provide a link to the encoded result via pm.
Maybe it´s not a pro solution but for me it was ok.
I´m always open for suggestions.

I'd just assume let MadVR upsample it with NGU during playback vs. SuperXBR in the script. But, getting it competently deinterlaced/IVTC'd as best as it can be would be nice.

What happens after 20 minutes? Why do you need to change parameters?

[Stereodude]

S06E01 is not a good sample IMHO. I just pulled it from the DVD to try bladerunner1982's script with my own twist. There are virtually no combed frames in the first 20 minutes that make it past the IVTC. The only real problem area in those 20 minutes is the wormhole opening at the end of the Intro. And, the script from bladerunner1982 does not handle it well. The default MI of 80 in TFM is way too high. It will pass visibly combed frames in that area. Cranking it down to 5 helps, but still doesn't get 100% of them. Basically you've got to take it to 1 which causes it to see lots small areas as needing deinterlacing in progressive sections, but the impact on the output seems negligible.

After playing around a bit I would suggest something like this:

source=MPEG2Source("VTS_02_1.d2v", idct=5, info=0)

deint=source.QTGMC( Preset="Slower", InputType=0, SourceMatch=3, Lossless=2, Sharpness=0.2, EZKeepGrain=0.0, ShowSettings=false ).SelectEven()

source.tfm(d2v="VTS_02_1.d2v", PP=7, MI=1, clip2=deint, display=false)

TDecimate(hybrid = 1, dupThresh=1.5, vidThresh=2.3, display=false)

MCTemporalDenoise(settings="low", radius=3)

f3kdb(range=16, grainY=15, grainC=10, sample_mode=2, dither_algo=3, dynamic_grain=true, keep_tv_range=false, output_depth=10)

Then encode it with a SAR of 8:9

[SaurusX]

Sounds like the series would require TFM override files for every episode of the series. Any time theres a special effect the regular TFM approach would need to be examined to make sure it’s catching everything. If not, then add the frame sequence to the override file to be deinterlace. Not impossible, but very time consuming.

[bladerunner1982]

Using MadVR requires cpu und gpu power during playback. What about people watching on their SmartTV oder standalone player?
So why not take enough computing power in the upscale process before encoding?
Ending up with a 1440x1080 video file, PAR 1:1

I only encoded 20 minutes because I only have an i5-7500, took me 2 hours to encode.
Tested mainly the parameters of SuperResXBR, trying to find the balance between smoothening and sharpening.

So what episode would be best for testing?

[Stereodude]

Originally Posted by bladerunner1982

Using MadVR requires cpu und gpu power during playback. What about people watching on their SmartTV oder standalone player?

I didn't say my preference was right for everyone.

So what episode would be best for testing?

I'm not sure. I haven't ripped them all. I've only looked at 2: S06E01 and S01E01. It might not be a bad episode for testing, but I didn't see it in the first ~20 minutes. It's only 88.29% film per the D2V. S01E01 is 97% film.

Maybe there aren't that many problem areas in the typical episode when using TFM. Much of the special effects are telecined film content. They're not all interlaced video. I do wish Tdecimate had some different parameters for controlling the blend vs. drop in the hybrid mode.

[scharfis_brain]

btw.: does anyone know wheter there is an NTSC DVD release of DS9/VOY with german audio?

[Chir]

Greetings --

I'm actually tackling this project. I am no expert on this, but I've developed workflow that produces decent results. I came here looking for solutions to my problem where it looks like the frame rate drops -- it gets all jerky in scenes. It's usually heavily CG scenes. You guys helped explain a lot, and I appreciate that. No I understand its the mix of video and film.

I just started to learn about working with video at this level, so please be patient. My current workflow is this:

1. Rip with dvddecryptor to get each episode into a vob file.
2. Open the VOB in DGIndex; enable forced film and save the D2V file.
3. Create an AVISynth script that consumes the D2V file with the following as my template:

LoadPlugin("D:\Video\Tools\DGMPGDec\DGDecode.dll")
SetFilterMTMode("QTGMC",2)
MPeg2Source("D:\DS9S1D1\VIDEO_TS\VTS_02_1 - 0xE0 - Video - MPEG-2 - 720x480 (NTSC) - 4~3.d2v")
ConvertToYV12()
AssumeTFF()
QTGMC(Preset="slower", FPSDivisor=2, Edithreads=4)
Prefetch(threads=6)

4. Pull the video out with ffmpeg:

ffmpeg.exe -i <full path to .avs> -map 0:v -c:v copy <destination.mkv>

6. At this point, I upscale the video with VideoEnhance AI. It spits out PNG files for all the frames in the video file (destination.mkv)

7. I then reassemble the PNGs into Video with FFMPG using the original framerate from the DVD

ffmpeg -r 23.976 -f image2 -s 1920x1080 -i <path to png files>\%06d.PNG -vcodec libx265 -preset medium -crf 28 -pix_fmt yuv420p EncodedOutput.mkv

8. And finally, I mux everything back together with the original audio from the disc, chapters, and subtitles.


Its far from perfect.... and I'm hoping to pick up some tips to streamline my project and get a better result. I'm going to look into the sample scripts above to try and get a better understanding of what they're doing. I have a feeling I'm in over my head and doing unnecessary steps -- but I'm willing (and trying!) to learn!

[SaurusX]

Using QTGMC to deinterlace the image essentially means you’ll be upscaling a 240p picture to 1080p or 4K or whatever. You need to inverse telecine as your main process and deinterlace only the scenes you need to.

[manono]

Originally Posted by Chir

2. Open the VOB in DGIndex; enable forced film and save the D2V file.

Unless the source is 100% FILM (check the bottom of the D2V file), that is a very bad idea.

ConvertToYV12()

It's already YV12.

[bladerunner1982]

@Stereodude

I encoded your script combined with SuperResXbr (without the output_depth=10 parameter in f3kdb).
Looked good but the space scenes (starships) had motion judder.

@scharfis_brain
I don't think there is an ntsc version disc with german audio. I use the ntsc discs, speed them up after processing to 25fps at the end of my script, then I encode. Speed up the english track to 25fps as well, leave the german as it is.

@Chir
A.I. Gigapixel? You can feed the video directly in it using newest version. No need for frame based encoding.

There is now a folder at mega.nz which contains the testvideos generated by scripts in this thread.
Everyone is invited to have a look at it for comparison. PM me

If you want to contribute your testvideo send me a link. Please encode only first 20min of S01E01 or S06E01 without audio to 1440x1080 (I use x264,crf16,preset slow)

[Stereodude]

Originally Posted by bladerunner1982

@Stereodude

...
Looked good but the space scenes (starships) had motion judder.

Can you give me an exact timecode for when you see judder? Pretty much all the special effects scenes appear to be telecined film when I skimmed through it.

[bladerunner1982]

Timecode 01:38 ....right after the look back scene...check PM...

[Stereodude]

Originally Posted by bladerunner1982

Timecode 01:38 ....right after the look back scene...check PM...

It's a bit complicated. If you comment out the Tdecimate line in your script and use this TFM line instead:

source.tfm(d2v="VTS_02_1.d2v", PP=1, MI=1, clip2=deint, display=true)

and then step through the section frame by frame in VD2. There's clearly a duplicated frame that should be removed by tdecimate.

In looking at this, I did discover that I misunderstood the meaning of this section of the TFM help. "...if PP >= 5 (i.e. using motion adaptation) then TFM will build the mask as usual and only pixels in moving areas will be taken from the frames in clip2." It doesn't replace only areas where combing was found it like I was thinking it did. It replaces anywhere that motion was found. In this case because of the motion this changes the output frames significantly (basically completely replaces them).

So, in this section the low MI value set in TFM causes TFM to pull some of the frames from the deinterlaced version which then causes areas of irregular motion in the final output.

Changing the TFM line to:
source.tfm(d2v="VTS_02_1.d2v", PP=7, MI=1, clip2=deint, display=true, flags=5)

fixes the problem in this area. Flags=5 will prevent TFM from looking for combing in any part of the source (except scene changes) that's encoded as 24p with a soft pulldown flag on the DVD. This still leaves areas that are encoded as video.

To address those I would suggest increasing the MI some. Some minor combing will get through, but it will cut down on unnecessary frame replacement with softer deinterlaced frames.

Short of making a full manual override file where you manually and visually inspect every frame with a MIC > 0 you have the trade off of letting minor combing through vs. unnecessary frame replacement. The MI value sets the threshold.

I'm prepping a sample to upload and maybe we'll get some better ideas from others.


Edit: At least for S06E01 bumping up the MI to 24, adding flags=5 to TFM, using an override file for TFM, and an override file for tdecimate seems to basically fix the episode. (Note, I didn't do a frame by frame examination of the whole show.)
TFM Override:

13329,13945 -
16383,16504 +

The first keeps the comet at the start of the "intro" from being seen as combed. The second fixes the combing in the wormhole at the end of the "intro".

Tdecimate Override:

79880,81234 v

This forces the end credits to be blended.

Other episodes will need the same areas fixed with overrides, but of course the frame numbers will be different.

[Stereodude]

Originally Posted by ryrynz

Well the pro solution would be something similar to this that would be created with the help of people here. The project creator has far less knowledge of Avisynth filters. Wouldn't hurt to flick that across to him for starters. You could also maybe provide a, sample for us with this to evaluate and maybe improve upon?

Here's a sample. It's starts with a section that's video and then it's a section that alternates between film/video as the titles appear on screen.

https://mega.nz/file/1g9DBSpC

and what you'll need to get it:
64cZaOQ8mHnbvDRTthMf6M0RenYwdb3M0A7I_ufaQqc

[mbcd]

My try, looks good, but not perfect ... (hope you can download it):

https://1drv.ms/u/s!AuYZhY0Moi2f5hxSoo6pSTQlKl2u

LWLibavVideoSource("VTS_02_1_sample.m2v")
TFM(mode=1)
TDecimate(mode=0,hybrid=1)
AssumeFps(23.976)

There are still some blended frames left, so movement is not that smooth, but that scenes look almost ""fine"" ...

[mbcd]

Originally Posted by mbcd

My try, looks good, but not perfect, no magic done ... (hope you can download it):

https://1drv.ms/u/s!AuYZhY0Moi2f5hxSoo6pSTQlKl2u



There are still some blended frames left, so movement is not that smooth, but that scenes look almost ""fine"" ...

Look like field oder is changing, I get mostly duplicated bad frames on those ...

I dont think you will get a good result, they layed two different framerate together, title changes, but background is still (doubled frames) ...

[bladerunner1982]

@mbcd
Thanks for participation. That's a rather basic approach. But where's the upscaling part?

@Stereodude
I believe to think that I can see your point. I changed MI to 24 but used PP=6. Used override files for comet and wormhole.
After some experimenting I came up to this:

LoadPlugin("d:\gordianknot\DGMPGDec\DGDecode.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\LSMASHSource.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\Plugins\nnedi3.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\Plugins\TIVTC.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\Plugins\vinverse.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\Plugins\Shader.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\Plugins\AddGrainC.dll")

SetFilterMTMode("DEFAULT_MT_MODE", 2)

myvideo = "ep1ntsc.d2v"

DGDecode_mpeg2source(myvideo, idct=5)

## choose deinterlacer ##
#deint = QTGMC( Preset="Slower", InputType=0, SourceMatch=3, Lossless=2, Sharpness=0.2, EZKeepGrain=0.0, ShowSettings=false ).SelectEven()
deint = tdeint()

## IVTC hybrid ntsc with override filles ##
tfm(d2v=myvideo,  PP=6, MI=24, clip2=deint, flags=5,ovr="s06e01_tfm_ovr.txt")
TDecimate(hybrid = 1,ovr="s06e01_tdecimate_ovr.txt")

## clean up ##
#MCTemporalDenoise(settings="low", radius=3)
#f3kdb(range=16, grainY=15, grainC=10, sample_mode=2, dither_algo=3, dynamic_grain=true, keep_tv_range=false)
#or
#QTGMC(InputType=1)   #using QTGMC on progressive frames again to clean up a little bit
#or
QTGMC( Preset="Slower", InputType=1 ) 

## color tweak ##
tweak(cont=1.3,sat=1.1)

## Upscale ##
ConvertToRGB24()
SuperResXBR(Passes=5, Str=1, Soft=0.0, XbrStr=2.7, XbrSharp=1.3, MatrixIn="PC601")
SuperResXBR(Passes=3, Str=1, Soft=0.0, XbrStr=2.7, XbrSharp=1.3, MatrixIn="PC601", fWidth=1440, fHeight=1080, fKernel="SSim")
ConvertToYV12()

## grain only use if you didn't use f3kdb
addgrainc()


trim(0,15000)

## PAL Speedup for Europe
AssumeFPS(25)

Result is version 3 in this folder https://mega.nz/folder/A10lULqT

[Stereodude]

Originally Posted by bladerunner1982

@Stereodude
I believe to think that I can see your point. I changed MI to 24 but used PP=6. Used override files for comet and wormhole...

FWIW, I don't think there is any difference between 6 or 7 if you're using clip2.

I'm also on the fence about using mild deblocking in the MPEG2 source filter on the luma channel. For example, the comet in the show intro has significant blocking when decoded with no deblocking.