Deep Space Nine upscale project - Page 23

Katie Boundary · 25th May 2020, 19:12

Quote:

Originally Posted by JoelHruska

I prefer the 640x480 crop to the 704, less for the CGI and more because it tends to make faces look a little stretched.

Well, that's because the faces ARE stretched, and so is everything else. These images are meant to be displayed on a device with non-square pixels. You need to resize to 640x480 (or some other resolution with a 4:3 aspect ratio) to get everything back to the right shape.

hello_hello · 25th May 2020, 19:27

Pardon my stupidness, but what's the difference between CG and HQ?

I'm only looking at them on a 1808p display, but I ran the upscaled versions at full size (I only see 1080p worth of picture) then upscaled the DVD encode by roughly the same amount with MPC-HC. The CG versions are an improvement for the CGI. It looks better than I expected, because I was half expecting the upscaling to do things to it I'd hate. I wouldn't mind seeing a live action comparison. The HQ versions are over-sharpened for my taste.

1080p screesnshots.zip

hello_hello · 25th May 2020, 19:33

Quote:

Originally Posted by JoelHruska

Hello_Hello,

I prefer the 640x480 crop to the 704, less for the CGI and more because it tends to make faces look a little stretched. But they're both absolutely gorgeous.

Also, the lighter QTGMC approach still cleans up the aliasing on the lower hull very nicely.

Technically you're better off cropping the DVD to 704x480 (which is an exact 4:3 display aspect ratio) and upscaling directly to 4:3 dimensions, as that way you might squeeze a tiny bit more detail out of it than you get resizing to 640x480 first. Can you specify the upscaled width and height individually?

JoelHruska · 25th May 2020, 19:52

Hello_Hello,

Those are the names of the new models they use. I've done a fair bit of comparison between them and I've run full encodes of both on episodes.

HQ typically retains more noise than CG if used on scenes with people in them. CG has stronger default noise filtering and deblocking applied. Gaia-CG is typically criticized for being overly aggressive on noise reduction outside of CG scenes, and for applying a bit too much smoothing. Gaia-HQ is typically criticized for retaining too much noise.

I generally prefer Gaia-CG over HQ for even the character scenes, but the models are still under active training and there's going to be a big update dropping in the next few weeks. Neither is perfect.

zapp7 · 25th May 2020, 20:26

I tried Gaia-CG on one of the first season episodes and it produced artifacts, kind of like lines or streaks. Is there something I'm missing?

JoelHruska · 25th May 2020, 21:52

Quote:

I tried Gaia-CG on one of the first season episodes and it produced artifacts, kind of like lines or streaks. Is there something I'm missing?

Which episode and where? I'll see if I can duplicate.

zapp7 · 25th May 2020, 21:54

Quote:

Originally Posted by JoelHruska

Which episode and where? I'll see if I can duplicate.

S01E04 - A Man Alone. You can see them in the big CGI ball in the first scene of the episode.

JoelHruska · 25th May 2020, 22:22

Quote:

S01E04 - A Man Alone. You can see them in the big CGI ball in the first scene of the episode.

I'll rip it and take a look.

SaurusX · 25th May 2020, 22:38

Gaia-CG is ugly as hell. Does way to much scrubbing away of detail.

JoelHruska · 25th May 2020, 23:05

Quote:

Gaia-CG is ugly as hell. Does way to much scrubbing away of detail.

The impact of Gaia-CG on CG scenes in DS9 is minimal compared to the uplift. It's more noticeable on non-CG content where, yes, there is some visible smoothing out of detail. I ran the denoise / deblock 100% render option on Gaia-CG multiple times on the same episode, just to see what would happen. In the "Sacrifice of Angels" recap of the previous episode, Sisko starts the clip wearing a quilted uniform top that has vertical lines on it -- seams, stitched into the fabric. You can see them clearly.

After a 2x runthrough Gaia-CG, they are noticeably less visible when he changes the angle of his stance. At three passes, they disappear altogether when he's standing at certain angles. Hopefully all of this will be tuneable one day.

zapp7 · 26th May 2020, 15:41

So my previous comments about noise filters had one caveat: I was using Artemis HQ in Topaz, which I have discovered already applies its own denoising. So in actuality, I was denoising twice and that's probably why the image looked too smooth for my tastes.

I've produced some sample clips using Gaia-HQ and Gaia-CG in an effort to see how the noise filters look with those. The filters I tested are MCDegrainSharp, TemporalDegrain2 and QTGMC. I just used the default settings for all of those, so I'm open to suggestions for tweaking. My QTGMC call is QTGMC(InputType=1). These filters are applied before upscaling with Topaz. (I wonder, would there be any theoretical advantage to applying them after upscaling?)

I'm trying to find a filter that I could apply to everything, since this project is big enough without having to edit noise filters scene-by-scene.

I only ran MCDegrainSharp with Gaia-HQ because it's obvious that the result isn't good - it seems to leave a pattern on the character scenes that doesn't look good. I'm also not too fond of the QTGMC result - perhaps someone can suggest better settings as I'm not very familiar with them. I kind of like the TemporalDegrain2 result, but I fear my eye for detail is lacking as the file size for this one is about half what it is without denoising.

TFM and TDecimate were applied before applying the noise filters. Input and output from Topaz was in .png format. These clips were created with ffmpeg libx265, using the slow preset and RF of 20.

The clips can be found here.

Forteen88 · 26th May 2020, 16:20

Quote:

Originally Posted by zapp7

...The filters I tested are MCDegrainSharp, TemporalDegrain2 and QTGMC. I just used the default settings for all of those, so I'm open to suggestions for tweaking.

When I use TemporalDegrain2, I prefer the setting: TemporalDegrain2(grainLevel=false)
Because that setting don't denoise as much (I'd like to know how to make TemporalDegrain2 denoise even less sometimes, but I don't know what setting to use for that).

Boulder · 26th May 2020, 16:22

A suggestion for x265: set the parameter no-sao. SAO will smooth the image even more, sucking out details on its way.

zapp7 · 26th May 2020, 17:58

Thanks, I'll implement those suggestions and generate some new clips tonight.

Another question, after the denoising I use gradfun3() for debanding since I saw it used elsewhere on this forum. Is this something that should always be used after denoising?

Boulder · 26th May 2020, 19:01

Quote:

Originally Posted by zapp7

Another question, after the denoising I use gradfun3() for debanding since I saw it used elsewhere on this forum. Is this something that should always be used after denoising?

I use debanding (neo_f3kdb) myself in all my encodes as the last item. Using the preset "low", I haven't noticed any ill effects aside from a slight increase in needed bitrate but it should keep those ugly artifacts away.

zapp7 · 27th May 2020, 00:45

I did a quality comparison and I'm pretty sure that more detail is retained by applying TemporalDegrain2 after upscaling. This makes sense since more detail is available for the input to Topaz.

I made a new clip with TemporalDegrain2(grainLevel=false) applied after upscale and encoded with no-sao. The result is in "S06E06_GaiaHQ_post_TempDeg2.grainLevel0_nosao.mkv" in this folder.

I think this result is pretty good. If I zoom in and compare specific frames with the GaiaHQ_NoDenoising clip, I can't see any significant difference in detail but the noise seems significantly reduced. Could use a second opinion to verify my sanity...

JoelHruska · 27th May 2020, 02:30

Quote:

Originally Posted by zapp7

I did a quality comparison and I'm pretty sure that more detail is retained by applying TemporalDegrain2 after upscaling. This makes sense since more detail is available for the input to Topaz.

I made a new clip with TemporalDegrain2(grainLevel=false) applied after upscale and encoded with no-sao. The result is in "S06E06_GaiaHQ_post_TempDeg2.grainLevel0_nosao.mkv" in this folder.

I think this result is pretty good. If I zoom in and compare specific frames with the GaiaHQ_NoDenoising clip, I can't see any significant difference in detail but the noise seems significantly reduced. Could use a second opinion to verify my sanity...

It looks to me like you conserve more detail with the -no sao and applying degrain after upscale.

https://i.imgur.com/YXqP44c.png

What does Gaia-CG look like with these settings?

zapp7 · 27th May 2020, 02:48

Quote:

Originally Posted by JoelHruska

It looks to me like you conserve more detail with the -no sao and applying degrain after upscale.

https://i.imgur.com/YXqP44c.png

What does Gaia-CG look like with these settings?

I put the GaiaCG version in the same folder. I believe GaiaCG already has denoising so I'm also making a GaiaCG_NoDenoising_nosao that should be there shortly.

JoelHruska · 27th May 2020, 03:27

Artemis_HQ and LQ models are both still incapable of upscaling DS9 properly, even when fed the samples H_H decoded. They fail in the same way -- mouths and faces cause meltdowns.

zapp7 · 27th May 2020, 04:50

Quote:

Originally Posted by JoelHruska

Artemis_HQ and LQ models are both still incapable of upscaling DS9 properly, even when fed the samples H_H decoded. They fail in the same way -- mouths and faces cause meltdowns.

I agree. What did you think of the MQ model in the previous versions? I read some people liked that one but I don't have it on hand.

25th May 2020, 19:52	#444 \| Link
JoelHruska Registered User Join Date: May 2020 Posts: 77	Hello_Hello, Those are the names of the new models they use. I've done a fair bit of comparison between them and I've run full encodes of both on episodes. HQ typically retains more noise than CG if used on scenes with people in them. CG has stronger default noise filtering and deblocking applied. Gaia-CG is typically criticized for being overly aggressive on noise reduction outside of CG scenes, and for applying a bit too much smoothing. Gaia-HQ is typically criticized for retaining too much noise. I generally prefer Gaia-CG over HQ for even the character scenes, but the models are still under active training and there's going to be a big update dropping in the next few weeks. Neither is perfect. Last edited by JoelHruska; 25th May 2020 at 21:53.

26th May 2020, 16:22	#453 \| Link
Boulder Pig on the wing Join Date: Mar 2002 Location: Finland Posts: 5,733	A suggestion for x265: set the parameter no-sao. SAO will smooth the image even more, sucking out details on its way. __________________ And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... Last edited by Boulder; 27th May 2020 at 05:21.

25th May 2020, 19:27	#442 \| Link
hello_hello Registered User Join Date: Mar 2011 Posts: 4,829	Pardon my stupidness, but what's the difference between CG and HQ? I'm only looking at them on a 1808p display, but I ran the upscaled versions at full size (I only see 1080p worth of picture) then upscaled the DVD encode by roughly the same amount with MPC-HC. The CG versions are an improvement for the CGI. It looks better than I expected, because I was half expecting the upscaling to do things to it I'd hate. I wouldn't mind seeing a live action comparison. The HQ versions are over-sharpened for my taste. 1080p screesnshots.zip

25th May 2020, 20:26	#445 \| Link
zapp7 Registered User Join Date: May 2020 Location: Canada Posts: 49	I tried Gaia-CG on one of the first season episodes and it produced artifacts, kind of like lines or streaks. Is there something I'm missing?

25th May 2020, 22:38	#449 \| Link
SaurusX Registered User Join Date: Feb 2017 Posts: 135	Gaia-CG is ugly as hell. Does way to much scrubbing away of detail.

26th May 2020, 15:41	#451 \| Link
zapp7 Registered User Join Date: May 2020 Location: Canada Posts: 49	So my previous comments about noise filters had one caveat: I was using Artemis HQ in Topaz, which I have discovered already applies its own denoising. So in actuality, I was denoising twice and that's probably why the image looked too smooth for my tastes. I've produced some sample clips using Gaia-HQ and Gaia-CG in an effort to see how the noise filters look with those. The filters I tested are MCDegrainSharp, TemporalDegrain2 and QTGMC. I just used the default settings for all of those, so I'm open to suggestions for tweaking. My QTGMC call is QTGMC(InputType=1). These filters are applied before upscaling with Topaz. (I wonder, would there be any theoretical advantage to applying them after upscaling?) I'm trying to find a filter that I could apply to everything, since this project is big enough without having to edit noise filters scene-by-scene. I only ran MCDegrainSharp with Gaia-HQ because it's obvious that the result isn't good - it seems to leave a pattern on the character scenes that doesn't look good. I'm also not too fond of the QTGMC result - perhaps someone can suggest better settings as I'm not very familiar with them. I kind of like the TemporalDegrain2 result, but I fear my eye for detail is lacking as the file size for this one is about half what it is without denoising. TFM and TDecimate were applied before applying the noise filters. Input and output from Topaz was in .png format. These clips were created with ffmpeg libx265, using the slow preset and RF of 20. The clips can be found here.

26th May 2020, 17:58	#454 \| Link
zapp7 Registered User Join Date: May 2020 Location: Canada Posts: 49	Thanks, I'll implement those suggestions and generate some new clips tonight. Another question, after the denoising I use gradfun3() for debanding since I saw it used elsewhere on this forum. Is this something that should always be used after denoising?

27th May 2020, 00:45	#456 \| Link
zapp7 Registered User Join Date: May 2020 Location: Canada Posts: 49	I did a quality comparison and I'm pretty sure that more detail is retained by applying TemporalDegrain2 after upscaling. This makes sense since more detail is available for the input to Topaz. I made a new clip with TemporalDegrain2(grainLevel=false) applied after upscale and encoded with no-sao. The result is in "S06E06_GaiaHQ_post_TempDeg2.grainLevel0_nosao.mkv" in this folder. I think this result is pretty good. If I zoom in and compare specific frames with the GaiaHQ_NoDenoising clip, I can't see any significant difference in detail but the noise seems significantly reduced. Could use a second opinion to verify my sanity...

27th May 2020, 03:27	#459 \| Link
JoelHruska Registered User Join Date: May 2020 Posts: 77	Artemis_HQ and LQ models are both still incapable of upscaling DS9 properly, even when fed the samples H_H decoded. They fail in the same way -- mouths and faces cause meltdowns.