Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 2nd September 2017, 23:51   #1  |  Link
OvejaNegra
ekTOMBE STUDIOS
 
OvejaNegra's Avatar
 
Join Date: Dec 2005
Location: Cuba
Posts: 254
Is my source upscaled? (And a few questions about upscaling)

I'm doing an encode of my Ghost in the shell BD (the movie from 1995).

And i was little dissapointed with the detail, when i found this:

https://forum.doom9.org/showthread.php?t=170832


So i decided to try:

DebilinearM(1280,720,showmask=0,kernel="bicubic",cubic=true)

and to my surprise, the mask (supposedly) reveals that the source seems to be 1280x720. The only places where the mask reveals higher resolution detail is:

-On the credits
-Lights glows (seems to be done on post production, even histrogram() on avisynth shows that those pixels are outside the 16-235 luma range).

-Text on computer's screens (maybe donde with post production).

¿Should i conclude that the source master is 720p and it was upscaled on post production?

i did a simple test using
Code:
ups = last .Debicubic(1280, 720).BicubicResize(1920, 1080).Subtitle("ups", x=8, y=0)

Interleave(last,ups)
And i can't see any difference! Even the grain stays on the same place!

I espected GITS to be scanned from film to full 1080p. But maybe they used an old master.
Or maybe im talking nonsense!

I also did this:

Code:
Debicubic(1280, 720)
nnedi3_rpow2(rfactor=2, nsize=0, nns=3, qual=1, etype=0, pscrn=1, threads=0, opt=0, fapprox=15,cshift="spline36resize",fwidth=1920,fheight=1080)
And the lines look better on my eyes. Should i leave it like that?

Can anybody post an example of how use DebilinearM to extract / protect/mask the 1080 pixels and import them over the
nnedi final image?

Should i do all this on 16bits or not?

And since we are talking about upscaling: Some anime grops are releasing y444 1080p encodes. ¿are they filtering/upscaling the
chroma for avoid aliasing on colored edges? or are they using 4k sources?

thanks
__________________
So, it works or not???
OvejaNegra is offline   Reply With Quote
Old 3rd September 2017, 00:35   #2  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
First thing first.
We don't actually have a sample, but it could be that the master file was 720p and that it has been upscaled to 1920x1080 (in Studios it's generally either Bilinear or Bicubic).
If that's the case, I would personally reverse upscale in 16bit to 1280x720 and encode it to 10bit 4:4:4.
Now... let's talk about anime. Generally, anime are made in 1280x720 or slightly higher resolution like 810p and animation studios work in RGB 'till the very last moment, when they export to their target (Generally Apple ProRes 10bit 4:2:2 planar yv16). The reason why people do reverse upscale and encode Blu-ray (1080p 4:2:0 8bit) in 1280x720 10bit 4:4:4 is because they try to preserve chroma, which makes sense. But, if they use a 1080p 4:2:0 8bit source and encode in 1080p 4:4:4, then they are just upscaling chroma, which doesn't make sense.


In your case, this would be my script:

Code:
#your favourite index (LWLibav, FFMpeg, DGI)

Dither_convert_8_to_16() 
ly = debicubicy(1280,720,lsb_inout=true) 
lu = utoy().dither_resize16(1280,720,kernel="spline64",invks=true,invkstaps=3,src_left=0.25,u=1,v=1) 
lv = vtoy().dither_resize16(1280,720,kernel="spline64",invks=true,invkstaps=3,src_left=0.25,u=1,v=1) 
ytouv(lu,lv,ly)
Dither_quantize (10, reducerange=true, mode=6)
Dither_out()

Last edited by FranceBB; 3rd September 2017 at 00:51.
FranceBB is offline   Reply With Quote
Old 3rd September 2017, 03:48   #3  |  Link
kgrabs
Registered User
 
Join Date: Jan 2017
Posts: 28
Quote:
Originally Posted by FranceBB View Post
Code:
#your favourite index (LWLibav, FFMpeg, DGI)

Dither_convert_8_to_16() 
ly = debicubicy(1280,720,lsb_inout=true) 
lu = utoy().dither_resize16(1280,720,kernel="spline64",invks=true,invkstaps=3,src_left=0.25,u=1,v=1) 
lv = vtoy().dither_resize16(1280,720,kernel="spline64",invks=true,invkstaps=3,src_left=0.25,u=1,v=1) 
ytouv(lu,lv,ly)
Dither_quantize (10, reducerange=true, mode=6)
Dither_out()
Quick reminder that 16 bit debicubic is broken: https://github.com/Zeght/Morefun/blob/master/README.md
kgrabs is offline   Reply With Quote
Old 3rd September 2017, 04:09   #4  |  Link
OvejaNegra
ekTOMBE STUDIOS
 
OvejaNegra's Avatar
 
Join Date: Dec 2005
Location: Cuba
Posts: 254
Hi!
I assume that the original was a full 1280x720 444 master. When upscaled to 1080p the chroma was downs caled to 540 (using bicubic kernel). Why are you using "spline64",invks=true? Chroma was downscaled in this case.

I prefer encode to device, so that's why i prefer the output to 1080 (i don't trust on tv upscaler), so i'm using this:

Dither_convert_8_to_16()
debicubic(1280,720,lsb_inout=true)
nnedi3_resize16(1920,1080,lsb_in=true,lsb=true)
ditherpost()

(Converting to 720p 444 is a valid choice) But in the end TVs and player renderer will perform some upscaling (MadVR is not always a choice).

How can i mask / copy the 1080 post effects to my nnedi output (computers screen text seems to be 1080 added on post production). I know DebilinearM
is for that but i need an example.

Quote:
But, if they use a 1080p 4:2:0 8bit source and encode in 1080p 4:4:4, then they are just upscaling chroma, which doesn't make sense.
Yes, but some groups are doing that, so i was thinking that maybe there was a reason (a new "magical" chroma upscaler or something like that).

One more thing: I have Toradora BD and DVD (both from japan) the BD upscale was a little dissapointing to me. So i was thinking on trying DebilinearM - nnedi3,
or shoul i use nnedi3 on the DVD?

Advice?

Thanks!
__________________
So, it works or not???
OvejaNegra is offline   Reply With Quote
Old 3rd September 2017, 04:43   #5  |  Link
kgrabs
Registered User
 
Join Date: Jan 2017
Posts: 28
OK I'm back home now I can do a full reply.

First off, to mask in 1080p with DebicubicM/DebilinearM would require a pointless extra debixxx call to do right, so I'd just take the mask like so:

Code:
b = 1.0/3.0
c = 1.0/3.0

# 16 bit
s16 = to16()
i16 = ConvertBits(16)

debic = s16.DebicubicY16(1280, 720, b=b c=c, iter=true)
sharp = debic.nnedi3_resize16(last.Width, last.Height, lsb=true, lsb_in=true, u=1, v=1)

error = debic.Dither_resize16(last.Width, last.Height, a1=b, a2=c,kernel="bicubic", u=1, v=1).ConvertFromStacked(16)
mask =i16.mt_lutxy(error, "x y - abs").mt_binarize(10).mt_expand().mt_expand().mt_expand().mt_inflate().removegrain(20,-1)

sharp.ConvertFromStacked(16).mt_merge(i16, mask, u=4, v=4)

# 8 bit
debic = last.debicubicY(1280, 720, b=b c=c)
sharp = debic.converttoy8().nnedi3_rpow2(2, fwidth=last.Width, fheight=last.Height, cshift="spline36resize").converttoyv12()

error = debic.ResizeX(last.Width, last.Height, a1=b, a2=c,kernel="bicubic", chroma=false)
mask = last.mt_lutxy(error, "x y - abs").mt_binarize(10).mt_expand().mt_expand().mt_expand().mt_inflate().removegrain(20,-1)

sharp.mt_merge(last, mask, u=4, v=4)
Also, about inverse scaling in general, there's a lot of potential for error, so just checking it with the DebixxM mask isn't really enough to say "that's how it was upscaled." I would recommend trying this Vapoursynth script here to get a graph of the error at every resolution

From there, I usually use Resfinder or something similar to check different kernels and bicubic b/c vales to get the right scaler: Original Resfinder / Mod I made to support mod 1 and use high precision scaling (AVS+ required, hopefully not broken)
kgrabs is offline   Reply With Quote
Old 3rd September 2017, 09:24   #6  |  Link
OvejaNegra
ekTOMBE STUDIOS
 
OvejaNegra's Avatar
 
Join Date: Dec 2005
Location: Cuba
Posts: 254
mmm
after some quick trial and error:

ResFinder(810, Kernel=2, Mode=1, PARNum=1, PARDen=1)
Gives me the less error (white points if understood correctly)

DebilinearM(1440,810,showmask=1,kernel="bicubic",cubic=true)
is also happy.

So... 1440x810??? (i tried on regular scenes and on scenes with post producction glow effects).

If that's the case, i think downscale / upscale is just overkill bu i'm open to advices.

(GITS being digitalized on <1080 seems a little improbable)

thanks!
__________________
So, it works or not???
OvejaNegra is offline   Reply With Quote
Old 3rd September 2017, 10:43   #7  |  Link
kgrabs
Registered User
 
Join Date: Jan 2017
Posts: 28
I would make the decision more based on the error level than the resolution. Even above 900p using debi+nnedi3 should have a better sharpening effect than any sharpener and be smoother than with any sane AA filter

If you're seeing aliasing, ringing or halos in the "debic" clip then it's probably not a great idea. But I mean, there are plenty of ways to get around the drawbacks, depending on what they are.

Code:
mask = mt_lutxy(error, "x y - abs").mt_expand()

strong = mask.mt_expand().mt_expand().mt_expand().mt_binarize(10).mt_inflate().removegrain(20,-1) 
mask = mask.mt_lut(strong, " x 5 scalef - 40 * y max") # mask gradually from 5 to 10 instead of just binarizing at 10

sharp.mt_merge(last, mask, u=4, v=4)
Or just mask the lines if you're getting sharpened grain/blocking/ringing:

Code:
credits = mt_lutxy(error, "x y - abs").mt_binarize(10).mt_expand().mt_expand().mt_expand().mt_expand()
lines = mt_edge("prewitt", 4, 24)
mask = credits.mt_logic(lines, "andn").removegrain(20, -1)

mt_merge(sharp, mask)
Anyway, if ResFinder's error numbers are consistently over 0.02, and none of them are really low (usually a few frames have 6 or so zeros after the decimal) then it might be a bit dodgy.
kgrabs is offline   Reply With Quote
Old 3rd September 2017, 23:21   #8  |  Link
OvejaNegra
ekTOMBE STUDIOS
 
OvejaNegra's Avatar
 
Join Date: Dec 2005
Location: Cuba
Posts: 254
well, i have numbers as low as ~0.016500 but is not constant. there are many frames with ~0.026700 and more, i think it's time to move on.
Anyway, this is new for me and very interesting, i will try with Toradora BD.

What should be considered a safe value? <0.02?

Thanks for all the advices!
__________________
So, it works or not???
OvejaNegra is offline   Reply With Quote
Old 4th September 2017, 13:36   #9  |  Link
kgrabs
Registered User
 
Join Date: Jan 2017
Posts: 28
that seems like a safe enough range to me. its probably fine as long as its not over 0.03 but I mean there have been plenty of times I've gone a lot higher, honestly.
if it makes it look better and doesn't have any halos or anything, I don't really see any reason not to.

stuff like Toradora probably isn't the best idea. i don't think you get shimmering like that from a normal upscale.
kgrabs is offline   Reply With Quote
Old 5th September 2017, 00:27   #10  |  Link
OvejaNegra
ekTOMBE STUDIOS
 
OvejaNegra's Avatar
 
Join Date: Dec 2005
Location: Cuba
Posts: 254
yeah, do you know the native resolution of toradora?
Maybe i should try the DVDs instead of the BD (as long as the DVDs resolution is not very far from native).

They used this:

http://av.watch.impress.co.jp/docs/news/466237.html
__________________
So, it works or not???
OvejaNegra is offline   Reply With Quote
Old 5th September 2017, 09:02   #11  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
Don't forget there is a difference between TV serie, old TV serie, movie, old movie in production.
GITS is in "old" movie, so, we are not talking about digital production, but we are talking about cells and final result on a film. So talking about resolution production make no sense in this specific case, but talking about the resolution of the film transfert, make sense.
Meaning, anything (and even more movies) which master is a film, can be a realy 1080p if film transfer has been made "properly". It's also true for old TV series, for which also master is a film.
jpsdr is offline   Reply With Quote
Old 5th September 2017, 10:24   #12  |  Link
kgrabs
Registered User
 
Join Date: Jan 2017
Posts: 28
I have no idea what the effective resolution for Toradora would be, but it definitely looks like SD.

jpsdr is right about the production resolution for older anime, the only way descaling would really work well is if it was scanned, then downscaled and stored before being upscaled back later for the BD

...or some other equally dumb reason. don't put anything past them when it comes to screwing up our cartoons :/
kgrabs is offline   Reply With Quote
Old 6th September 2017, 18:57   #13  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
Nevertheless, even if for old things on film, we can't directly talk of resolution production, we can still talk of film format : 16mm, 24mm, 32mm, etc... It's inderectely somehow related to a resolution aspect, but don't aks me how...
jpsdr is offline   Reply With Quote
Old 6th September 2017, 23:26   #14  |  Link
OvejaNegra
ekTOMBE STUDIOS
 
OvejaNegra's Avatar
 
Join Date: Dec 2005
Location: Cuba
Posts: 254
Left: Untouched, Right: Debicubic and:
Code:
nnedi3_rpow2(rfactor=2, nsize=0, nns=3, qual=1, etype=0, pscrn=1, threads=0, opt=0, fapprox=15,cshift="spline36resize",fwidth=1920,fheight=1080)


Not a big deal in the end, but i espected more detail from the scaned film.
__________________
So, it works or not???

Last edited by OvejaNegra; 6th September 2017 at 23:30.
OvejaNegra is offline   Reply With Quote
Old 7th September 2017, 11:42   #15  |  Link
kgrabs
Registered User
 
Join Date: Jan 2017
Posts: 28
Oh, i remember now; for the original Ghost in the Shell, the TV airing didn't have any of the problems that the Blu-ray had.

https://diff.pics/mVI2lGLXTowi/1


feelsbad

Last edited by kgrabs; 8th September 2017 at 09:04. Reason: image 404
kgrabs is offline   Reply With Quote
Old 9th September 2017, 01:36   #16  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,657
Quote:
Originally Posted by kgrabs View Post
Also, about inverse scaling in general, there's a lot of potential for error, so just checking it with the DebixxM mask isn't really enough to say "that's how it was upscaled." I would recommend trying this Vapoursynth script here to get a graph of the error at every resolution
I downloaded the script and installed all the required file, how do I use it? There's no mention of input file.
lansing is offline   Reply With Quote
Old 9th September 2017, 05:41   #17  |  Link
OvejaNegra
ekTOMBE STUDIOS
 
OvejaNegra's Avatar
 
Join Date: Dec 2005
Location: Cuba
Posts: 254
kgrabs:

Mmm, maybe it's me, but the TV version has edge enhancement / halo

Look at her chin.
__________________
So, it works or not???
OvejaNegra is offline   Reply With Quote
Old 9th September 2017, 09:08   #18  |  Link
kgrabs
Registered User
 
Join Date: Jan 2017
Posts: 28
It's a WMV9 encode of an MPEG-2 TV broadcast, after all. A little edge repair might do it good but it's probably best it not get encoded any more

Quote:
Originally Posted by lansing View Post
I downloaded the script and installed all the required file, how do I use it? There's no mention of input file.
save this as a .bat
Code:
@echo off
echo Calculating the native resolution...
REM  -a 1.7777777778
"C:\Users\kgrabs\AppData\Local\Programs\Python\Python36\python.exe" "C:\Users\kgrabs\AppData\Local\Programs\Python\Python36\Lib\site-packages\getnative.py" -i "%~1" -k bicubic -b 0.0 -c 0.5 -f 1271 -out true -min 700 -max 1000
echo Done.
PAUSE
-f is the frame number to analyse, -min/-max are the first & last height to be calculated, -a is the aspect ratio (you have to crop window/pillar/letterboxed stuff first and examine a screenshot), -k -b and -c are you-know-what
kgrabs is offline   Reply With Quote
Old 9th September 2017, 15:32   #19  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,657
Quote:
Originally Posted by kgrabs View Post
save this as a .bat


-f is the frame number to analyse, -min/-max are the first & last height to be calculated, -a is the aspect ratio (you have to crop window/pillar/letterboxed stuff first and examine a screenshot), -k -b and -c are you-know-what
Thanks, got it working. I tested on my old anime blu-ray, the report said the best native resolution: 810p, 1085p, 1054p. And the graph



So 810p is its native resolution?
Also with the mask image, the 810p has more white dots than 1085p and 1054p, what does that mean?
lansing is offline   Reply With Quote
Old 10th September 2017, 04:31   #20  |  Link
kgrabs
Registered User
 
Join Date: Jan 2017
Posts: 28
Quote:
Originally Posted by lansing View Post
So 810p is its native resolution?
Also with the mask image, the 810p has more white dots than 1085p and 1054p, what does that mean?
At a glance I can tell you none of those resolutions are correct. The spikes should be very close to zero. I would probably make 3 batch files for convenience, one bilinear, and two bicubic; one with -b 0.0 -c 1.0, and one set to defaults.

Anyway I can't say much since I have really nothing to go on, but if it's old it's probably bilinear or some awful irreversible thing, and might have a 1.5 aspect ratio (720x480)

The white part of the mask is from the absolute value of the error of debicubic(w,h).bicubic(1920,1080) and the input
kgrabs is offline   Reply With Quote
Reply

Tags
anime, debicubic, debilinear, upscaling

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 23:02.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.