View Single Post
Old 21st July 2020, 15:56   #16  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 1,070
"Can it also be used as a brick-wall low-pass filter for antialiasing only, possibly with a cutoff that the user can choose? I'm just asking 'cause now I'm curious."

If you attempt to do 'anti-alias' work with 'ideal brick wall low-pass' filter you will typically got ringing caused by 'Gibbs-phenomenon' ( https://en.wikipedia.org/wiki/Gibbs_phenomenon ) . So as my practice shows good 'conditioning' filter must AND cut-off illegal frequencies outside Nyquist limit AND control shape of remaining spectrum components inside legal Nyquist frequencies (from zero to Fd/2).

To do filtering with ideal brick-wall low-pass filter I think you can use already existed in avisynth SincResize filter.

'but isn't that gonna shoot to the roof (i.e give out of range values) on signals that peak at 770mV (0.7V), namely 235 8bit?'

Legal values for 8bit video data encoding samples values (for limited transitions) can reach limits from 1 to 254 (0 and 255 are system sync code values). So smart digital encoding of video data in Rec.601 (709, etc) already have some headroom for proper handling (limited) over and under shoots even below system-black 16 level and system-nominal-white 235 level. So it can encode and transfer more sharper image data in compare with 0(system black)..255(system while and max white) encoding with <0 and >255 non-legal and non-possible data values. More (visually) sharper with still good conditioning against ringing.

Example: if we try to encode sharp transition from system-black 16 level to system-nom-white 235 level we can either stuck with 16..235 range (for example using non-ringing/non-over/under-shooting Gauss kernel filter) and got something like:
16,16,70,191,235,235,235,... samples sequence.
But it will give less sharp transition - more time (or actually restored image space dimention) from 0.1 to 0.9 level rise.
Like 1.6T from 0.1 to 0.9 system nominal amplitude.


in compare with over/under shooting like: 16,16,1,92,254,254,235,235,... samples sequence. This samples sequence has 'out-of range' 16..235 encoding samples, but only shortly around transition.
It allows to get about 1.05T rise transition.


So it is legal to encode over/undershoots with 1..15 and 235..254 code values, and also 235..254 code levels may be used to encode super-white 'knee-d' values (with losing more and more ability to encode properly over-shooting for small details and transitions on large detail borders).
More exact you can test in in web-simulator at the link in the first post.

Last edited by DTL; 22nd July 2020 at 09:53. Reason: added images
DTL is offline   Reply With Quote