Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 5th April 2019, 21:37   #121  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,187
Now that's a novel idea. I was thinking of maybe merging the luma-converted U and V planes.
__________________
Nostalgia's not what it used to be

Last edited by WorBry; 5th April 2019 at 21:47.
WorBry is offline   Reply With Quote
Old 5th April 2019, 23:33   #122  |  Link
VS_Fan
Registered User
 
Join Date: Jan 2016
Posts: 89
With the same general idea, one could obtain an “an 'overall' score” by stacking all the planes:

Code:
ret = some_video_source
#This should work to preserve vertical resolution, and stacking all Y,U and V, no matter the css (chroma subsampling: 411, 420, 422, 444)
retc = core.std.StackHorizontal(clips=[retU,retV]) if ret.format.subsampling_h == 0 else core.std.StackVertical(clips=[retU,retV])
ret = core.std.StackHorizontal(clips=[retY,retc])
I would also expect this ‘overall’ result to be close to that of applying the weights as you exposed they do in ffmpeg: “each plane is scaled by the resolution it has”, and I guess they are something like:

Code:
sub-		Y 		U 		V
sampling	ratio		ratio		ratio

4:1:1		2/3		1/6		1/6 
4:2:0		2/3		1/6		1/6 
4:2:2		1/2		1/4		1/4 
4:4:4		1/3		1/3		1/3
VS_Fan is offline   Reply With Quote
Old 5th April 2019, 23:36   #123  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
@WorBry: I'm glad u finally woke up and entertain the idea of "weights" - a concept used in scientific formulas and equations since the dawn of time, and that includes the field of color science.

This is good, this could indicate progress !

Maybe next time before u have ur temper tantrum, u can put the bottle aside, and read posts properly. Zorr did directly mention weights to u in regards to ffmpeg's implementation but u seem to have chosen to ignore that... until u remembered.

well, to quote u: what gives ?


Quote:
Originally Posted by ChaosKing View Post
I don't know what a good weighted total score should look like.
yeah, hence me throwing it out there and asking for opinions... ;-)
Iron_Mike is offline   Reply With Quote
Old 6th April 2019, 00:02   #124  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
Quote:
Originally Posted by VS_Fan View Post
Code:
sub-		Y 		U 		V
sampling	ratio		ratio		ratio

4:1:1		2/3		1/6		1/6 
4:2:0		2/3		1/6		1/6 
4:2:2		1/2		1/4		1/4 
4:4:4		1/3		1/3		1/3
good stuff getting the ball rolling...

this was my pretty much my first approach.

so what about the idea - in case of src/ref and dist/enc having different css - of flipping that table and the more css the distorted clip has (compared to src/ref), to give chroma score more weight than luma ?

that would bring the total score down, but with a 420 dist clip, the chroma drift/delta will have a larger perceived impact than with a 444 dist clip...

for example:
source is 444 and the dist is 420... we could consider giving the chroma score more weight than luma, b/c that color drift will have a larger impact


for WorBry: no, just b/c u put the 420 dist in a 444 clip to run the metric does not make the 420 data inside that new clip a "444" encode... it's resampled 420 data
Iron_Mike is offline   Reply With Quote
Old 6th April 2019, 00:48   #125  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,793
I think the dist. should always be converted to the ref. pixel format

The metric should take into account the results of resizing the data - that's the whole idea of metrics measuring the effect of chroma subsampling . It should be a measureable phenomenon (and it is with standard metrics)

eg
a) 444 source reference
b) a is converted to "b" 4:2:0 by algorithm "z" (this means the U,V planes are resized by 1/2 width, 1/2 height)

You need to convert b back to 444 to compare against "a" . ie. resize back the U,V planes *2 width/height .

- you expect 422 to do better than 420 - and you do with ssim, psnr
- you expect to get slightly different results with different resize algorithms (e.g. sharper isn't always "better" ; sharper actually tends to get "penalized" by some metrics like ssim/psnr)
- serial conversions get blurrier colors, and worse results, as expected (because really you're up/down scaling the chroma planes)

Or if you're starting with a 420 reference ; and you're testing against something at got converted to 444 . That up/down conversion should be reflected in the scores as a larger deviation from the original reference.



Don't mix in "weights" ; it's a separate concept dealing with human perception of color vs. black/white . Your perception doesn't change , based on different pixel format of videos - it's the source that is changing . Weighting should be the same whatever you decide the formula is . You don't become less sensitive to color with 444 vs 420 . The effect of chroma subsampling should be measured and reflected by the metric

Last edited by poisondeathray; 6th April 2019 at 01:07.
poisondeathray is offline   Reply With Quote
Old 6th April 2019, 01:45   #126  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
Quote:
Originally Posted by poisondeathray View Post
I think the dist. should always be converted to the ref. pixel format

The metric should take into account the results of resizing the data - that's the whole idea of metrics measuring the effect of chroma subsampling . It should be a measureable phenomenon (and it is with standard metrics)

eg
a) 444 source reference
b) a is converted to "b" 4:2:0 by algorithm "z" (this means the U,V planes are resized by 1/2 width, 1/2 height)

You need to convert b back to 444 to compare against "a" . ie. resize back the U,V planes *2 width/height .

- you expect 422 to do better than 420 - and you do with ssim, psnr
- you expect to get slightly different results with different resize algorithms (e.g. sharper isn't always "better" ; sharper actually tends to get "penalized" by some metrics like ssim/psnr)
- serial conversions get blurrier colors, and worse results, as expected (because really you're up/down scaling the chroma planes)

Or if you're starting with a 420 reference ; and you're testing against something at got converted to 444 . That up/down conversion should be reflected in the scores as a larger deviation from the original reference.

Don't mix in "weights" ; it's a separate concept dealing with human perception of color vs. black/white . Your perception doesn't change , based on different pixel format of videos - it's the source that is changing . Weighting should be the same whatever you decide the formula is . You don't become less sensitive to color with 444 vs 420 . The effect of chroma subsampling should be measured and reflected by the metric
@pd: all agreed, but we were elaborating combining scores of various planes into a combined metric score for all planes... and in a 420 dist (from a 444 ref), the chroma gets a different treatment than the luma in the encoding process...

so when u combine metric score for luma and chroma planes, one could consider that... or are u saying the chroma metric scores already state that and u would just pull the avg ?

Btw, weights weren't mixed in (in my posts) - weights are constants - luma vs. chroma perception is one, but u can have additional weights for other effects... for example, the luma vs. chroma weight also only kicks in if you're actually dealing w/ color footage (so it is a conditional factor)...
Iron_Mike is offline   Reply With Quote
Old 6th April 2019, 01:46   #127  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,187
Quote:
Originally Posted by Iron_Mike View Post
@WorBry: I'm glad u finally woke up and entertain the idea of "weights" - a concept used in scientific formulas and equations since the dawn of time, and that includes the field of color science.

This is good, this could indicate progress !

Maybe next time before u have ur temper tantrum, u can put the bottle aside, and read posts properly. Zorr did directly mention weights to u in regards to ffmpeg's implementation but u seem to have chosen to ignore that... until u remembered.

well, to quote u: what gives ?

yeah, hence me throwing it out there and asking for opinions... ;-)
I have never not had an 'idea of weights'. That's what my last series of tests in the 'SSIM and GMSD metrics' thread was about - to see if any weighting or resizing is being applied to the luma-converted U and V planes in the muvsfunc SSIM and GMSD implementations:

https://forum.doom9.org/showthread.p...63#post1870863

And your original question about 'weights' here was:

Quote:
Originally Posted by Iron_Mike View Post
...so what are opinions about how to weigh the GMSD/SSIM results if the css of the dist differs ?
Which both I and ChaosKing answered:

Quote:
Originally Posted by WorBry View Post
You don't. You either convert the css of the 'distorted' clip to that of the reference or vice versa. They must be the same format - period. It's the fundamental requirement of Full Reference Image Quality Assessment. You seem to be laboring under this notion that it should somehow be different. 'Full Reference' - think about it.
(You still seem to have that notion)

Quote:
Originally Posted by ChaosKing View Post
But to compare different css you have to resize the chroma plane to match the same clip format for ssim/gmsd. It's similar to a rgb and yuv clip, one has to be converted wich alters your final score a little bit.
Now enough of your insults and overt gas-lighting.

@VS_Fan and Poisondeathray. Thanks. Alot to think about there.
__________________
Nostalgia's not what it used to be

Last edited by WorBry; 6th April 2019 at 02:12.
WorBry is offline   Reply With Quote
Old 6th April 2019, 02:03   #128  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,793
Quote:
Originally Posted by Iron_Mike View Post
@pd: all agreed, but we were elaborating combining scores of various planes into a combined metric score for all planes... and in a 420 dist (from a 444 ref), the chroma gets a different treatment than the luma in the encoding process...

so when u combine metric score for luma and chroma planes, one could consider that... or are u saying the chroma metric scores already state that and u would just pull the avg ?

Btw, weights weren't mixed in (in my posts) - weights are constants - luma vs. chroma perception is one, but u can have additional weights for other effects... for example, the luma vs. chroma weight also only kicks in if you're actually dealing w/ color footage (so it is a conditional factor)...



The idea of "weights" for a combined Y/U/V aggregate metric score, should reflect that the Y plane should recieve a proportionally higher weighting due to human perceptions - I think everyone will agree on the general idea, but might disagree on the actual formula for the weighting

All I'm saying is the relative weighting shouldn't change because of subsampling . Your perception of the proportion of black/white vs. color importance doesn't suddenly change if you watch a 4:4:4 video or a 4:2:0 video.

That lower quality of 4:2:0 should already be reflected in the lower U, V scores for 4:2:0 . That up/down converison is the "penalty" already incurred . The "different treatment" of chroma is exactly what you're measuring in the first place when you measure U-SSIM, V-SSIM, or U-PSNR, V-PSNR or whatever metric

Yes you can have other weights, other categories, combine /mix/match in any way you want , analyze it in whatever way you want, call it whatever you want ; but eitherway you're measureing Y,U,V separately - so you should have the "raw" scores
poisondeathray is offline   Reply With Quote
Old 6th April 2019, 12:17   #129  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
Quote:
Originally Posted by poisondeathray View Post
The idea of "weights" for a combined Y/U/V aggregate metric score, should reflect that the Y plane should recieve a proportionally higher weighting due to human perceptions - I think everyone will agree on the general idea, but might disagree on the actual formula for the weighting

All I'm saying is the relative weighting shouldn't change because of subsampling . Your perception of the proportion of black/white vs. color importance doesn't suddenly change if you watch a 4:4:4 video or a 4:2:0 video.

That lower quality of 4:2:0 should already be reflected in the lower U, V scores for 4:2:0 . That up/down converison is the "penalty" already incurred . The "different treatment" of chroma is exactly what you're measuring in the first place when you measure U-SSIM, V-SSIM, or U-PSNR, V-PSNR or whatever metric

Yes you can have other weights, other categories, combine /mix/match in any way you want , analyze it in whatever way you want, call it whatever you want ; but eitherway you're measureing Y,U,V separately - so you should have the "raw" scores
@pd: yup, all agreed.

now stop dancing around the subject, what would be your weights/formula for a combined metric score of all planes ?
Iron_Mike is offline   Reply With Quote
Old 6th April 2019, 16:19   #130  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,793
I don't think there is a commonly accepted formula/value for all planes. There are no scientific studies that conclusively measure this value to be "x/y/z"

Just pick one and as long as you state your assumptions, criteria, and reasoning- it's ok . e.g if you decide 0.8/0.1/0.1 , say so

What's not ok - is if you post results, but no details - ie. lack of transparency .

e.g. if you measure a black/white movie - you might still decide to look at U/V . There are old movies that have discoloration and chroma noise / rainbows due to the transfer process for example .

Everyone knows (or should know) , that all metrics have various pros/cons and are context dependent. But the trends can still be useful to look at
poisondeathray is offline   Reply With Quote
Old 7th April 2019, 19:54   #131  |  Link
Wilbert
Moderator
 
Join Date: Nov 2001
Location: Netherlands
Posts: 6,331
@Iron_Mike, refrain your language. There is no need to be disrespectful to others.
Wilbert is offline   Reply With Quote
Old 21st April 2019, 17:46   #132  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,525
I have a small feature request regarding cases where colorspace conversion to RGB is required. It would be nice if Zopti as an option could do the conversion just once and save the resulting file as an intermediate file to use in all the subsequent iterations. I don't know how much time consuming it actually is, but I'd expect that the number of iterations is usually so big that over a long test run, it would same a nice amount of time.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 22nd April 2019, 22:07   #133  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 213
Quote:
Originally Posted by Boulder View Post
I have a small feature request regarding cases where colorspace conversion to RGB is required. It would be nice if Zopti as an option could do the conversion just once and save the resulting file as an intermediate file to use in all the subsequent iterations.
That's a bit tricky, currently Zopti doesn't know what the input file is so it's unable to change it. I think it could save the converted file using clip.output() (someone can correct me here if I'm wrong) and then read that one instead, but I'm not sure if it's any faster if the file is in uncompressed format which the output() supports. Does someone have experience on using output() and reading the resulting file?
zorr is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 12:45.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.