Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 14th March 2019, 21:15   #101  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 845
done
ChaosKing is offline   Reply With Quote
Old 19th March 2019, 09:51   #102  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 845
https://github.com/theChaosCoder/zoptilib
Mdsi and butteraugli no longer contaminate the clip. That means metrics=['ssim', 'gmsd', 'mdsi', 'butteraugli'] is the same as metrics=['mdsi', 'butteraugli', 'ssim', 'gmsd'] now.
This also means that you won't see the heatmap...
ChaosKing is offline   Reply With Quote
Old 19th March 2019, 10:03   #103  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
Quote:
Originally Posted by ChaosKing View Post
https://github.com/theChaosCoder/zoptilib
Mdsi and butteraugli no longer contaminate the clip. That means metrics=['ssim', 'gmsd', 'mdsi', 'butteraugli'] is the same as metrics=['mdsi', 'butteraugli', 'ssim', 'gmsd'] now.
This also means that you won't see the heatmap...
could you elaborate on that ?

is this in regards to the RGB conversion for MDSI/Butteraugly ?

Also, running Butteraugly via Zopti takes quite a while on my system... way longer than VMAF, GMSD, SSIm etc... is this normal ?

Any settings to speed this up ?

Thanks.
Iron_Mike is offline   Reply With Quote
Old 19th March 2019, 10:17   #104  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 845
Yes, it doesn't pass the rgb converted clip but instead it copies the clip props to the old clip now.
Butteraugli is sloooow, can't do much about it.
ChaosKing is offline   Reply With Quote
Old 19th March 2019, 10:25   #105  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
Quote:
Originally Posted by ChaosKing View Post
Yes, it doesn't pass the rgb converted clip but instead it copies the clip props to the old clip now.
Butteraugli is sloooow, can't do much about it.
okay, so if I run metrics=['ssim', 'gmsd', 'mdsi', 'butteraugli'] in the same Zopti call, it will convert the clips to RGB for MDSI/Butter but will do GMSD/SSIM calc in YUV ?
Iron_Mike is offline   Reply With Quote
Old 19th March 2019, 10:31   #106  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 845
Quote:
Originally Posted by Iron_Mike View Post
okay, so if I run metrics=['ssim', 'gmsd', 'mdsi', 'butteraugli'] in the same Zopti call, it will convert the clips to RGB for MDSI/Butter but will do GMSD/SSIM calc in YUV ?
Yes and yes and the "auto-rgb-convert" is still present for mdsi and butter.

Last edited by ChaosKing; 19th March 2019 at 10:34.
ChaosKing is offline   Reply With Quote
Old 4th April 2019, 23:52   #107  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
does SSIM/GMSD via Zopti/muvsfunc upscale the chroma planes to the src resolution of the clip if we specifiy planes=1|2 (e..g is src is 444 but dist is 420) ?
Iron_Mike is offline   Reply With Quote
Old 5th April 2019, 00:05   #108  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,187
No it does not. That was the whole point of my last post in the 'SSIM/GMSD metrics thread':

https://forum.doom9.org/showthread.p...63#post1870863


You posted the same question there and before I could answer, deleted it and are now asking it here. What gives ?

No resize or weights are applied to the 'processed-as-luma' chroma planes.

Quote:
Originally Posted by Iron_Mike View Post
e..g is src is 444 but dist is 420) ?
That combination would throw an error irrespective of the plane being processed.
Code:
 "clip1" and "clip2" must be of the same format!
__________________
Nostalgia's not what it used to be

Last edited by WorBry; 5th April 2019 at 00:24.
WorBry is offline   Reply With Quote
Old 5th April 2019, 00:16   #109  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
Quote:
Originally Posted by WorBry View Post
No it does not. That was the whole point of my last post in the 'SSIM/GMSD metrics thread':

https://forum.doom9.org/showthread.p...63#post1870863

You posted the same question there and before I could answer, deleted it and are now asking it here. What gives ?
what gives ?

well, Brian... take a really hard guess why I opted to move this to the ZOPTI thread instead of leaving it in the GMSD thread.... ? considering we're using Zopti as a wrapper...


in any case - back to the point - so in case of GMSD/SSIM scoring planes 1/2 of a 444 src vs 420 dist, is Zopti/muvsfunc downscaling the src to the dist resolution or does the GMSD/SSIM metric not require both clips to be in the same resolution ?
Iron_Mike is offline   Reply With Quote
Old 5th April 2019, 00:27   #110  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,187
I repeat...

Quote:
Originally Posted by WorBry View Post
That combination would throw an error irrespective of the plane being processed.
Code:
 "clip1" and "clip2" must be of the same format!
__________________
Nostalgia's not what it used to be
WorBry is offline   Reply With Quote
Old 5th April 2019, 00:33   #111  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
Quote:
Originally Posted by WorBry View Post
I repeat...
yeah, that's why I ask here in the Zopti thread...

I just ran a 444 ref against a 420 dist, scored GMSD/SSIM in planes 1|2 and it did not throw an error (meaning: chroma resolution of src/dist was NOT the same)... also (as far as I understand it), vs.format does not include src/Y resolution...

so, question remains...
Iron_Mike is offline   Reply With Quote
Old 5th April 2019, 00:46   #112  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 845
Quote:
Originally Posted by Iron_Mike View Post
yeah, that's why I ask here in the Zopti thread...

I just ran a 444 ref against a 420 dist, scored GMSD/SSIM in planes 1|2 and it did not throw an error (meaning: chroma resolution of src/dist was NOT the same)... also (as far as I understand it), vs.format does not include src/Y resolution...

so, question remains...
Could you provide your script? I always get "clip1" and "clip2" must be of the same format!

My test
Code:
#c is 420
a = mvf.GetPlane(c, 1)
c = core.fmtc.resample(c, css="444") 
b = mvf.GetPlane(c, 1)#.fmtc.resample(css="444") 

c = muf.SSIM(a,b)
ChaosKing is offline   Reply With Quote
Old 5th April 2019, 00:49   #113  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,187
Quote:
Originally Posted by ChaosKing View Post
Could you provide your script? I always get "clip1" and "clip2" must be of the same format!
Quite !
__________________
Nostalgia's not what it used to be
WorBry is offline   Reply With Quote
Old 5th April 2019, 02:21   #114  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
Quote:
Originally Posted by ChaosKing View Post
Could you provide your script? I always get "clip1" and "clip2" must be of the same format!
my bad, u're right - I have a block of code there for SSIM/GMSD (via Zopti) that auto-converts dist to src format in case format differs - to avoid that exact error... ;-)

so what are opinions about how to weigh the GMSD/SSIM results if the css of the dist differs ?
Iron_Mike is offline   Reply With Quote
Old 5th April 2019, 03:56   #115  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,187
Quote:
Originally Posted by Iron_Mike View Post
so what are opinions about how to weigh the GMSD/SSIM results if the css of the dist differs ?
You don't. You either convert the css of the 'distorted' clip to that of the reference or vice versa. They must be the same format - period. It's the fundamental requirement of Full Reference Image Quality Assessment. You seem to be laboring under this notion that it should somehow be different. 'Full Reference' - think about it.
__________________
Nostalgia's not what it used to be

Last edited by WorBry; 5th April 2019 at 04:26.
WorBry is offline   Reply With Quote
Old 5th April 2019, 04:27   #116  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
Quote:
Originally Posted by WorBry View Post
You don't. You either convert the css of the 'distorted' clip to that of the reference or vice versa. They must have the same format - period. It's the fundamental requirement of Full Reference Image Quality Assessment. You seem to be laboring under this notion that it should somehow be different.
well Brian, a "fundamental requirement" is to read posts properly... or to put 1+1 together

I was referring to weighing the chroma plane results (if dist had different css than ref) when combining them with the Y plane result for a total GMSD/SSIM score based on all three planes...............

in addition... chroma plane results can also be (optionally) weighed (in a combined score) even if both ref/dist have the exact same css considering the fact that color drift has less perceived impact in human vision...


whatever it is that u're mumbling in regards to "format" has nothing to do with any of it... just because u're matching a format so u can run the metric does not mean the data is (or should be considered) equal........

Last edited by Iron_Mike; 5th April 2019 at 04:33.
Iron_Mike is offline   Reply With Quote
Old 5th April 2019, 04:35   #117  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,187
Someone else deal with this please. I'm done with this nonsense.

Also Iron_Mike, please desist in referring to me by anything other than my forum user name. I don't know what you are trying to convey, but it's rather creepy and smacks of trollism.
__________________
Nostalgia's not what it used to be

Last edited by WorBry; 5th April 2019 at 05:43.
WorBry is offline   Reply With Quote
Old 5th April 2019, 12:27   #118  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 845
Quote:
Originally Posted by Iron_Mike View Post
well Brian, a "fundamental requirement" is to read posts properly... or to put 1+1 together

I was referring to weighing the chroma plane results (if dist had different css than ref) when combining them with the Y plane result for a total GMSD/SSIM score based on all three planes...............
But to compare different css you have to resize the chroma plane to match the same clip format for ssim/gmsd. It's similar to a rgb and yuv clip, one has to be converted wich alters your final score a little bit.

hmm a score based on all three planes... For that I tried to see how the video looks like if you extremely blur the chroma plane. Surprisingly only the colors are a bit off.
I guess the final score could be as simple as Plane1 80% + P2 10% + P3 10% ... I don't know what a good weighted total score should look like.

Code:
c2_1 = mvf.GetPlane(c2, 1).std.BoxBlur(hradius=5, vradius=5)
c2_2 = mvf.GetPlane(c2, 2).std.BoxBlur(hradius=5, vradius=5)
c2 = core.std.ShufflePlanes(clips=[mvf.GetPlane(c2, 0), c2_1, c2_2], planes=[0, 0, 0], colorfamily=vs.YUV)

a = mvf.GetPlane(c, 1)
b = mvf.GetPlane(c2, 1)
#a=c 
#b=c2
c = muf.SSIM(b,a).text.FrameProps()
c.set_output()
p.s the ssim score for the blurred chroma plane is 0.8


ChaosKing is offline   Reply With Quote
Old 5th April 2019, 15:31   #119  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,187
Quote:
Originally Posted by ChaosKing View Post
I don't know what a good weighted total score should look like.
I'm not sure. As mentioned in the other 'SSIM: GMSD metric thread', Zorr looked at the code for the AVISynth and FFMPEG SSIM metrics to see how they derive an overall score for the Y, U and V channels:

Quote:
Originally Posted by WorBry View Post
Zorr checked out the code for ffmpeg SSIM (AVISynth SSIM also) earlier in the thread. In essence, it's akin to the 'fast' SSIM metric in the MSU Quality Tool:

https://forum.doom9.org/showthread.p...30#post1866030

https://forum.doom9.org/showthread.p...62#post1866162
Quote:
Originally Posted by zorr View Post
The Avisynth version is not doing the gaussian kernel at all - it's implemented using summed area tables. That's a faster but lower quality way to calculate it, the MSU Quality measurement tool page has an example of the difference.

.....Avisynth SSIM has a plane argument which defaults to 0 and then it returns a weighted sum of the luma and chroma channels:
(0.8 * Y) + (0.1*(U+V))
But the AVISynth version is YV12 only.

Quote:
Originally Posted by zorr View Post
I looked at the ffmpeg source. It's doing the fast version too - no gaussian kernels there. The total SSIM takes into account all the planes but the weighting is different, each plane is scaled by the resolution is has. So for example with YUV420 the color planes have 4 times smaller weight. The constants 0.01 and 0.03 appear in the code and I assume that means the weights k1 and k2 are the same as in other implementations.
And:

Quote:
To improve speed, this implementation uses the standard approximation of overlapped 8x8 block sums, rather than the original gaussian weights
https://github.com/FFmpeg/FFmpeg/blo...lter/vf_ssim.c

So the ffmpeg variant applies different weights according to pixel format of the clips being tested, but they must have the same pixel format and resolution

https://ffmpeg.org/ffmpeg-filters.html#ssim

Of course the FFMPEG metric can also be run on RGB clips, generating scores for the individual R,G,B channels and an overall aggregate score. But it's still using the 'fast' and less precise 'standard approximation of overlapped 8x8 block sums'.

Is there provision in the original Matlab SSIM code for measuring and weighting the chroma or some other reference where this is defined - I don't know ?

I checked out the demo 'Pro' version of the MSU-QMT, btw - the demo version intentionally generates incorrect results as a kind of 'metric watermark' but it shows the options that are available for the different measures. Looks like the 'precise' (gauss blur) and 'fast' SSIM variants are measuring Y luma only.

And then there are these other 'SSIM algorithm-based' variants - DSSIM and SSIMULACRA - intended (like Butteraugli) primarily for measuring differences in still images, which process in CIE Lab* color space and apply weights to the L, a and b channels to derive an overall score:

https://github.com/kornelski/dssim

https://github.com/cloudinary/ssimulacra

https://cloudinary.com/blog/detectin...ing_ssimulacra

Personally, I'd be more interested in deriving a composite SSIM score for the combined chroma channels, rather than (or in addition to) an 'overall' score, but that would also require consideration of the weightings.
__________________
Nostalgia's not what it used to be

Last edited by WorBry; 5th April 2019 at 18:56.
WorBry is offline   Reply With Quote
Old 5th April 2019, 21:22   #120  |  Link
VS_Fan
Registered User
 
Join Date: Jan 2016
Posts: 91
Quote:
Originally Posted by WorBry View Post
... Personally, I'd be more interested in deriving a composite SSIM score for the combined chroma channels, rather than (or in addition to) an 'overall' score
How about stacking (horizontally or vertically) the u and v planes in vapoursynth, then generate YUV/Y4M files so you can obtain SSIM metrics by using any of the tools mentioned?

I would expect the result to be very close to averaging the results of the individual u and v planes.
VS_Fan is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 13:38.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.