Zopti

[ChaosKing]

When I try to show the clip in vsedit I get this error msg:
GREY color family cannot have RGB matrix coefficients

orig = mvf.ToRGB(orig)
muv.MDSI(clip, orig, show_maps=True)[1]

and with mvf.ToYUV( muv.MDSI(clip, orig, show_maps=True)[1] )
Resize error 1026: YUV color family cannot have RGB matrix coefficients.

How can I remove RGB matrix coefficients? Or what should I do?

[WorBry]

Originally Posted by ChaosKing

Yeah show_map returns not a clip but a clip tuple https://github.com/WolframRhodium/mu...sfunc.py#L4584
I think FrameEval awlays expects a clip as return!?

So what to do ?

[WorBry]

Originally Posted by ChaosKing

When I try to show the clip in vsedit I get this error msg:
GREY color family cannot have RGB matrix coefficients

orig = mvf.ToRGB(orig)
muv.MDSI(clip, orig, show_maps=True)[1]

and with mvf.ToYUV( muv.MDSI(clip, orig, show_maps=True)[1] )
Resize error 1026: YUV color family cannot have RGB matrix coefficients.

How can I remove RGB matrix coefficients? Or what should I do?

Originally Posted by HolyWu

core.std.SetFrameProp(clip, prop='_Matrix', delete=True)

I'm confused now. Is this a way to output the maps ? What does the [1] at the end of the MDSI call line do ?

[ChaosKing]

Originally Posted by WorBry

I'm confused now. Is this a way to output the maps ? What does the [1] at the end of the MDSI call line do ?

show_maps returns multiple clips

if show_maps:
        return clip1, gs_hvs, cs, gcs

muv.MDSI(clipA, clipB, show_maps=True)[0] <-- 0 means clip1
muv.MDSI(clipA, clipB, show_maps=True)[1] <-- 1 = gs_hvs, 2 = cs, 3 = gcs

OR 

clips = muv.MDSI(clipA, clipB, show_maps=True)
clip = clips[0]
gs_hvs = clips[1]
...

This works now, thx HolyWu

orig = muv.MDSI(orig, orig.grain.Add(42), show_maps=True)[2]
orig = core.std.SetFrameProp(orig, prop='_Matrix', delete=True)

[WorBry]

Yep, that works. Thanks.

[Iron_Mike]

Hi guys,

trying to run some metrics via Zopti... got a couple of questions...

(1) what is the easiest way to pass an image sequence (specifically EXR/DPX) to Zopti ?

(2) @WolframRhodium: why does muvsfunc only allow clips of the exact same format for the SSIM / GMSD metric ?

Thanks.

[poisondeathray]

Originally Posted by Iron_Mike

(1) what is the easiest way to pass an image sequence (specifically EXR/DPX) to Zopti ?

ImageMagick .

The vapoursynth version supports many pixel formats, 10bit, 16bit, float .... all in their native format (e.g. DPX will be 10bit RGB, instead of 16bit)
http://www.vapoursynth.com/doc/plugins/imwri.html

(The avs version is very outdated, 8bit, limited format support)

[WolframRhodium]

Originally Posted by Iron_Mike

(2) @WolframRhodium: why does muvsfunc only allow clips of the exact same format for the SSIM / GMSD metric ?

I expect people to convert format of clips themselves. I believe that allows the program to be correct, stable and concise.

[Iron_Mike]

Originally Posted by poisondeathray

ImageMagick .

The vapoursynth version supports many pixel formats, 10bit, 16bit, float .... all in their native format (e.g. DPX will be 10bit RGB, instead of 16bit)
http://www.vapoursynth.com/doc/plugins/imwri.html

(The avs version is very outdated, 8bit, limited format support)

thank you, that worked.

[Iron_Mike]

Originally Posted by WolframRhodium

I expect people to convert format of clips themselves. I believe that allows the program to be correct, stable and concise.

trying to understand the approach here... your implementation can only compare the exact same video format ?

so if we want to compare a 16bit full RGB master with a 12/10/8 bit encode in various chroma sub sampling formats, we would convert the master to the format of the encode... ?

or how is this done correctly ?

Thanks.

[WorBry]

Originally Posted by WolframRhodium

I have updated the code of MDSI. Usage:

src, gs, cs, gcs = MDSI(src, ref, show_maps=True)

You may have to normalize these maps yourself for better visualization.

The traces are very faint, but an Unsharp Mask (two passes if needs be) brings them up nicely.

[WolframRhodium]

Originally Posted by Iron_Mike

your implementation can only compare the exact same video format ?

Yes.

Originally Posted by Iron_Mike

so if we want to compare a 16bit full RGB master with a 12/10/8 bit encode in various chroma sub sampling formats, we would convert the master to the format of the encode... ?

Normally it would be:

yuv420p8 = core.resize.Bicubic(rgb48, format=vs.YUV420P8, matrix_s='709')

for Blu-ray material.

But it won't work for every case, like chroma locations other than left, or different matrix specification.

[Iron_Mike]

Originally Posted by WolframRhodium

yuv420p8 = core.resize.Bicubic(rgb48, format=vs.YUV420P8, matrix_s='709')

thanks for that, ran some test with the resize cmd you posted and with core.fmtc... results seems to be almost identical...

now, why do you downscale the master and not upscale the encode to the masters format before comparison ?

Thanks.

[WolframRhodium]

Originally Posted by Iron_Mike

now, why do you downscale the master and not upscale the encode to the masters format before comparison ?

It's just a trade-off. Downscaling is fast but inaccurate, while upscaling is accurate but slow. I'm just make a demonstration here instead of providing a high-quality conversion. (The quality of upscaling by nnedi3 is even higher, for example)

[WorBry]

Originally Posted by WorBry

@ WolframRhodium,

Regarding MDSI. Is there provision in the original code for producing the Gradient Similarity (GS), Chromacity Similarity (CS) and pooled Gradient-Chromacity Similarity (GCS) maps shown in the original paper ?



https://arxiv.org/pdf/1608.07433.pdf

I've been using quality metrics to compare various 'visually lossless' intermediate codecs and I'm seeing some interesting results with MDSI that appear to be chroma-related - they don't occur when testing with greyscale images. The similarity maps would helpful in identifying the implicated image components.

Thought the results might be of interest:

https://forum.doom9.org/showthread.p...59#post1868159

[ChaosKing]

https://github.com/theChaosCoder/zoptilib
New parameter: showstats - show metric stats per frame (vmaf and time are not supported) - Zopti(r'results.txt', metrics=['ssim', 'gmsd', 'mdsi', 'butteraugli', 'time'], showstats=True, matrix='709')
Updated WaDIQaM (pytorch) and added WaDIQaM nr/fr (chainer), untested bcs it needs cupy and other stuff. Lots of gigabytes und modules needs to be installed first.


Now I don't like the forced heatmap in butteraugli :/

[WorBry]

Originally Posted by ChaosKing

https://github.com/theChaosCoder/zoptilib
New parameter: showstats - show metric stats per frame (vmaf and time are not supported) - Zopti(r'results.txt', metrics=['ssim', 'gmsd', 'mdsi', 'butteraugli', 'time'], showstats=True, matrix='709')

Very useful. Thanks for that.

Originally Posted by ChaosKing

Now I don't like the forced heatmap in butteraugli :/

Me neither.

[zorr]

Originally Posted by ChaosKing

https://github.com/theChaosCoder/zoptilib
New parameter: showstats - show metric stats per frame

Great addition. It would look cleaner if Butteraugli stats were lined with the others. So can we shorten it to... butt?

[ChaosKing]

done

[ChaosKing]

https://github.com/theChaosCoder/zoptilib
Mdsi and butteraugli no longer contaminate the clip. That means metrics=['ssim', 'gmsd', 'mdsi', 'butteraugli'] is the same as metrics=['mdsi', 'butteraugli', 'ssim', 'gmsd'] now.
This also means that you won't see the heatmap...