Quote:
Originally Posted by benwaggoner
VMAF isn't a still image metric. Has anyone run a correlation for VMAF against subjective testing for still images?
|
Here you go, based on the
TID2013 dataset:
Code:
Actual profile:
Spearman: | Kendall:
PSNRHA 0.938 | PSNRHA 0.787
PSNRHMA 0.934 | PSNRHMA 0.777
PSNRHVS 0.926 | PSNRHVS 0.766
PSNRHVSM 0.917 | PSNRHVSM 0.749
FSIMc 0.915 | FSIMc 0.742
FSIM 0.911 | FSIM 0.736
WSNR 0.897 | WSNR 0.718
MSSIM 0.887 | MSSIM 0.697
VSNR 0.882 | VSNR 0.690
VMAF_v0.6.1 0.863 | VMAF_v0.6.1 0.675
VMAF_rb_v0.6.3 0.862 | VMAF_rb_v0.6.3 0.674
NQM 0.857 | NQM 0.666
PSNR 0.825 | PSNR 0.624
VIFP 0.815 | VIFP 0.621
PSNRc 0.803 | PSNRc 0.596
SSIM 0.788 | SSIM 0.577
Simple profile:
Spearman: | Kendall:
PSNRHA 0.953 | PSNRHA 0.818
PSNRHVS 0.951 | PSNRHVS 0.809
FSIM 0.949 | FSIM 0.795
FSIMc 0.947 | FSIMc 0.792
PSNRHVSM 0.938 | PSNRHMA 0.785
PSNRHMA 0.937 | PSNRHVSM 0.780
WSNR 0.933 | WSNR 0.772
PSNR 0.913 | PSNR 0.745
VSNR 0.912 | VSNR 0.731
MSSIM 0.905 | MSSIM 0.720
VIFP 0.897 | VIFP 0.714
VMAF_rb_v0.6.3 0.891 | VMAF_rb_v0.6.3 0.698
VMAF_v0.6.1 0.889 | VMAF_v0.6.1 0.696
PSNRc 0.876 | PSNRc 0.689
NQM 0.875 | NQM 0.681
SSIM 0.837 | SSIM 0.628
Full profile:
Spearman: | Kendall:
FSIMc 0.851 | FSIMc 0.666
PSNRHA 0.819 | PSNRHA 0.643
PSNRHMA 0.813 | PSNRHMA 0.631
FSIM 0.801 | FSIM 0.629
MSSIM 0.787 | MSSIM 0.607
VMAF_rb_v0.6.3 0.749 | VMAF_rb_v0.6.3 0.564
VMAF_v0.6.1 0.748 | VMAF_v0.6.1 0.563
PSNRc 0.687 | VSNR 0.508
VSNR 0.681 | PSNRHVS 0.507
PSNRHVS 0.654 | PSNRc 0.496
PSNR 0.640 | PSNRHVSM 0.481
SSIM 0.637 | PSNR 0.470
NQM 0.635 | NQM 0.466
PSNRHVSM 0.625 | SSIM 0.463
VIFP 0.608 | VIFP 0.456
WSNR 0.580 | WSNR 0.446
All bitmap images have been converted to raw full range YUV444P with ffmpeg and then measured with the vmafossexec program.
Code:
ffmpeg.exe -i i01_01_1.bmp -vf "scale=flags=accurate_rnd+bitexact+full_chroma_int+full_chroma_inp,format=yuvj444p" i01_01_1.bmp.yuv
vmafossexec.exe yuv444p 512 384 reference_images/i01.bmp.yuv distorted_images/i01_01_1.bmp.yuv model/vmaf_v0.6.1.pkl
vmafossexec.exe yuv444p 512 384 reference_images/i01.bmp.yuv distorted_images/i01_01_1.bmp.yuv model/vmaf_rb_v0.6.3/vmaf_rb_v0.6.3.pkl --ci
I'm also attaching the raw scores, for completeness sake.
A note on how to read the numbers:
from
the paper I get the following: a SROCC of 0.95 is considered excellent, 0.90 is good, and 0.85 is barely acceptable.