Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
|
7th November 2018, 16:52 | #4 | Link |
Registered User
Join Date: Dec 2005
Location: Germany
Posts: 1,795
|
It calculates a score.... You have 2 clips, the score tells you by how much the the seconds clips differs from the first one. The VMAF algo tries take the human perception into account. https://en.wikipedia.org/wiki/Video_...essment_Fusion
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth VapourSynth Portable FATPACK || VapourSynth Database |
8th November 2018, 11:11 | #6 | Link |
Registered User
Join Date: Dec 2005
Location: Germany
Posts: 1,795
|
Also available via vsrepo now
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth VapourSynth Portable FATPACK || VapourSynth Database |
9th November 2018, 02:56 | #7 | Link |
Registered User
Join Date: Dec 2009
Posts: 72
|
Very timely for me. Thanks.
I've had mixed results getting yuv422p10le to work correctly with vmafossexec and/or ffmpeg builds with libvmaf. That's probably my fault for trying on Windows, but I was too lazy to try on a Linux VM at the time. |
14th November 2018, 12:21 | #9 | Link |
Registered User
Join Date: Aug 2006
Location: Taiwan
Posts: 392
|
Update r2.
|
1st March 2019, 05:51 | #10 | Link | |
Registered User
Join Date: Jul 2010
Posts: 132
|
Quote:
does that mean we have to scale both inputs to 8bit before calculating VMAF, or does VMAF do it automatically ? I'm comparing a 16bit exr sequence to itself (as control) and I get a 98.2 VMAF score... I know that the FAQ states that this is normal, just making sure I do this correctly... most of the examples here use 8bit source/ref footage... When I compare a 12bit 444 (yuv444p12le) CRF 10 x265 encode to the 16 bit exr ref footage I get a 96.4 VMAF score... little bit low considering the tests that WorBry has done... Also, how do I use a "stricter linear frame request" ? Thanks. Last edited by Iron_Mike; 1st March 2019 at 05:55. |
|
1st March 2019, 15:19 | #11 | Link | |
Registered User
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,197
|
Quote:
Bear in mind though that all of those tests were done on a single source. Could be any number of factors weighing in there. What scores to you get if you get if you test the ref and x265 clips against themselves ?
__________________
Nostalgia's not what it used to be Last edited by WorBry; 1st March 2019 at 15:57. |
|
1st March 2019, 22:42 | #12 | Link | |
Registered User
Join Date: Jul 2010
Posts: 132
|
Quote:
16bit EXR ref clip is 400 frames - control test to itself via 0.6.1 results in 98.2 x265 12bit 444 encode (from EXR), control tested to itself via 0.6.1 results in 98.08 x265 12bit 444 encode (from EXR) tested against it's source (16 bit EXR) via 0.6.1 results in 96.4 interesting that the clip you tested had 399 out of 400 frames a perfect 100 in the control test... |
|
1st March 2019, 23:09 | #13 | Link | |
Registered User
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,197
|
Quote:
Go through the list of per-frame VMAF scores from your 'self' tests and you'll be able to identify which frames are skewing the aggregate score.
__________________
Nostalgia's not what it used to be |
|
1st March 2019, 16:43 | #14 | Link | |
Registered User
Join Date: Sep 2007
Posts: 5,374
|
Quote:
I'm not sure how valid that test would be. EXR is usually sRGB linear , and 16bit half float . For any metric you usually need a common ground to compare. This means same pixel format (same colorspace, same bit depth, same chroma subsampling) . Otherwise you introduce other variables that are not controlled for. e.g. if one run uses one algorithm to scale (e.g. bicubic vs. bilinear, vs...) , or another dithers down using one algorithm, but another does not... or if you convert to RGB using different matrix, etc... there are many factors that invalidate your testing |
|
1st March 2019, 22:12 | #15 | Link | |
Registered User
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,197
|
Not to mention the potential for frame shifts/misalignment when using different decoders for the reference and test clips, although the filter will report an error if the number of frames is different.
Also needs to be appreciated that the VMAF models are 'trained' for predicting perceptual quality at streaming bitrates primarily. Came across this quote from Netflix: Quote:
I would assume a similar focus was applied in training the 4K model. So at CRF 10 you are well into uncharted territory. Personally, I'd be more inclined to look at other metrics available for VapourSynth that are (maybe) better attuned for VQA in the visually lossless domain - GMSD, MDSI and yes, SSIM....Butteraugli, possibly. My own journey of discovery in that vein continues: https://forum.doom9.org/showthread.php?t=176101
__________________
Nostalgia's not what it used to be Last edited by WorBry; 1st March 2019 at 22:19. |
|
1st March 2019, 22:59 | #16 | Link | |
Registered User
Join Date: Jul 2010
Posts: 132
|
Quote:
ffmpeg reads an EXR frame and then reads a frame from the mp4 encode (that was done from the EXR via ffmpeg) and then passes the decoded frames to VMAF for comparison... is there a setting to avoid frame shifts/misalignment ? you yourself validated the test clip up to CRF 0 and results on the charts make sense... your encodes went up to VMAF 100... not sure I understand your concern for CRF 10 ? Thanks Last edited by Iron_Mike; 1st March 2019 at 23:07. |
|
1st March 2019, 22:51 | #17 | Link | |
Registered User
Join Date: Jul 2010
Posts: 132
|
Quote:
for streaming the movie was encoded via ffmpeg and x265 from EXR (rgb48le) to x265 12bit 444 (yuv444p12le) - final result looks very good, we're using VMAF to compare various encodes (presets/CRF/etc) against each other... exact same as NF does it w/ VMAF... the master one delivers to NF is obviosly also not 8bit 420, they encode from that (high quality) master for NF streaming... so the "common ground" you state is the same movie in the same resolution, which is the only thing that NF states in their VMAF instructions... the whole reason for comparison is different output bit depth w/ different output chroma subsampling, on top of different encoding settings, so I do not understand your point... Last edited by Iron_Mike; 1st March 2019 at 22:53. |
|
14th November 2018, 15:51 | #19 | Link |
Registered User
Join Date: Dec 2005
Location: Germany
Posts: 1,795
|
This would be awesome.
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth VapourSynth Portable FATPACK || VapourSynth Database |
15th November 2018, 03:43 | #20 | Link |
Registered User
Join Date: Aug 2006
Location: Taiwan
Posts: 392
|
Not possible with the exposed API of libvmaf. Only when all frames are deliverd and processed does the library print the aggregate scores and optionally write the log file. There is no way to access the library's internal data to get the per-frame score and add that to the frame property.
Last edited by HolyWu; 15th November 2018 at 03:57. |
Thread Tools | Search this Thread |
Display Modes | |
|
|