Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Closed Thread
 
Thread Tools Search this Thread Display Modes
Old 28th October 2019, 16:04   #141  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by richardpl View Post
Again FUD spreading from you! You also never provided commands you use and input files.

Again, which part do you believe is FUD ? This is a known issue with FFmpeg.

Quote:
(1) is still a potential issue -

for some types of input formats (seek issues with some open gop files, and those with "off" timecodes, you might need to reset PTS) - you get mismatched frames. Or use an indexed frame accurate input like avisynth or vapoursynth into ffmpeg

That you need to make adjustments when there is jitter in timestamps, or slightly different container timebases, or slightly "off" timestamps such as non zero start times



Here are 2 examples. I chose PSNR, because it's easy to demonstrate the issue (should get inf). In both cases "0" is the reference (but it shouldn't matter, since they are "lossless" equivalents)

If you were to follow the docs/ instructions, without making adjustments - you will get the wrong values
https://ffmpeg.org/ffmpeg-filters.html#psnr

*In both cases (and other cases, where there are timestamp issues, container timebase differences, "off" timecodes etc...) , if you just use avisynth or vapoursynth, it works . If the source filter you are using reads "off" frame rate (e.g. 23.976 vs. 24000/1001 exactly, or similar, you need to use AssumeFPS to make them equivalent if using FFmpeg; as long as the frames are aligned with a frame accurate source filter, it will work and give correct values)

https://www.mediafire.com/file/rd2zr...ample1.7z/file
https://www.mediafire.com/file/c9ieb...ample2.7z/file

example 1)
In the first case both are "23.976" by mediainfo (or 23.98 by ffmpeg) but there are tbn container differences . This is known as MP4 and MKV express timestamps differently

Also notice the non zero start time in "0.mkv" and duration difference because of it

ffmpeg -i 0.mkv
Duration: 00:00:05.01, start: 0.084000, bitrate: 38321 kb/s
Stream #0:0: Video: h264 (High), yuv420p(tv, bt709, progressive), 1920x1080
[SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc (default)

ffmpeg -i 1.mp4
Duration: 00:00:04.92, start: 0.000000, bitrate: 74719 kb/s
Stream #0:0(und): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), y
uv420p, 1920x1080 [SAR 1:1 DAR 16:9], 74723 kb/s, 23.98 fps, 23.98 tbr, 24k tbn,
47.95 tbc (default)


If you were to follow the docs, you will get wrong values

Code:
ffmpeg -i 1.mp4 -vcodec rawvideo -vf "movie=0.mkv,setpts=PTS-STARTPTS[main];[main][ref]psnr="stats_file=stats1.log" [out]" -f rawvideo -y /NUL
If you were to follow the docs that specific different containers, you will still get wrong values
Code:
ffmpeg -i 1.mp4 -i 0.mkv -lavfi  "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]psnr" -f null -
-i avs , or -i vpy works, or forcing the framerate -r 24000/1001 before each -i works in this particular case (but not necessarily others)

Code:
ffmpeg -r 24000/1001 -i 1.mp4 -r 24000/1001 -i 0.mkv -lavfi  "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]psnr" -f null -

example 2)
ffmpeg -i 0.mkv
Duration: 00:00:10.26, start: 0.000000, bitrate: 18583 kb/s
Stream #0:0(eng): Video: h264 (High), yuv420p(progressive), 1920x1080 [SAR 1
:1 DAR 16:9], 23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc (default)

ffmpeg -i 1.mp4
Duration: 00:00:10.26, start: 0.000000, bitrate: 120532 kb/s
Stream #0:0(eng): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), y
uv420p, 1920x1080 [SAR 1:1 DAR 16:9], 120539 kb/s, 23.98 fps, 23.98 tbr, 24k tbn
, 47.95 tbc (default)

Same issue with container tbn difference, but start time is zero in this case

If you were to follow the docs, you will get wrong values
Code:
ffmpeg -i 1.mp4 -vcodec rawvideo -vf "movie=0.mkv,setpts=PTS-STARTPTS[main];[main][ref]psnr="stats_file=stats2.log" [out]" -f rawvideo -y /NUL
If you were to follow the docs that specific different containers, you will still get wrong values
Code:
ffmpeg -i 1.mp4 -i 0.mkv -lavfi  "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]psnr" -f null -
If you were to follow the docs that specific different containers, and use the -r force framerate adjustment that worked in example #1 above, you will still get wrong values
Code:
ffmpeg -r 24000/1001 -i 1.mp4 -r 24000/1001 -i 0.mkv -lavfi  "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]psnr" -f null -
-i avs, or -vpy works
poisondeathray is offline  
Old 28th October 2019, 17:36   #142  |  Link
richardpl
Registered User
 
Join Date: Jan 2012
Posts: 272
I downloaded both your uploaded files.
First file contains two video files that have same decoded video frames hashes. And for this files example in documentation about using psnr with different containers works just fine (I dunno what was your point in trying command that works only with same containers and with nice timestamps). Perhaps you used wrong command or faulty build?

I used this command:
Code:
ffmpeg -i 0.mkv -i 1.mp4 -lavfi "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]psnr=shortest=1" -f null -
Extra shortest is for cases when user supplied videos of different duration.

Second file contain two video files with different video frames hashes. One file have duplicated first two frames with normal timestamps. No point in using psnr on them. If you have same results with vpy or avs then there is something seriously broken there with them
richardpl is offline  
Old 28th October 2019, 17:48   #143  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by richardpl View Post
I downloaded both your uploaded files.
First file contains two video files that have same decoded video frames hashes. And for this files example in documentation about using psnr with different containers works just fine (I dunno what was your point in trying command that works only with same containers and with nice timestamps). Perhaps you used wrong command or faulty build?

I used this command:
Code:
ffmpeg -i 0.mkv -i 1.mp4 -lavfi "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]psnr=shortest=1" -f null -
Extra shortest is for cases when user supplied videos of different duration.

Second file contain two video files with different video frames hashes. One file have duplicated first two frames with normal timestamps. No point in using psnr on them. If you have same results with vpy or avs then there is something seriously broken there with them

Shortest works for the 1st one, as does forcing -r .





The 2nd one has open GOP's with leading b's . Different decoders can handle open GOP's differently. But the same decoder should handle it consistently.

In fact, FFmpeg was used to create 1.mp4 from 0.mkv. And the decoded elementary streams are identical

Code:
ffmpeg -i 0.mkv -c:v rawvideo 0.yuv

ffmpeg -i 1.mp4 -c:v rawvideo 1.yuv

ffmpeg -pix_fmt yuv420p -s 1920x1080 -r 24000/1001 -i 1.yuv -pix_fmt yuv420p -s 1920x1080 -r 24000/1001 -i 0.yuv -lavfi  "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]psnr" -f null -
Or, if you use avs or vpy input, works too


This is why you need to make adjustments for some types of files. The docs don't cover these cases or others like them; if a user just followed the instructions, they'd get the wrong values

Last edited by poisondeathray; 28th October 2019 at 18:35.
poisondeathray is offline  
Old 28th October 2019, 19:11   #144  |  Link
richardpl
Registered User
 
Join Date: Jan 2012
Posts: 272
No, 2nd one is completely invalid, and they are not identical.
richardpl is offline  
Old 28th October 2019, 19:20   #145  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by richardpl View Post
No, 2nd one is completely invalid, and they are not identical.
Explain.

How is that possible, when 1.mp4 was created from 0.mkv

ffmpeg -i 0.mkv -c:v libx264 -qp 0 1.mp4


Also, rawvideo says they are identical

Code:
ffmpeg -i 0.mkv -c:v rawvideo 0.yuv

ffmpeg -i 1.mp4 -c:v rawvideo 1.yuv

ffmpeg -pix_fmt yuv420p -s 1920x1080 -r 24000/1001 -i 1.yuv -pix_fmt yuv420p -s 1920x1080 -r 24000/1001 -i 0.yuv -lavfi  "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]psnr" -f null -
poisondeathray is offline  
Old 29th October 2019, 12:54   #146  |  Link
richardpl
Registered User
 
Join Date: Jan 2012
Posts: 272
They are not identical, and you know it as you created such files.

There is visible gap in pts of first and second frame of 0.mkv. Making it not CFR but VFR. rawvideo will just make CFR from VFR.
richardpl is offline  
Old 29th October 2019, 14:38   #147  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by richardpl View Post
They are not identical, and you know it as you created such files.

There is visible gap in pts of first and second frame of 0.mkv. Making it not CFR but VFR. rawvideo will just make CFR from VFR.

They are identical.

It's a lossless encode -qp 0 , encoded by FFmpeg. The commandlines are posted. Do you see conceptually the problem ? Someone uses FFmpeg to encode a "lossless" video in the same colorspace/sampling/bit depth - they expect it to be "lossless", they expect PSNR to say "lossless"

It's not an encoding issue, since the YUV streams match, encoded frames are identical.


It should have been CFR, because it was cut from a BD. BD's are CFR only.

This has open GOP with 2 leading b's. Decoders are supposed to either duplicate the leading b's or drop them (risk of desync). Eitherway , it should be consistently handled



If you go back, that PTS example for different container in the documentation is new - It wasn't there even 2 months ago if I look at my local documentation for a FFmpeg build. That's great, the docs need to be improved with more examples


That's why I said -
Quote:
(1) is still a potential issue -

for some types of input formats (seek issues with some open gop files, and those with "off" timecodes, you might need to reset PTS) - you get mismatched frames. Or use an indexed frame accurate input like avisynth or vapoursynth into ffmpeg

That you need to make adjustments when there is jitter in timestamps, or slightly different container timebases, or slightly "off" timestamps such as non zero start times
If you think that's "FUD", we wouldn't have these types of issues . Everything would just "work" without having to do these workarounds. The warning is so people don't get "burned"

Note that 1st example was a lossless encode from "0.mkv" too. The lengths are actually the same in terms of framecount. The -shortest or -r workaround are not mentioned in the docs. People expect things to just work; a casual user would have been "burned" there too

This illustrates some of the issues with how FFmpeg handles this calculation. There should be a way to reset all the PTS, you can improve it farther.
poisondeathray is offline  
Old 29th October 2019, 18:20   #148  |  Link
richardpl
Registered User
 
Join Date: Jan 2012
Posts: 272
Quote:
Originally Posted by poisondeathray View Post
It's not an encoding issue, since the YUV streams match, encoded frames are identical.


It should have been CFR, because it was cut from a BD. BD's are CFR only.

This has open GOP with 2 leading b's. Decoders are supposed to either duplicate the leading b's or drop them (risk of desync). Eitherway , it should be consistently handled
You are terribly mistaken. Closely inspect timestamp and you will see that 2 frames are completely missing from 0.mkv
There is gap created by ffmpeg decoder.
It works with vpy, avs just by pure luck because they duplicate right frames in those gaps. fps filter duplicates currently always first frame encountered so far across pts gap, which does not mach with frame hashes in 1.mp4

See for yourself with -vf showinfo,trim=end_frame=5 filter on both files.
The -r option is hacky beacuse it tries to round to nearest frame but is slightly off for some strange reason.
richardpl is offline  
Old 29th October 2019, 18:26   #149  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by richardpl View Post
You are terribly mistaken. Closely inspect timestamp and you will see that 2 frames are completely missing from 0.mkv
There is gap created by ffmpeg decoder.
It works with vpy, avs just by pure luck because they duplicate right frames in those gaps. fps filter duplicates currently always first frame encountered so far across pts gap, which does not mach with frame hashes in 1.mp4

See for yourself with -vf showinfo,trim=end_frame=5 filter on both files.
The -r option is hacky beacuse it tries to round to nearest frame but is slightly off for some strange reason.

Those 2 are leading b's frames from open GOP. The ffmpeg AVC libavcodec decoder places 2 duplicate frames there as "placeholder frames" to keep sync (had it been a real example with audio). This occurs with DVD / mpeg2 as well . Same handling with leading b's open GOP

(Other decoders might handle it differently; the point is it should be consistent with the same decoder; if it drops, then drop in all cases, not just some of the time or depending on the phase of the moon. Most AVC decoders will keep them as duplicates)

But that's why the encode has them as well, and the decoded YUV frames are identical, the 1st 2 are duplicates from the open gop . Same framecount. Frames are the same . Input = output. ( The encode was made from decoding first AVC stream with open GOP; and the decode of the lossless encode is also decoding AVC but they are not longer leading b's, they've been decoded and reencoded - both are consistent and have those 2 "placeholder" frames)

The problem is the FFmpeg timestamp handling . At the very least , you have to admit it's inconsistent behaviour here. On one hand FFmpeg places those b-frames. Physically decodes and encodes therm. On the other hand, when doing the calculations it drops them

Just take a step back and look at the big picture for a second: Someone supposedly encodes a lossless video with ffmpeg and it's "not lossless" according to FFmpeg PSNR . Or -
FFmpeg VMAF is not going to match vmafossexec or vapoursynth-vmaf in cases like this - unless you decode to raw YUV first for use in FFMpeg .

Last edited by poisondeathray; 29th October 2019 at 19:33.
poisondeathray is offline  
Old 16th December 2019, 11:23   #150  |  Link
Shevach
Video compressionist
 
Join Date: Jun 2009
Location: Israel
Posts: 126
Quote:
Originally Posted by Boulder View Post
Yes, looking at the models page suggests that.. Looking at the linked slideshow there, it also seems that for proper analysis, you have to upscale to 4K.

EDIT: with video upscaled to 4K, it runs out of memory quite often. Even with core.max_cache_size = 1024, the usage jumps to over 8GB quite fast and then the errors appear. I have 16GB on the machine so it's really not using all the memory.

Code:
Start calculating VMAF score...
Script exceeded memory limit. Consider raising cache size.
error: aligned_malloc failed for data_buf.
error: aligned_malloc failed for data_buf.
error: aligned_malloc failed for data_buf.
Exec FPS: 3.574859
VMAF score (harmonic_mean) = nan
Dear experts

i am faced with same problem while running libvmaf by means of ffmpeg:

.\ffmpeg.exe -hide_banner -r 60 -s 1920x1080 -i test60fps.yuv -r 60 -s 1920x1080 -i ref.yuv -vframes 2372 -lavfi libvmaf="model_path=vmaf_v0.6.1.pkl:log_path=test30Mbps60fps.xml" -report -f null -

...
frame= 81 fps= 19 q=-0.0 size=N/A time=00:00:03.24 bitrate=N/A speed=0.761x
frame= 89 fps= 19 q=-0.0 size=N/A time=00:00:03.56 bitrate=N/A speed=0.748x
frame= 98 fps= 19 q=-0.0 size=N/A time=00:00:03.92 bitrate=N/A speed=0.74x
error: aligned_malloc failed for data_buf.
error: aligned_malloc failed for data_buf.

If i downscale raw video resolution to 720x480 the ffmpeg succeeds to compute vmaf scores for all frames.

Task Manager does not reveal any problem (the memory usage does not exceed 5%).

How can i resolve this issue?
Shevach is offline  
Old 16th December 2019, 12:06   #151  |  Link
Sharc
Registered User
 
Join Date: May 2006
Posts: 3,997
I can't help you with ffmpeg/libvmaf. However, you may want to make the VMAF analysis with vmafossexec.exe instead.
Sharc is offline  
Old 17th December 2019, 10:22   #152  |  Link
Shevach
Video compressionist
 
Join Date: Jun 2009
Location: Israel
Posts: 126
Quote:
Originally Posted by Sharc View Post
I can't help you with ffmpeg/libvmaf. However, you may want to make the VMAF analysis with vmafossexec.exe instead.
Thanks for the prompt reply.

i can confirm that vmafossexec.exe works (at least calculation of vmaf scores of above 2K frames succeeds).

i downloaded vmafossexec.exe from Jan Ozer's paper:
https://streaminglearningcenter.com/...af-master.html

There is a link where Windows executable are located (incl. vmafossexec.exe) and a small note:
"this executable is not the most current version. To get the most current version, youÂ’ll have to download from Github and compile yourself."
Shevach is offline  
Old 18th December 2019, 23:55   #153  |  Link
Sharc
Registered User
 
Join Date: May 2006
Posts: 3,997
I am using the same version from the same source. Don't know whether a more recent compiled executable exists from somebody.
Sharc is offline  
Old 8th December 2020, 15:15   #154  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
v2 was just released.
https://github.com/Netflix/vmaf/releases/tag/v2.0.0
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline  
Old 8th December 2020, 15:58   #155  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
a non-referential quality assessment tool would be more useful I guess since access to the groundtruth (reference) is usually not viable...
like when you need to dynamically adjust the filtering strength depending on the quality of each frame

Last edited by feisty2; 8th December 2020 at 16:25.
feisty2 is offline  
Old 8th December 2020, 16:27   #156  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
Something like Av1an + vmaf does on the av1 encoder level?

https://github.com/master-of-zen/Av1...eases/tag/1.10

Quote:
From my testing, result size can be 50-80% of compared to usual encode, with great visual quality.
But it needs 40% more encoding time.
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 01:59.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.