Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 20th March 2019, 09:25   #201  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
Quote:
Originally Posted by Iron_Mike View Post
another thing I just saw via HEVC browser...

re (1): the encode places all 5x IDR keyframes @ 100 frames interval (as specified)...

re (2) the encode places the 4x simple I frames (following the only real IDR keyframe) in incorrect position, they are a few frames off

re (3) the encode places the 1x simple I frame (following the only real IDR keyframe) in correct position
HEVC browser stats of the 3 files I uploaded - note: frame count starts at 1, not at 0 in the below stats

re (1) - x265 w/ closed-gop and custom keyframes intval 100 - IDR keyframes at positions: 1, 101, 201, 301, 401 | simple (non-IDR) I frames: none

re (2) - x265 w/ open-gop and custom keyframes intval 100 - IDR keyframes at positions: 1 | simple (non-IDR) I frames at positions: 97, 197, 297, 397

re (3) - x265 w/ open-gop and no custom keyframes intval - IDR keyframes at positions: 1 | simple (non-IDR) I frames at positions: 247


not sure how ffms2 works, but if it reads file (2), then sees the specified GOP size with keyintval of 100 (in the encoding settings), and then does not find the IDR frames in the correct positions, or matter of fact does not find any IDR frames at all (b/c ffmpeg w/ open-gop only places simple I frames), then that may be the reason why it acts up...

Last edited by Iron_Mike; 20th March 2019 at 09:28.
Iron_Mike is offline   Reply With Quote
Old 20th March 2019, 09:50   #202  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
There are new ffms2 / lsmash builds up. But seems to behave the same.
And I tested the mp4 with the old ffms2 and it has no seeking problems. https://github.com/FFMS/ffms2/releas...2.23.1-msvc.7z
(This is the second file what has no problems with the old version)

I tested also a build by HolyWu (ffmpeg 4.1.1) https://forum.doom9.org/showthread.p...11#post1866411
-> Seeking issues
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline   Reply With Quote
Old 20th March 2019, 10:29   #203  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,565
Quote:
Originally Posted by Iron_Mike View Post
not sure how ffms2 works, but if it reads file (2), then sees the specified GOP size with keyintval of 100 (in the encoding settings), and then does not find the IDR frames in the correct positions
It doesn't work like that. No decoder cares about keyint in custom/optional "encoding settings" SEI that x264/x265 write.
sneaker_ger is offline   Reply With Quote
Old 20th March 2019, 15:27   #204  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
It's ok display order wise.

The offset position and encoding order does not necessarily reflect the display order of the frames. That utility does not display the GOP number , display frame number, encoded frame number , but hey it's free

There is another analyzer, GitlHEVCAnalyzer on sourceforge but it only supports 8bit HEVC . But you guys are using 8bit here, but it will display a graph like elecard with "I" slices in red. But you can see they are placed correctly. Another method, with avisynth ffms2 is to use FFInfo() and it will display the pic type. But again, it does not distinguish between IDR types , or non IDR "i". Just simple I,P,B slices . Or commercial analyzers ($) .
poisondeathray is offline   Reply With Quote
Old 20th March 2019, 16:11   #205  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
Quote:
Originally Posted by poisondeathray View Post
It's ok display order wise.

with avisynth ffms2 is to use FFInfo() and it will display the pic type. But again, it does not distinguish between IDR types , or non IDR "i". Just simple I,P,B slices . Or commercial analyzers ($) .
VS version: core.ffms2.Source(source=r"D:\Download\crowd_run_1080p50.y4m").text.FrameProps(props=['_PictType'])

or just text.FrameProps() to show all frame props
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline   Reply With Quote
Old 20th March 2019, 16:14   #206  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,197
Quote:
Originally Posted by poisondeathray View Post
@WorBry - for x265 it's --frame-threads . I guess ffmpeg -threads get passed as libx265 frame threads. A value of 6 would correspond to a 32 core machine . You can see on the chart
https://x265.readthedocs.io/en/latest/threading.html

The frame thread effect is smaller than with x264, but it's the reason why your values were slightly higher, because it should have been 2 with a 6C/6T machine. But if you specified -threads 6 , and that got passed to libx265 as frame-threads, then I would have expected the scores to be lower...

Quote:
Originally Posted by WorBry View Post
It was really oversight on my part I have my commands on a text file - copied another script for the file paths and neglected to delete the -threads 6.
So, I've re-encoded the x265 files omitting the "-threads 6". MediaInfo reports 'frame-threads=2'.

These are the SSIM:GMSD (Downsample=False) obtained with Wolfberry's ffms2 (default seekmode=1)

Code:
crowd_run_1080p50_y4m_x265_CRF28_v2.mp4:

stop 436.0761918885030863357599170 83.36467719049633404160282170 
stop 436.0761918885030863357599170 83.36467719049633404160282170 
stop 436.0761918885030863357599170 83.36467719049633404160282170 
stop 436.0761918885030863357599170 83.36467719049633404160282170 
stop 436.0761918885030863357599170 83.36467719049633404160282170 

crowd_run_1080p50_y4m_x265_CRF28_KeyInt100_v2.mp4:

stop 435.9938560956790122302706430 83.19534706841788751496835637 
stop 435.9938560956790122302706430 83.19534706841788751496835637 
stop 435.9938560956790122302706430 83.19534706841788751496835637 
stop 435.9938560956790122302706430 83.19534706841788751496835637 
stop 435.9938560956790122302706430 83.19534706841788751496835637
Compared with the x265 files that were encoded with '-threads 6' (MediaInfo reports 'frame-threads=6')

Code:
crowd_run_1080p50_y4m_x265_CRF28.mp4:

stop 436.0761918885030863357599170 83.36467719049633404160282170 
stop 436.0761918885030863357599170 83.36467719049633404160282170 
stop 436.0761918885030863357599170 83.36467719049633404160282170 
stop 436.0761918885030863357599170 83.36467719049633404160282170 
stop 436.0761918885030863357599170 83.36467719049633404160282170 

crowd_run_1080p50_y4m_x265_CRF28_KeyInt100.mp4:

stop 435.9938560956790122302706430 83.19534706841788751496835637 
stop 435.9938560956790122302706430 83.19534706841788751496835637 
stop 435.9938560956790122302706430 83.19534706841788751496835637 
stop 435.9938560956790122302706430 83.19534706841788751496835637 
stop 435.9938560956790122302706430 83.19534706841788751496835637
So it actually made no difference to the results.

In fact the two crowd_run_1080p50_y4m_x265_CRF28.mp4 files are byte identical and the crowd_run_1080p50_y4m_x265_CRF28_KeyInt100.mp4 files differ by only 5 bytes.

Still, here are the re-encoded files (omitting -threads 6):

crowd_run_1080p50_y4m_x265_CRF28_v2
__________________
Nostalgia's not what it used to be

Last edited by WorBry; 20th March 2019 at 21:37.
WorBry is offline   Reply With Quote
Old 20th March 2019, 16:54   #207  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by WorBry View Post

So it actually made no difference to the results.
Maybe not enough vertical motion to exhibit the frame threads effect in this shot. No idea why your values where consistently slightly higher than Iron_Mikes' (frame threads would have been a nice way to explain it) . Maybe some more inconsistencies between runs or systems

But beyond that , look at the bitrate / filesize between your uploads and his. Way different. Some settings are actually different too looking at mediainfo . psy-rdoq is 0 for his 1 for yours for the keyint 100 encode, rd4 vs 2 etc... . Need to compare apples to apples . Or at least look at the same physical file
poisondeathray is offline   Reply With Quote
Old 20th March 2019, 17:30   #208  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,197
He modified the 'original' test encode settings for those uploads - changed -preset slow to fast and added -rc-lookahead=100:

https://forum.doom9.org/showthread.p...02#post1869402

Why, I don't know.

So he should upload x265 files encoded with the 'original' scripts for valid comparison:

https://forum.doom9.org/showthread.p...90#post1869390
__________________
Nostalgia's not what it used to be

Last edited by WorBry; 20th March 2019 at 21:28.
WorBry is offline   Reply With Quote
Old 20th March 2019, 23:21   #209  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
Quote:
Originally Posted by WorBry View Post
He modified the 'original' test encode settings for those uploads - changed -preset slow to fast and added -rc-lookahead=100:

https://forum.doom9.org/showthread.p...02#post1869402

Why, I don't know.

So he should upload x265 files encoded with the 'original' scripts for valid comparison:

https://forum.doom9.org/showthread.p...90#post1869390

you misread my post (the last post you linked to):

rc-lookahead was ALWAYS a parameter I included if I specified custom keyframes... but I did ONE TEST to see if specifically the rc-lookahead parameter was the problem for ffms2, hence I removed that param in that ONE encode...

which is stated in my original post, and as you can see in point (2) in that post - which you removed from the quote - I have used the rc-lookahead param...


All files uploaded by me were done w/ the rc-lookahead param (if custom keyframes were specified), only reason I had to re-upload once is b/c I used incorrect syntax for the no-open-gop flag on one of the three files...

so, if a custom keyframes intval was specifed these three params were added to the x265-params: keyint=100:min-keyint=100:rc-lookahead=100
Iron_Mike is offline   Reply With Quote
Old 21st March 2019, 00:41   #210  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,197
No I didn't misread your post. You changed the preset to fast also. First you are following my encode script, as a common ground - now you're not. Have you even taken the trouble to test the files I uploaded and post the results, as I did yours ? Really can't be bothered with this.

@Poisondeathray, I'm interested to know how your results with the files I uploaded compare anyway.

Cheers.
__________________
Nostalgia's not what it used to be

Last edited by WorBry; 21st March 2019 at 00:46.
WorBry is offline   Reply With Quote
Old 21st March 2019, 01:24   #211  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,197
Quote:
Originally Posted by ChaosKing View Post
And I tested the mp4 with the old ffms2 and it has no seeking problems. https://github.com/FFMS/ffms2/releas...2.23.1-msvc.7z
(This is the second file what has no problems with the old version)
Trouble is, VS crashes when I try to test it.

Think I'll switch to LWLibavSource for future metric testing.
__________________
Nostalgia's not what it used to be
WorBry is offline   Reply With Quote
Old 21st March 2019, 01:30   #212  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by WorBry View Post
@Poisondeathray, I'm interested to know how your results with the files I uploaded compare anyway.
Not repeatable results here , unless seekmode=0 . I only tested your keyint 100, 6 frame thread version . Larger errors on some runs, clearly frame mismatches

Not able to replicate the 12th or whatever tiny decimal error with seekmode=0 over 10 runs . 100% consistent

Tried 2 computers, but both were 4C/8T Intels . I also have a really old AMD (with XP!) lying around but I didn't test on that



EDIT #1: also on the other file, with default settings (no custom keyint) , it was inconsistent too. Iron_Mike reported "big" inconsistent results on some runs only when using custom keyint . I'm seeing inconsistent everywhere with ffms2 default seekmode on these.

EDIT #2: remuxing to mkv with mkvmerge made everything consistent for the keyint 100 file (10 runs matched) with default seekmode, as CK also reported . This makes me think it has something to do with ffms2 reading the timebase and timecodes in the container , or container mux, or interpretation issue

ffmpeg -i input.ext

Code:
mkv
    Stream #0:0: Video: hevc (Main), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR
16:9], 50 fps, 50 tbr, 1k tbn, 50 tbc (default)

mp4
    Stream #0:0(und): Video: hevc (Main) (hev1 / 0x31766568), yuv420p(tv, bt709,
 progressive), 1920x1080 [SAR 1:1 DAR 16:9], 15411 kb/s, 50 fps, 50 tbr, 12800 t
bn, 50 tbc (default)
ffmpeg "thinks" the tbn is 12800 for the mp4 . tbn is the container time base

Last edited by poisondeathray; 21st March 2019 at 01:52.
poisondeathray is offline   Reply With Quote
Old 21st March 2019, 02:01   #213  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
Quote:
Originally Posted by WorBry View Post
No I didn't misread your post. You changed the preset to fast also. First you are following my encode script, as a common ground - now you're not. Have you even taken the trouble to test the files I uploaded and post the results, as I did yours ? Really can't be bothered with this.

@Poisondeathray, I'm interested to know how your results with the files I uploaded compare anyway.

Cheers.
You did misread as your own post clearly proves, b/c I state in detail in my post that I left out rc-lookadhead in one encode...... !?

The preset does NOT matter in any of this - no idea why you say any of that.

I also don't care if you test ffms2 or not. I posted and proved that there are issues w/ seekmode=1, other people confirmed.

Don't care whether you personally test this or not.

The only reason why I even bothered to write here is b/c you spend some time doing some tests and if you ever are subject to this behavior - or other inconsistent behavior we've not yet discovered - then your posted results here would be pointless.

Hence, it's always good if another source confirms general trends of test results of these metrics... which I did.

Last edited by Iron_Mike; 21st March 2019 at 02:05.
Iron_Mike is offline   Reply With Quote
Old 21st March 2019, 02:23   #214  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
Quote:
Originally Posted by poisondeathray View Post
Not repeatable results here , unless seekmode=0 . I only tested your keyint 100, 6 frame thread version . Larger errors on some runs, clearly frame mismatches

Not able to replicate the 12th or whatever tiny decimal error with seekmode=0 over 10 runs . 100% consistent

Tried 2 computers, but both were 4C/8T Intels . I also have a really old AMD (with XP!) lying around but I didn't test on that

EDIT #1: also on the other file, with default settings (no custom keyint) , it was inconsistent too. Iron_Mike reported "big" inconsistent results on some runs only when using custom keyint . I'm seeing inconsistent everywhere with ffms2 default seekmode on these.
tested WorBry's files...


WorBry - crowd_run_1080p50_y4m_x265_CRF28.mp4 - seekmode=1 - GMSD/SSIM

stop 83.36467719049633404160282170 436.0761547550154326735594167
stop 83.36467719049633404160282170 436.0761547550154326735594166
stop 83.36467719049633404160282170 436.0761547550154326735594166
stop 83.36467719049633404160282170 436.0761547550154326735594166
stop 83.36467719049633404160282170 436.0761547550154326735594166


WorBry - crowd_run_1080p50_y4m_x265_CRF28_KeyInt100.mp4 - seekmode=1 - GMSD/SSIM

stop 124.0039187143357256870590535 227.9924869038146211797091216
stop 83.19534706841788751496835637 435.9938219159915119282899813
stop 120.1705765900797210909978223 248.1439760410638495358170989
stop 122.2412107559194712758099402 236.4289261580102229909350346
stop 122.2412107559194712758099402 236.4289261580102229909350346


matches my previously reported behavior...

Last edited by Iron_Mike; 21st March 2019 at 02:27.
Iron_Mike is offline   Reply With Quote
Old 21st March 2019, 02:26   #215  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
Quote:
Originally Posted by ChaosKing View Post
And I tested the mp4 with the old ffms2 and it has no seeking problems. https://github.com/FFMS/ffms2/releas...2.23.1-msvc.7z
As I posted before, this old ffms2 version crashes for me w/ your Fatpack... I simply copied the x64 plug into the Fatpack's VS plugins folder... anything else I need to do ?
Iron_Mike is offline   Reply With Quote
Old 21st March 2019, 03:10   #216  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,197
Quote:
Originally Posted by Iron_Mike View Post
The only reason why I even bothered to write here is b/c you spend some time doing some tests and if you ever are subject to this behavior - or other inconsistent behavior we've not yet discovered - then your posted results here would be pointless.
Which clearly is not the case - unlike your comparison of medium vs fast presets which, if the attachment ever gets approved, must be viewed as suspect in the light of your unresolved issues. How can you post the results of study knowing that your test system is unreliable? Very unscientific ?

Quote:
Originally Posted by Iron_Mike View Post
ran a couple of tests to see how preset behave for x265...

here I tested medium vs. fast... same range as WorBry CRF 0-30, used the same graph layout

OK, are we done ?
__________________
Nostalgia's not what it used to be

Last edited by WorBry; 21st March 2019 at 03:20.
WorBry is offline   Reply With Quote
Old 21st March 2019, 03:18   #217  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,197
Quote:
Originally Posted by poisondeathray View Post
Not repeatable results here , unless seekmode=0 . I only tested your keyint 100, 6 frame thread version . Larger errors on some runs, clearly frame mismatches

Not able to replicate the 12th or whatever tiny decimal error with seekmode=0 over 10 runs . 100% consistent

Tried 2 computers, but both were 4C/8T Intels . I also have a really old AMD (with XP!) lying around but I didn't test on that

EDIT #1: also on the other file, with default settings (no custom keyint) , it was inconsistent too. Iron_Mike reported "big" inconsistent results on some runs only when using custom keyint . I'm seeing inconsistent everywhere with ffms2 default seekmode on these.
So ffms2 is unreliable on some machines. What about LWLibavSource ?
__________________
Nostalgia's not what it used to be

Last edited by WorBry; 21st March 2019 at 03:36.
WorBry is offline   Reply With Quote
Old 21st March 2019, 03:23   #218  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
Quote:
Originally Posted by WorBry View Post
Which clearly is not the case - unlike your comparison of medium vs fast presets which, if the attachment ever gets approved, must be viewed as suspect in the light of your unresolved issues. How can you post the results of study knowing that your test system is unreliable? Very unscientific ?
b/c contrary to you - who has not tested his system and only learned from me that there are potential problems - I've only run all metrics w/ seekmode=0 and confirmed each metric multiple times (consistent, repeatable) before posting them...

yes, you are done, Brian. go take a nap.
Iron_Mike is offline   Reply With Quote
Old 21st March 2019, 03:38   #219  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,197
Quote:
Originally Posted by Iron_Mike View Post
b/c contrary to you - who has not tested his system and only learned from me that there are potential problems ..
Ridiculous statement.

Quote:
Originally Posted by Iron_Mike View Post
..go take a nap.
Very juvenile, but I had that sense already.
__________________
Nostalgia's not what it used to be

Last edited by WorBry; 21st March 2019 at 03:50.
WorBry is offline   Reply With Quote
Old 21st March 2019, 03:47   #220  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
Quote:
Originally Posted by WorBry View Post
So ffms2 is unreliable on some machines. What about LWLibavSource ?
well... that took a while.

as far as I can see, the ONLY one that it has been fully "reliable" w/ seekmode=1 w/ custom keyframes is you...

I posted it, CK confirmed and poisondeathray confirmed...

As I stated pages ago, it would not surprise me if it turns out to be a package difference, and by that I mean a specific VS plugin difference, but w/o a direct, convenient method to list version/release data of all VS plugins involved in the VS chain, it'll be hard to compare...

easiest way to confirm it is to create a minimal VS venv - only what's needed for this ffms2 read test - zip it, upload it and then everybody test in that venv on their machines...

for VS, I created an isolated venv, but I used CK's Fatpack as a base, and there's too many other things included that are not needed for this minimal test...


good news is that seekmode=0 is rock solid on my end (and not slower), and Lsmash as well, so it's not a show stopper...
Iron_Mike is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 08:16.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.