Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 Encoder GUIs

Reply
 
Thread Tools Search this Thread Display Modes
Old 24th May 2015, 03:56   #4481  |  Link
kuchikirukia
Registered User
 
Join Date: Oct 2014
Posts: 476
E: never mind, I see you were talking about FFMS threads.
What the heck speed are you encoding at where FFMS decoding would be the bottleneck?

FFMS threads=1, x264 8 bit superfast crf=19, 720x368:
encoded 2067 frames, 182.71 fps, 2983.32 kb/s

Threads=4
encoded 2067 frames, 234.97 fps, 2983.32 kb/s

Last edited by kuchikirukia; 24th May 2015 at 04:37.
kuchikirukia is offline   Reply With Quote
Old 24th May 2015, 08:02   #4482  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,829
Quote:
Originally Posted by stax76 View Post
Should be very easy to reproduce, it happened to many people including myself, I reported it in the ffms2 thread.
I ran four test encodes, 20,000 frames, or about 14 minutes of video. Source 1080p. Output 704x396, default x264 settings. The source wasn't particularly high bitrate for 1080p. Something like 5500kbps. It's just a file I had handy. I used my ageing Q9450 PC.

Frame rate conversion + threads=1:
8 minutes, 4 seconds, 41.40fps, 20000 frames.

Frame rate conversion only:
4 minutes, 54 seconds, 68.23fps, 20000 frames.

Threads=1 only:
8 minutes, 41.75fps, 20000 frames

No frame rate conversion, no threads=1:
4 minutes, 51 seconds, 68.55fps, 20000 frames.

I took the above from MeGUI's log file. I'm going to try again with the previous version of ffms2 to see if the number of threads makes a similar difference, but at the moment it's changed my perspective a little on using threads=1 for HD sources. At least when I'm not using slow filtering.

"Unfortunately" I didn't duplicate the audio sync issue. I encoded the above by adding Trim(10000,29999) to the end of the script. The output was identical in size each time down the the number of bytes. I even cut out the matching section of audio and muxed it with each encode in order to check for audio sync issues, but they're weren't any.

Maybe it's frame rate dependant? My source was h264 in an MKV with a frame rate of 24fps.

Edit: I tried again using the previous ffms2 version and the encoding times above barely changed, so at least the threads=1 speed difference is only due to using less threads for decoding and not a bug.

Last edited by hello_hello; 24th May 2015 at 08:44.
hello_hello is offline   Reply With Quote
Old 24th May 2015, 08:39   #4483  |  Link
Zathor
Registered User
 
Join Date: Nov 2009
Posts: 2,405
Quote:
Originally Posted by kalehrl View Post
It works as expected when using 2pass xvid encoding.
If you use 1pass encoding - CQ, the credits are not set using 'Credit' button in 'video preview'.
I looked at xvid_encraw encoding options to make sure different credits quantiser can be used in CQ mode and it can.
It doesn't have anything to do with using a custom command line but with 2pass or 1pass encoding.
Yes, that is correct. When using CQ mode the zones are not applied. It is that way at least since 2009. Not sure why... also the zones command line does not match with the help. I will have a look.
Zathor is offline   Reply With Quote
Old 24th May 2015, 08:55   #4484  |  Link
Zathor
Registered User
 
Join Date: Nov 2009
Posts: 2,405
Quote:
Originally Posted by hello_hello View Post
I took the above from MeGUI's log file. I'm going to try again with the previous version of ffms2 to see if the number of threads makes a similar difference, but at the moment it's changed my perspective a little on using threads=1 for HD sources. At least when I'm not using slow filtering.
With more than 1 thread there may be also access violations when quickly seeking/jumping in the avs (e.g. during automatic deinterlacing detection of MeGUI). This was the main reason why I have made threads=1 the default. Not sure if this still applies to 2.21
Zathor is offline   Reply With Quote
Old 24th May 2015, 09:19   #4485  |  Link
mini-moose
Registered User
 
Join Date: Oct 2007
Posts: 385
Quote:
Originally Posted by hello_hello View Post
That sounds bad. Did the out of sync encode have the same number of frames as the source?
I'll try a few test encodes without threads=1 later on today to see what happens, but what sort of video were you encoding (resolution and codec)? Was that with or without the frame rate conversion and do you know if that also makes a difference?
What I used is: bluray 1920x1080 (video demuxed to mkv with eac3to and indexed with ffms2), resized to 720x404. CRF 25, Preset fast.

Encode 1 with:
Code:
LoadPlugin("C:\megui\tools\ffms\ffms2.dll")
FFVideoSource("D:\video.mkv", fpsnum=24000, fpsden=1001, threads=1)
Spline36Resize(720,404)
Encode 2 with:
Code:
LoadPlugin("C:\megui\tools\ffms\ffms2.dll")
FFVideoSource("D:\video.mkv")
Spline36Resize(720,404)
(also tried with fpsnum=24000, fpsden=1001, threads=0 and fpsnum=24000, fpsden=1001, threads=16, and results were identical/similar to Encode 2 in speed and sync).

Did those with ffms 2.2.0 and ffms 2.2.1.

Encode done with previous gen i7 6-core. with threads=1 I got around 40fps, without about 160fps.

Encode 1 on both ffms version - in sync
Encode 2 on both ffms version - in synch with 2.2.0, async with 2.2.1.

I checked the frame number on the 2.2.1 one, they are the exact same. However, on the t=1 encode first visible video frame is at frame 51, on t=0 it's at frame 62. Same 11 frames difference for the very last visible frame on the video.

I did the encode itself with batch file but the indexing and avs files created with megui. Was just faster for me to do several test this way.

Honestly, this may be more a topic for FFMS thread, for some reason I didn't find one that seemed active, but I'll look again. But it's relevant for megui users as well.

edit: I found the thread stax76 and I see reports on similar issues with ffms 2.2.1. Dev seem to be already doing RC versions to fix issues.

Last edited by mini-moose; 24th May 2015 at 09:39.
mini-moose is offline   Reply With Quote
Old 24th May 2015, 09:42   #4486  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,829
I looked at the bug report in the ffms2 thread, and as a result I tried to duplicate it again, this time by encoding the video from the very beginning.

This time I was successful. Too successful.
The video in question has a bunch of black frames before a frame with a picture so it was easy to step through the frames using MPC-HC and count them until I hit picture.

Source video, 25 frames
New ffms2, threads=1, 27 frames (I encoded and checked that twice)
New ffms2, no threads specified, 30 frames
L-Smash, no threads or frame conversion, 25 frames,
Old ffms2, threads=1, 25 frames
Old ffms2, no threads or frame rate conversion, 25 frames

%@$%!!!!!!!
Can I be allowed to say that again? %@$%!!!!!!!
How many ffms2 encodes have I made since the last update. One more time. %@$%!!!!!!!

Now I'm preying the problem only effects h264 sources because so far most of my encodes since the last update haven't been h264 sources, I don't think. %@$%!!!!!!!

Oh well..... off to check them all, after a bit of a lie down.

Can we make L-Smash the preferred decoder?

Last edited by hello_hello; 24th May 2015 at 09:44.
hello_hello is offline   Reply With Quote
Old 24th May 2015, 10:36   #4487  |  Link
mini-moose
Registered User
 
Join Date: Oct 2007
Posts: 385
Quote:
Originally Posted by hello_hello View Post
I looked at the bug report in the ffms2 thread, and as a result I tried to duplicate it again, this time by encoding the video from the very beginning.
This time I was successful. Too successful.
The video in question has a bunch of black frames before a frame with a picture so it was easy to step through the frames using MPC-HC and count them until I hit picture.

Source video, 25 frames
New ffms2, threads=1, 27 frames (I encoded and checked that twice)
New ffms2, no threads specified, 30 frames
L-Smash, no threads or frame conversion, 25 frames,
Old ffms2, threads=1, 25 frames
Old ffms2, no threads or frame rate conversion, 25 frames
I didn't have sync issues with threads=1, but maybe I'm not sensitive enough to notice less frames async, not sure

I indexed the video I made with ffms 2.2.0 (without threads=) and the first visible video frame there is 49 (with new one it's 51 when threads=1 is used and 62 when not).

Last edited by mini-moose; 24th May 2015 at 11:07.
mini-moose is offline   Reply With Quote
Old 24th May 2015, 11:19   #4488  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,829
I'm not sure I'd necessarily notice a 2 frame async most of the time either (it's only about 40ms), but I'd prefer to know there isn't one. I've checked a few encodes of old AVI sources (that's what I've mostly been doing lately to run some filtering on them and clean them up a bit) and so far there appear not to be any extra frames at the beginning, so maybe it is only a problem with h264 sources. I haven't checked any of those re-encodes yet. That makes me feel a bit better. I've rolled back to the previous ffms2 for the time being, but I think I'll start getting into the habit of using L-Smash.

Last edited by hello_hello; 26th May 2015 at 14:37.
hello_hello is offline   Reply With Quote
Old 24th May 2015, 11:32   #4489  |  Link
mini-moose
Registered User
 
Join Date: Oct 2007
Posts: 385
Quote:
Originally Posted by hello_hello View Post
I'm not sure I'd necessarily notice a 2 frame async most of the time either (it's only about 40ms), but I'd prefer to know there isn't one.
I've rolled back to the previous ffms2 for the time being, but I think I'll start getting into the habit of using L-Smash.
Yeah, I prefer to know there isn't one as well

I've rolled back too.

Trying L-Smash now, at least on the speed side it seems to be doing a on better than ffms with threads=1.

Need to read up whether there any possible bugs/issues with using L-Smash.
mini-moose is offline   Reply With Quote
Old 24th May 2015, 13:32   #4490  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,829
Thinking about it, a 1 frame async is about 40ms, so 2 frames must be about 80ms. I don't know what I was thinking before, but 80ms is definitely heading into "noticeable audio and video de-sync" territory.
hello_hello is offline   Reply With Quote
Old 24th May 2015, 13:52   #4491  |  Link
mini-moose
Registered User
 
Join Date: Oct 2007
Posts: 385
Quote:
Originally Posted by hello_hello View Post
80ms is definitely heading into "noticeable audio and video de-sync" territory.
Indeed, it might be still in a range I might not notice too much unless I sit and watch closely (maybe not even then).

Weird thing though is that my complete encode with all possibilities are the same number of frames in total and yet there are some additional frames at the beginning. 11 on one and 13 on the other.

The speed drop is not relevant to the version though. I see from some results posted it's not just me experiencing it. However, maybe my results are more on the extreme slow side by comparison, than what others get.
Seems bumping up the threads increases the speed quite a bit, but I don't know what I should use. Say for i7 Q-Core, would it be 4,8 or 12? On my test I used what x264 shows with threads auto which is the number of physical cores x3. Threads 0 is maybe similar to Auto function (need to read up on that)? But then Zathor said:

Quote:
Originally Posted by Zathor View Post
With more than 1 thread there may be also access violations when quickly seeking/jumping in the avs (e.g. during automatic deinterlacing detection of MeGUI). This was the main reason why I have made threads=1 the default. Not sure if this still applies to 2.21
from User Manual:
Quote:
int threads = -1
The number of decoding threads to request from libavcodec. Setting it to less than or equal to zero means it defaults to the number of logical CPU's reported by Windows. Note that this setting might be completely ignored by libavcodec under a number of conditions; most commonly because a lot of decoders actually do not support multithreading.

Last edited by mini-moose; 24th May 2015 at 14:28.
mini-moose is offline   Reply With Quote
Old 24th May 2015, 20:16   #4492  |  Link
kalehrl
Registered User
 
Join Date: Feb 2011
Posts: 331
Quote:
Originally Posted by Zathor View Post
Yes, that is correct. When using CQ mode the zones are not applied. It is that way at least since 2009. Not sure why... also the zones command line does not match with the help. I will have a look.
I added
Code:
-zw 0 1 -zq 2860 14
in the custom command line and encoded just fine using CQ. So, the syntax is the same for both CQ and 2pass xvid encoding but for some reason it is not added in CQ mode.
kalehrl is offline   Reply With Quote
Old 24th May 2015, 20:31   #4493  |  Link
Zathor
Registered User
 
Join Date: Nov 2009
Posts: 2,405
Yes, as written it will not be applied in CQ mode - thx to your test with the custom coommand line I know that it is (now?) supported.
EDIT: Will be changed in the next release
Zathor is offline   Reply With Quote
Old 25th May 2015, 00:07   #4494  |  Link
Zathor
Registered User
 
Join Date: Nov 2009
Posts: 2,405
Code:
2547 [XviD Encoder]         adjusted settings for recent Xvid (requires Xvid 1.3.x)
2546 [FFmpeg Muxer]         added FFmpeg muxer to support ASP elementary streams > 2GB
     [XviD Encoder]         MKV and AVI output will always be muxed with FFmpeg
2545 [Settings]             merged "Always mux mkv encodings" with "use external muxer"
After nearly 6 years finally an update to Xvid. Please test!

Last edited by Zathor; 25th May 2015 at 00:10.
Zathor is offline   Reply With Quote
Old 25th May 2015, 14:19   #4495  |  Link
Zathor
Registered User
 
Join Date: Nov 2009
Posts: 2,405
Code:
2551 [DGAVCIndex Indexer]   removed DGAVCIndex
     [DGIndexIM Indexer]    added DGIndexIM. it has to be enabled in the settings and
                            the license.txt file has to be placed manually in the directory /tools/dgindexim
     [Indexer]              new default order: DGIndexNV, DGIndexIM, DGIndex, L-SMASH, FFMS
2550 [Aften Encoder]        removed aften encoder. please switch to FFmpeg AC-3.
2549 [XviD Encoder]         added additional log information
2548 [Muxer]                fixed wrong fps value if fps is changed in AVS. Bug #799
Zathor is offline   Reply With Quote
Old 25th May 2015, 14:59   #4496  |  Link
kalehrl
Registered User
 
Join Date: Feb 2011
Posts: 331
There are no updates available.
kalehrl is offline   Reply With Quote
Old 25th May 2015, 15:16   #4497  |  Link
Zathor
Registered User
 
Join Date: Nov 2009
Posts: 2,405
There are. Your client will only check every 24 hours.
Zathor is offline   Reply With Quote
Old 25th May 2015, 19:33   #4498  |  Link
kalehrl
Registered User
 
Join Date: Feb 2011
Posts: 331
Just updated successfully.
There is a bug with HVS masking selection.
There is no difference in the commandline when choosing Lumi or Variance.
In both cases it shows -masking 1 whereas for Variance it should be -masking 2.
kalehrl is offline   Reply With Quote
Old 25th May 2015, 20:22   #4499  |  Link
Zathor
Registered User
 
Join Date: Nov 2009
Posts: 2,405
Thanks & confirmed. Will be fixed in the next release.
Too many bigger changes these days - and too many bug fix releases
Zathor is offline   Reply With Quote
Old 25th May 2015, 20:34   #4500  |  Link
Zathor
Registered User
 
Join Date: Nov 2009
Posts: 2,405
And some benchmarks measured with AVSMeter:

Machine 1:
1: DGIndexNV (377 fps)
2: L-SMASH (146 fps)
3: FFMS2 (125 fps, threads=0)
4: FFMS2 (40 fps, threads=1)
5: DGIndexIM (30 fps, SW engine)

Machine 2:
1: DGIndexIM (273 fps, HW engine)
2: L-SMASH (229 fps)
3: FFMS2 (191 fps, threads=0)
4: FFMS2 (55 fps, threads=1)
5: DGIndexIM (43 fps, SW engine)

Machine & source specs does not matter so much. More important is the order of the indexer plugins. Sadly I do not have currently a machine wich is capable of DGIndexNV + DGIndexIM HW. But based on the numbers I guess that DGIndexNV is on the top.
Zathor is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:12.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.