eac3to - audio conversion tool

[Atak_Snajpera]

My bench works in queue 1.

Copying is done by simple windows function CopyFileEX with COPY_FILE_NO_BUFFERING flag. (https://msdn.microsoft.com/en-us/lib...=vs.85%29.aspx)

If think that eac3to also works in queue 1 hence big size of data is needed to saturate all channels in SSD's controller.

[Q-the-STORM]

Originally Posted by madshi

Imagine 100 tracks with a chunk size of 16MB. That would already bring us to 1.6GB, and a win32 process is only allowed to allocate 2GB. So increasing the output chunk size can quickly result in out-of-memory situations, if I'm not careful.

couldn't you simply give the user the option to use a smaller chunk size? 16MB as default, and flags -1MB -2MB -4MB -8MB for smaller sizes...
or is implementing different chunk sizes a lot of work? 16MB default and an optional -1MB flag would probably do just as well then... or 16 default and 8MB, I don't think there will be many sources with more than 200 tracks... you could also just let eac3to decide, if a source has more than 100 tracks -> use 8MB, more than 200 tracks -> use 4MB...
though I do think 16MB default and flags to let the user decide would be the best thing, especially for people that use most of their RAM in normal use and don't always want or aren't able to spare more for eac3to..

anyways, most times you got 3-20 tracks, so 16MB default would almost always be a good thing...

[r0lZ]

The buffer size could also be computed at run time, according to the number of streams and available free RAM. IMO, it's the best option, and it should ideally be available if the idea of an option with several buffer sizes is implemented, as an additional "auto" size.

[Lenmaer]

This an SSD benchmark thread now?

[madshi]

Originally Posted by arrgh

is there an Option to suppress the deletion of "duplicate" playlists? In times of "playlist obsfucation / screen pass" that might be a useful Option...

eac3to only drops duplicate playlists. That should actually help with obfuscation because the insane number of playlists is reduced a bit by this.

-------

About eac3to performance: It's a bit unfortunate that everybody is talking about chunk sizes now, without looking at anything else. I don't think the chunk size is the problem, nor the key to salvation. As I wrote earlier:

Originally Posted by madshi

There are various possible reasons for slowdown. Of course it's quite possible that some processing filters in eac3to are wasting time. Or it's possible some external software is wasting time (e.g. Haali MKV muxer). Or it's possible that the SSD or the OS is at fault. It's really hard for me to say. A couple tests you could do:

1) Try "eac3to source -check". That will just read the source and demux it, but not encode or write anything.
2) If 1) already shows slow speeds, try a simply "copy" command line command to check how fast that runs through.
3) If 1) already shows slow speeds, try different containers to see if the problem might be specific to one (or several) containers.

I've tried to increase chunk sizes here on my Samsung Pro SSD, and it didn't help performance *at all*. Same performance as before. The problem is likely to be somewhere else. One suspicion I have is that eac3to's parsing of the video track might be at fault - especially when h264 is involved. eac3to fully parses every bit of every video track, and for h264 even rewrites the stream every time, "bit by bit". This *may* be the decisive slow down factor. Or maybe not. I hope that now not everyone will talk about how to fix h264 slowdown problems in eac3to. I'm just throwing around some thoughts here. None of this is tested or proven. I don't have a lot of time atm, anyway, so if the real cause of the problem is more complex than the chunk size (which I think it probably is), I won't have time to fix that soon. But instead of discussing things I could do, please guys, first try to find out by doing analytical tests where the real bottleneck is.

[Atak_Snajpera]

Originally Posted by madshi

1) Try "eac3to source -check". That will just read the source and demux it, but not encode or write anything.
2) If 1) already shows slow speeds, try a simply "copy" command line command to check how fast that runs through.
3) If 1) already shows slow speeds, try different containers to see if the problem might be specific to one (or several) containers.

AVC.mkv

General
Unique ID                                : 245455972442929439196667259541358469211 (0xB8A919C9CFCD744DB899E6E03D99605B)
Complete name                            : C:\Users\Dave\Desktop\Drive-AVC-remux.mkv
Format                                   : Matroska
Format version                           : Version 4 / Version 2
File size                                : 2.44 GiB
Duration                                 : 10mn 15s
Overall bit rate mode                    : Variable
Overall bit rate                         : 34.0 Mbps
Encoded date                             : UTC 2015-03-14 16:43:32
Writing application                      : mkvmerge v7.2.0 ('On Every Street') 32bit built on Sep 13 2014 15:42:11
Writing library                          : libebml v1.3.0 + libmatroska v1.4.1
DURATION                                 : 00:10:15.031000000
NUMBER_OF_FRAMES                         : 14746
NUMBER_OF_BYTES                          : 2617507214
_STATISTICS_WRITING_APP                  : mkvmerge v7.2.0 ('On Every Street') 32bit built on Sep 13 2014 15:42:11
_STATISTICS_WRITING_DATE_UTC             : 2015-03-14 16:43:32
_STATISTICS_TAGS                         : BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4.1
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 4 frames
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 10mn 15s
Bit rate mode                            : Variable
Bit rate                                 : 33.4 Mbps
Maximum bit rate                         : 37.0 Mbps
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.671
Stream size                              : 2.39 GiB (98%)
Default                                  : Yes
Forced                                   : No

VC-1.mkv

General
Unique ID                                : 178944820121177913025717090745631236997 (0x869F84CC91034C4D9ED0E1E12775E385)
Complete name                            : C:\Users\Dave\Desktop\Last_Man_Standing_VC1_10Min-remux.mkv
Format                                   : Matroska
Format version                           : Version 4 / Version 2
File size                                : 1.70 GiB
Duration                                 : 10mn 0s
Overall bit rate                         : 24.3 Mbps
Encoded date                             : UTC 2015-03-14 16:33:38
Writing application                      : mkvmerge v7.2.0 ('On Every Street') 32bit built on Sep 13 2014 15:42:11
Writing library                          : Lavf52.110.0

Video
ID                                       : 1
Format                                   : VC-1
Format profile                           : Advanced@L3
Codec ID                                 : V_MS/VFW/FOURCC / WVC1
Codec ID/Hint                            : Microsoft
Duration                                 : 10mn 0s
Bit rate                                 : 23.8 Mbps
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Compression mode                         : Lossy
Bits/(Pixel*Frame)                       : 0.479
Stream size                              : 1.66 GiB (98%)
Default                                  : Yes
Forced                                   : No
DURATION                                 : 00:10:00.058000000
NUMBER_OF_FRAMES                         : 14387
NUMBER_OF_BYTES                          : 1822959885
_STATISTICS_WRITING_APP                  : mkvmerge v7.2.0 ('On Every Street') 32bit built on Sep 13 2014 15:42:11
_STATISTICS_WRITING_DATE_UTC             : 2015-03-14 16:33:38
_STATISTICS_TAGS                         : BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES

eac3to -check

[madshi]

Ok, thanks, and maybe something without any video tracks?

[Atak_Snajpera]

What do you mean? Audio only?

[trevaaar]

I have a file that eac3to erroneously detects as being HDCD-encoded. It's a 24-bit 5.1-channel FLAC (though most samples are zero-padded 16-bit). The problem with this is that HDCD decoding appears to be force-enabled when converting to a lossy format, so attempts to transcode to AC3 always fail with the error "there is no HDCD data to decode" regardless of whether I pass in -decodeHdcd or not.

First few seconds of the file that exhibits the problem: https://mega.co.nz/#!59kX0ASb!MwwBiF...MSAiEs26I1rBHk

[madshi]

Originally Posted by Atak_Snajpera

What do you mean? Audio only?

Yes, some file with a similar file length (e.g. some DTS-HD or TrueHD track muxed to MKV or so) without a video track. Just to check if video stream parsing might be responsible for the slowdown.

Originally Posted by trevaaar

I have a file that eac3to erroneously detects as being HDCD-encoded. It's a 24-bit 5.1-channel FLAC (though most samples are zero-padded 16-bit). The problem with this is that HDCD decoding appears to be force-enabled when converting to a lossy format, so attempts to transcode to AC3 always fail with the error "there is no HDCD data to decode" regardless of whether I pass in -decodeHdcd or not.

First few seconds of the file that exhibits the problem: https://mega.co.nz/#!59kX0ASb!MwwBiF...MSAiEs26I1rBHk

http://eac3to.bugs.madshi.net

[Thunderbolt8]

what to think of this? http://forum.timramich.com/viewtopic.php?f=16&t=26

[madshi]

It's known that the DTS encoding suite can decode 7.1 channels losslessly. But that's a copy protected software and not easy to remote control from eac3to. So it's not really a good option for eac3to. There might be an open source decoder available soon, though, which might solve this problem. Or maybe not, we'll have to wait and see...

[tebasuna51]

Originally Posted by Thunderbolt8

what to think of this? http://forum.timramich.com/viewtopic.php?f=16&t=26

To decode "7.1 - L, R, C, LFE, Ls, Rs, Lsr, Rsr" (strange setup) losslessly, like if was "7.1 - L, R, C, LFE, Lss, Rss, Lsr, Rsr", you can use MakeMKV, the dtsdecoderdll.dll from ArcSoft and a profile like the attached.

[nevcairiel]

Originally Posted by madshi

It's known that the DTS encoding suite can decode 7.1 channels losslessly. But that's a copy protected software and not easy to remote control from eac3to. So it's not really a good option for eac3to. There might be an open source decoder available soon, though, which might solve this problem. Or maybe not, we'll have to wait and see...

The good news is that the "strange setup" can already be decoded completely lossless with this potential decoder, well, at least the sample I had.

Its fun to see what other "reference" decoders do with strange-setup though. Somehow they postprocess the channels to make a "normal" surround setup out of them, because they assume any software couldn't possibly comprehend the extra front-surround channels.

[madshi]

Originally Posted by nevcairiel

The good news is that the "strange setup" can already be decoded completely lossless with this potential decoder, well, at least the sample I had.

Great!

Originally Posted by nevcairiel

Its fun to see what other "reference" decoders do with strange-setup though. Somehow they postprocess the channels to make a "normal" surround setup out of them, because they assume any software couldn't possibly comprehend the extra front-surround channels.

Yeah. And some ArcSoft versions even produced corrupted sound data...

[Nebudchanezzer]

Originally Posted by tebasuna51

To decode "7.1 - L, R, C, LFE, Ls, Rs, Lsr, Rsr" (strange setup) losslessly, like if was "7.1 - L, R, C, LFE, Lss, Rss, Lsr, Rsr", you can use MakeMKV, the dtsdecoderdll.dll from ArcSoft and a profile like the attached.

Didn't know that, very interesting!

Did a test with a small track I have encoded with MA-suite to both
"normal" setup and "strange setup", seems to work great except there seems to be one extra DTS-frame at the end of the "strange setup"-track.

Don't know if that is something added by the MA-suite or if MakeMKV add an extra though.

Thank you for the info anyway!

Looking forward to the new codec!

EDIT:From Foobar:

Differences found in compared tracks.

Comparing:
"N:\Decode.test.reencoded.to.standard.setup.flac"
"N:\title00_track2_eng_DELAY 0ms.flac"
Length mismatch : 4:18.154667 vs 4:18.165333, 12391424 vs 12391936 samples.
Compared 12391424 samples, discarded last 512 samples from the longer file.
No differences in decoded data found within the compared range.

[arrgh]

...just a tiny small remark, not really very critical :

for me it is inconvinient that "-progressnumbers" gives a line feed after each number; I would prefer, if the Output would remain in the same line (CR without LF)...

[Atak_Snajpera]

progressnumber switch was added for gui makers. It is easier to capture output if you have new line.

[r0lZ]

It is impossible to change the --progressnumbers behaviour. It is used by many GUIs, and even a very small change would break them.
If you want only a single line of output, do not specify the --progressnumbers argument, and watch the progress bar.

[madshi]

Btw, does the GUI error message capturing work now with the latest eac3to?