BD Rebuilder Beta - Bug Reports Only

[jdobbs]

I think the bottom line for now is that I am going to have to insert the headers myself from within BD-RB, which will increase the overall job time by about 10 or 15 minutes. It's a shame, but based on the last response I saw, I have doubts that the issue is going to be addressed. Apparently no one except me is even attempting to create a UHD-BD that is standards compliant (at least with NVENCC), or the --repeat-headers issue would have been brought up long ago.

[gonca]

I should have checked the cmd line a little better
Still, 45 min encode and 15 min clean up for 4K movie isn't bad
By the way, username eddie haskell
Leave it to Beaver fan?

[Mike-uk]

how do you view if repeat headers worked ?? i tried another encoder that uses nvencc and added --repeat-headers and it started off allocating 1,2 gig of memory and stayed around there but no idea if headers are repeated

[jdobbs]

Originally Posted by gonca

I should have checked the cmd line a little better
Still, 45 min encode and 15 min clean up for 4K movie isn't bad
By the way, username eddie haskell
Leave it to Beaver fan?

I watched it after school when I was a kid. Eddie Haskell was quite a character -- I've been using that name for one thing or another for years. If you look hard enough you might even find the name "Haskell Industries" buried somewhere in the BD Rebuilder code...

[jdobbs]

Originally Posted by Mike-uk

how do you view if repeat headers worked ?? i tried another encoder that uses nvencc and added --repeat-headers and it started off allocating 1,2 gig of memory and stayed around there but no idea if headers are repeated

It's not easy without the right tools. I have an HEVC browser.

If you're good with a hex editor, you can find the delimiters between headers, slices etc by searching for the start code delimiter (00 00 00 01) -- it is prohibited from appearing anywhere else in the elementary stream. But even if you find the delimiters you'll need to reference the H265 standard (T-REC-H.265-201802-I) to find the format of all the segments that might follow a delimeter (see page 68). The standard is available for free download as a PDF.

The best way to see if the headers are there is to use TSMUXER to create a UHD-BD structure and play it back on a UHD compatible BD player. If FF/REW works... it's probably safe to assume the headers are repeated.

Just out of curiosity... what is the other encoder? It would likely be random luck if it would use the same command line parameter. I think NVENCC uses it because I suggested it (so it would match the parameter in X265).

[GreasyGreedo]

Hello,

I tried searching to see if this problem has been reported otherwise, but didn't see it.

When I author a UHD disc with multiple movie files (quick reauthored import), the subtitles are lost from the final disc. I have not had this problem with HD discs. Is there some difference that prevents the subtitles from being present on a UHD disc of this type?

[cartman0208]

Originally Posted by jdobbs

[Edit] Yeah. I just set the GOP length to 240 frames and it is barely growing at all... but, of course, that encode would fail to meet the UHD-BD standard. I would assume that it would also mean that as you attempt to fast forward, each picture step would jump 10 seconds instead of 1. The table used for these actions only logs IDR positions.

Ok, so that's why I saw blocky artifacts for several seconds when jumping to random positions in the SW Player...
So I started a new run with gop-len 24 and the memory increase is actually higher ... started out at around 600 and went up to 1500MB after 180000 frames.
Maybe with a 3hr movie file the 2GB mark would have cracked ... maybe...

Then I tried your commandline and an 1080p video ... stable at around 300MB, no increase at all
The only difference ... I didn't use the keyfile-parameter

.\NVEncC.exe --avhw -i U:\Temp\Spectre_temp\Spectre.h264 --codec h264 --preset quality --bluray --qp-min 0 --vbr 22429 --sar 1:1 --aud --pic-struct --vbv-bufsize 30000 --max-bitrate 35000 --gop-len 24 -o U:\Temp\Spectre_temp\Spectre.264 --repeat-headers

--------------------------------------------------------------------------------
U:\Temp\Spectre_temp\Spectre.264
--------------------------------------------------------------------------------
NVEncC (x86) 5.08 (r1585) by rigaya, Jul  1 2020 14:48:28 (VC 1900/Win/avx2)
OS Version     Windows 10 x64 (18363)
CPU            Intel Core i7-6800K @ 3.40GHz [TB: 3.80GHz] (6C/12T)
GPU            #0: GeForce GTX 1660 Ti (1536 cores, 1770 MHz)[2147483.64]
NVENC / CUDA   NVENC API 9.1, CUDA 11.0, schedule mode: auto
Input Buffers  CUDA, 20 frames
Input Info     avcuvid: H.264/AVC, 1920x1080, 24000/1001 fps
Vpp Filters    copyDtoD
Output Info    H.264/AVC high @ Level auto
               1920x1080p 1:1 23.976fps (24000/1001fps)
Encoder Preset quality
Rate Control   VBR
Bitrate        22429 kbps (Max: 35000 kbps)
Target Quality auto
Initial QP     I:20  P:23  B:25
QP range       I:0-51  P:0-51  B:0-51
VBV buf size   30000 kbit
Lookahead      off
GOP length     24 frames
B frames       3 frames [ref mode: disabled]
Ref frames     3 frames, MultiRef L0:auto L1:auto
AQ             off
Others         mv:auto cabac deblock adapt-transform:auto bdirect:auto aud pic-struct repeat-headers

encoded 213072 frames, 412.48 fps, 21838.53 kbps, 23135.71 MB
encode time 0:08:36, CPU: 9.5%, GPU: 7.1%, VE: 96.2%, VD: 78.7%
frame type IDR   8878
frame type I     8878,  total size   1702.42 MB
frame type P    53268,  total size   8475.27 MB
frame type B   150926,  total size  12958.02 MB

Apparently I cannot reproduce the error ... only a memory increase

[jdobbs]

@cartman0208

I removed the keyfile parameter and the memory leak persists for me. Not sure how my test differs from yours. As for the encode failure... I think the problem would also depend on the source you are using. So it will happen, it's just a matter of time. You actually need the keyfile to make sure there is an IDR frame at each chapter point.

But... I have good news. If you change the output format from an elementary stream (.hevc) to a container (e.g. .m2ts)... the problem seems to go away. That provides a good workaround that won't require BD-RB to have to rewrite the entire HEVC stream! I should be able to use the .m2ts as an input file for the TSMUXER muxing. It'll require some code changes -- but I don't think it should be anything too extreme.

My thanks to jkiku101 for experimenting with encodes and finding this potential workaround!

[jdobbs]

Originally Posted by GreasyGreedo

Hello,

I tried searching to see if this problem has been reported otherwise, but didn't see it.

When I author a UHD disc with multiple movie files (quick reauthored import), the subtitles are lost from the final disc. I have not had this problem with HD discs. Is there some difference that prevents the subtitles from being present on a UHD disc of this type?

I can't think of a difference. But I'll look and see.

[cartman0208]

Originally Posted by jdobbs

@cartman0208

I removed the keyfile parameter and the memory leak persists for me. Not sure how my test differs from yours. As for the encode failure... I think the problem would also depend on the source you are using. So it will happen, it's just a matter of time. You actually need the keyfile to make sure there is an IDR frame at each chapter point.

But... I have good news. If you change the output format from an elementary stream (.hevc) to a container (e.g. .m2ts)... the problem seems to go away. That provides a good workaround that won't require BD-RB to have to rewrite the entire HEVC stream! I should be able to use the .m2ts as an input file for the TSMUXER muxing. It'll require some code changes -- but I don't think it should be anything too extreme.

My thanks to jkiku101 for experimenting with encodes and finding this potential workaround!

Ok, then maybe it's really my 1909 Win10 Build for which I did not receive an update yet ... and 2004 is messed up with memory leak ...

Doesn't matter ... good to have a solution for this ... now to find the best settings for awesome quality

[Mike-uk]

Originally Posted by cartman0208

Doesn't matter ... good to have a solution for this ... now to find the best settings for awesome quality

yes being hardware encoding and to contend with x265/264 we need to get the best possible settings dialed in, but will be nice to have the option of considerably faster encodes but slightly less quality with HW, or still the SW/cpu at 16Hrs for absolute best :/

[jdobbs]

Originally Posted by Mike-uk

yes being hardware encoding and to contend with x265/264 we need to get the best possible settings dialed in, but will be nice to have the option of considerably faster encodes but slightly less quality with HW, or still the SW/cpu at 16Hrs for absolute best :/

Yep. It's all about choice.

[gonca]

NVEncC 5.09 is out
Supposed to fix memory leak

[Mike-uk]

Originally Posted by gonca

NVEncC 5.09 is out
Supposed to fix memory leak

darn you beat me too it

[gonca]

Presently testing 5.09, looking promising

[gonca]

Completed fine but have to repeat test
Forgot --repeat-headers in cmd line

[gonca]

cmd line

nvencc64.exe --codec hevc --preset quality --cqp 22 --profile main10 --master-display G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(40000000,50) --max-cll 902,182 --chromaloc 2 --colormatrix bt2020nc --colorprim bt2020 --transfer smpte2084 --gop-len 24 --lookahead 32 --sar 1:1 --repeat-headers

tested only on software player
no memory leak during encode
appears ok but requires more testing

[gonca]

Burned to disc and played on stand alone player (Sony)
Seems fine

[Mike-uk]

Originally Posted by gonca

Burned to disc and played on stand alone player (Sony)
Seems fine

nice ,

[Mike-uk]

Originally Posted by gonca

cmd line

tested only on software player
no memory leak during encode
appears ok but requires more testing

yep can confirm it seems fixed stayed arounf 700 Mb, not tested on a disc as not got any RWs

--avhw --vpp-pad 0,276,0,278 -i F:\Movies\uhd2019\BDMV\STREAM\00014.m2ts --codec hevc --preset quality --profile main10 --output-depth 10 --repeat-headers --chromaloc 2 --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --master-display G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1) --qp-min 0 --vbrhq 0 --vbr-quality 25 --sar 1:1 --aud --pic-struct --vbv-bufsize 45000 --max-bitrate 48000 --gop-len 24 --slices 4 -o "D:\test\VID_00000.hevc

RTX 2070 Super

encoded 175243 frames, 62.57 fps, 12390.37 kbps, 10795.88 MB
encode time 0:46:40, CPU: 7.2%, GPU: 6.3%, VE: 99.2%, VD: 18.1%
frame type IDR 7302
frame type I 7302, total size 1246.97 MB
frame type P 43811, total size 4712.77 MB
frame type B 124130, total size 4836.14 MB