BdRebuilder changing UHD from BT.2020 to BT.709?

[kpic]

Hello, using BDRebuilder 0.61.22, for full backup of a UHD title. It seems to be converting the metadata information for a few titles on the UHD disc (the main movie and several trailers that are in 4K).

The titles in question are 4K SDR but use the BT.2020 color space (apparently seems to be a rare situation as most SDR use BT.709?) anyway BDRebuilder correctly identifies it as "UHD" but then somewhere along it converts those BT.2020 titles over to BT.709, the result is that of course those titles now have a washed out\muted color look to them that is way off from the originals. I've never had a problem using BdRebuilder usng 'True HDR' UHD discs but this is the first time encountering this issue.

So here are the Mediainfo for the titles that used BT.2020;

Video
ID : 4113 (0x1011)
Menu ID : 1 (0x1)
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L5.1@High
Codec ID : 36
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0 (Type 2)
Bit depth : 10 bits
Color range : Limited
Color primaries : BT.2020
Transfer characteristics : BT.2020 (10-bit)
Matrix coefficients : BT.2020 non-constant

And here is the info for the BDRebuilder finished version (main movie but the trailers converted the same way);

Video
ID : 4113 (0x1011)
Menu ID : 1 (0x1)
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L5.1@High
Codec ID : 36
Duration : 459 ms
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0 (Type 2)
Bit depth : 10 bits
Color range : Limited
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

So am I missing a setting somewhere? I suppose I thought BdRebuilder would just pass the correct metadata through but apparently it switched it somewhere.

Here is my BdRebuilder Ini;

[Options]
VERSION=0.61.0.22
ENCODER=0
MODE=0
ENCODE_QUALITY=0
ONEPASS_ENCODING=2
AUTO_QUALITY=1
TARGET_SIZE=40500
AUDIO_TO_KEEP=all
SUBS_TO_KEEP=all
SD_CONVERT=0
OPEN_GOP=0
RESIZE_1080=0
RESIZE_1440=0
RESIZE_720=0
DEINTERLACE=0
SD_TO_1080=0
IGNORE_3D=0
CONVERT_WIDE=0
DTS_REENCODE=0
AC3_REENCODE=0
AC3_640=1
AC3_192=0
KEEP_HD_AUDIO=1
AUDIO_DRC=0
DECODER=0
AVCHD=1
REMOVE_WORKFILES=0
REMOVE_OUTPUT=0
USE_FILTERS=0
BDMV_CERT_ONLY=0
IVTC_PULLDOWN=0
ASSUME_DVD_PAL=0
FRIMSOURCE=0
COMPLETION_BEEP=0
OUTPUT_SBS=0
NEROAAC=0
SUPTITLE=0
PGSTOSRT=0
AUDIO_TRACK_LIMIT=0
SUBTITLE_TRACK_LIMIT=0
CUSTOM_TARGET_SIZE=40500
NVENC_CAPABLE=1
QUICK_CRF=28
UHD_MAXRATE=37500
QUICK_EXTRAS=1
[Paths]

Thank you for any help!

[Emulgator]

The most recent version is 0.61.29 ATM, maybe you should update and try again.

[kpic]

Ok, used 0.61.29 and same outcome.

Here is the log;

[11/01/23] Checking System Settings
- BD-Rebuilder v0.61.29
- Windows Version: 6.2 [9200]
- Working Path Free Space: 304.27GB
- AVISYNTH Version: 2.6.0.6, Ok
- LAVFILTERS: Ok
- AnyDVD settings check: Ok.
- X264: Ok
- X265: Ok
- AFTEN: Ok
- FAAC: Ok
- MP4BOX: Ok
- WAVI: Ok
- TSMUXER: Ok
- FRIMEncode: Ok
- FRIMDecode: Ok
[11/01/23] Systems Settings Check complete
----------------------
[11/01/23] BD Rebuilder v0.61.29
[22:46:24] Source: BDROM
- Input BD size: 84.82 GB
- Approximate total content: [03:54:47.990]
- Target BD size: 47.36 GB
- Windows Version: 6.2 [9200]
- Auto Quality: Very Good (Very Fast), 1-Pass VBR
- UHD-BD source detected.
- Decoding/Frame serving: FFMPEG
- Audio Settings: AC3=0 DTS=0 HD=1 Kbs=640
[22:46:24] PHASE ONE, Encoding
- [22:46:24] Processing: VID_00020 (1 of 15)
- [22:46:24] Extracting A/V streams [VID_00020]
- [22:46:30] Reencoding video [VID_00020]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23.976fps, 3,996 frames
- [22:46:30] Reencoding: VID_00020, Pass 1 of 1
- [22:46:40] Video Encode complete
- [22:46:40] Processing audio tracks
- Track 4352 (jpn): Keeping original audio
- [22:46:40] Multiplexing M2TS
- [22:46:44] Processing: VID_00018 (2 of 15)
- [22:46:44] Extracting A/V streams [VID_00018]
- [22:46:48] Reencoding video [VID_00018]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23.976fps, 4,552 frames
- [22:46:48] Reencoding: VID_00018, Pass 1 of 1
- [22:47:00] Video Encode complete
- [22:47:00] Processing audio tracks
- Track 4352 (jpn): Keeping original audio
- [22:47:00] Multiplexing M2TS
- [22:47:04] Processing: VID_00022 (3 of 15)
- [22:47:04] Extracting A/V streams [VID_00022]
- [22:47:09] Reencoding video [VID_00022]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23.976fps, 6,233 frames
- [22:47:09] Reencoding: VID_00022, Pass 1 of 1
- [22:47:23] Video Encode complete
- [22:47:23] Processing audio tracks
- Track 4352 (jpn): Keeping original audio
- [22:47:23] Multiplexing M2TS
- [22:47:28] Processing: VID_00017 (4 of 15)
- [22:47:28] Extracting A/V streams [VID_00017]
- [22:47:33] Reencoding video [VID_00017]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23.976fps, 6,820 frames
- [22:47:33] Reencoding: VID_00017, Pass 1 of 1
- [22:47:49] Video Encode complete
- [22:47:49] Processing audio tracks
- Track 4352 (jpn): Keeping original audio
- [22:47:49] Multiplexing M2TS
- [22:47:53] Processing: VID_00010 (5 of 15)
- [22:47:53] Extracting A/V streams [VID_00010]
- [22:47:59] Reencoding video [VID_00010]
- Source Video: HEVC, 3840x2160
- Rate/Length: 23.976fps, 1,520 frames
- [22:47:59] Reencoding: VID_00010, Pass 1 of 1
- [22:52:13] Video Encode complete
- [22:52:13] Processing audio tracks
- Track 4352 (jpn): Keeping original audio
- [22:52:13] Multiplexing M2TS
- [22:52:17] Processing: VID_00009 (6 of 15)
- [22:52:17] Extracting A/V streams [VID_00009]
- [22:52:22] Reencoding video [VID_00009]
- Source Video: HEVC, 3840x2160
- Rate/Length: 23.976fps, 1,520 frames
- [22:52:22] Reencoding: VID_00009, Pass 1 of 1
- [22:56:22] Video Encode complete
- [22:56:22] Processing audio tracks
- Track 4352 (jpn): Keeping original audio
- [22:56:22] Multiplexing M2TS
- [22:56:27] Processing: VID_00021 (7 of 15)
- [22:56:27] Extracting A/V streams [VID_00021]
- [22:56:32] Reencoding video [VID_00021]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23.976fps, 12,202 frames
- [22:56:32] Reencoding: VID_00021, Pass 1 of 1
- [22:56:59] Video Encode complete
- [22:56:59] Processing audio tracks
- Track 4352 (jpn): Keeping original audio
- [22:56:59] Multiplexing M2TS
- [22:57:05] Processing: VID_00015 (8 of 15)
- [22:57:05] Extracting A/V streams [VID_00015]
- [22:57:10] Reencoding video [VID_00015]
- Source Video: HEVC, 3840x2160
- Rate/Length: 23.976fps, 2,242 frames
- [22:57:10] Reencoding: VID_00015, Pass 1 of 1
- [23:03:33] Video Encode complete
- [23:03:33] Processing audio tracks
- Track 4352 (jpn): Keeping original audio
- [23:03:33] Multiplexing M2TS
- [23:03:37] Processing: VID_00012 (9 of 15)
- [23:03:37] Extracting A/V streams [VID_00012]
- [23:03:43] Reencoding video [VID_00012]
- Source Video: HEVC, 3840x2160
- Rate/Length: 23.976fps, 3,114 frames
- [23:03:43] Reencoding: VID_00012, Pass 1 of 1
- [23:12:22] Video Encode complete
- [23:12:22] Processing audio tracks
- Track 4352 (jpn): Keeping original audio
- [23:12:22] Multiplexing M2TS
- [23:12:27] Processing: VID_00014 (10 of 15)
- [23:12:27] Extracting A/V streams [VID_00014]
- [23:12:33] Reencoding video [VID_00014]
- Source Video: HEVC, 3840x2160
- Rate/Length: 23.976fps, 3,114 frames
- [23:12:33] Reencoding: VID_00014, Pass 1 of 1
- [23:20:58] Video Encode complete
- [23:20:58] Processing audio tracks
- Track 4352 (jpn): Keeping original audio
- [23:20:58] Multiplexing M2TS
- [23:21:03] Processing: VID_00011 (11 of 15)
- [23:21:03] Extracting A/V streams [VID_00011]
- [23:21:09] Reencoding video [VID_00011]
- Source Video: HEVC, 3840x2160
- Rate/Length: 23.976fps, 3,114 frames
- [23:21:09] Reencoding: VID_00011, Pass 1 of 1
- [23:29:42] Video Encode complete
- [23:29:42] Processing audio tracks
- Track 4352 (jpn): Keeping original audio
- [23:29:42] Multiplexing M2TS
- [23:29:47] Processing: VID_00019 (12 of 15)
- [23:29:47] Extracting A/V streams [VID_00019]
- [23:29:56] Reencoding video [VID_00019]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23.976fps, 40,919 frames
- [23:29:56] Reencoding: VID_00019, Pass 1 of 1
- [23:31:23] Video Encode complete
- [23:31:23] Processing audio tracks
- Track 4352 (jpn): Keeping original audio
- [23:31:23] Multiplexing M2TS
- [23:31:33] Processing: VID_00013 (13 of 15)
- [23:31:33] Extracting A/V streams [VID_00013]
- [23:31:42] Reencoding video [VID_00013]
- Source Video: HEVC, 3840x2160
- Rate/Length: 23.976fps, 6,450 frames
- [23:31:42] Reencoding: VID_00013, Pass 1 of 1
- [23:48:45] Video Encode complete
- [23:48:45] Processing audio tracks
- Track 4352 (jpn): Keeping original audio
- [23:48:45] Multiplexing M2TS
- [23:48:51] Processing: VID_00016 (14 of 15)
- [23:48:51] Extracting A/V streams [VID_00016]
- [23:49:20] Reencoding video [VID_00016]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23.976fps, 106,596 frames
- Bitrate: 9,545 Kbs
- [23:49:20] Reencoding: VID_00016, Pass 1 of 1
- [23:55:09] Video Encode complete
- [23:55:09] Processing audio tracks
- Track 4352 (jpn): Keeping original audio
- [23:55:09] Multiplexing M2TS
- [23:55:28] Processing: VID_00008 (15 of 15)
- [23:55:28] Extracting A/V streams [VID_00008]
- [23:57:37] Reencoding video [VID_00008]
- Source Video: HEVC, 3840x2160
- Rate/Length: 23.976fps, 127,534 frames
- Bitrate: 36,988 Kbs
- [23:57:37] Reencoding: VID_00008, Pass 1 of 1
- [05:37:39] Video Encode complete
- [05:37:39] Processing audio tracks
- Track 4352 (jpn): Keeping original audio
- Track 4353 (jpn): Keeping original audio
- Track 4354 (jpn): Keeping original audio
- [05:37:39] Multiplexing M2TS
[05:39:25]PHASE ONE complete
[05:39:25]PHASE TWO - Rebuild Started
- [05:39:25] Rebuilding BD file Structure
[05:39:35] - Encode and Rebuild complete
[05:39:35] JOB: BDROM finished.

There is the main movie & 7 trailers with changes.

Went in as;
Color primaries : BT.2020
Transfer characteristics : BT.2020 (10-bit)
Matrix coefficients : BT.2020 non-constant

Came out as;
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

And of course playing back they look either completely washed out or completely over-exposed, none look like the originals.

So is this a bug or do I have a setting I have to enter into the .ini file?
I'm not sure I understand why BdRebuilder is even bothering to make these changes when it encodes? Is it assuming 'HDR' when a 'UHD' is input?

[kpic]

The main movie is stream is 00008 (the last entry in the log) but it also changed the other HEVC 3840x2160 streams (7 trailers).

[Emulgator]

Just running a test reencode using NVEncC on a RTX3080 (from 56GiB onto BD-R25)
I just updated today 2023 11 03: NVEncC 0.7.35 works in BD-RB 0.61.29 UHD @ 74,5fps (3,11x) on a RTX3080 (Driver 537.42)
The first stream (FBI Warnings) came in and out as BT.2020, all parameters seem correct.
Main movie encode still running.
P.S. Well, the Main encode aborted twice, but the remaining .hevc ES stub was playable, looked correct and had all the correct metadata.
A third attempt with a different movie aborted as well, all 3 were stalled after 35.000-45.000 frames.
Some other fault going on, but I am not investigating further now, back to work.

[kpic]

Emulgator, thank you for checking that! I was going to say my title is specifically SDR with the BT.2020 reference, the HDR titles I have never had an issue. But I see the first stream you did (FBI warnings) came out ok with its BT.2020 intact, I'm assuming that wouldn't be encoded HDR? So if that stream is indeed like mine (SDR with BT.2020) then for some reason my version alone changed the BT.2020 to BT.709?

I have done some further testing as well...

The first time I tried I used version .22 and that one the colors were completely washed out playing in PowerDVD (and in MPC-HC player) and Mediainfo reported the change from original BT.2020 to BT.709

The next time when I used version .29 the colors were all overblown and overexposed playing in MPC-HC and mediainfo again reported the change from original BT.2020 to BT.709, at that point I assumed it would be the same when viewing in PowerDVD so I didn't bother trying. Well when I did eventually play it in PowerDVD it looked absolutely fine (no washed out or overexposed\blown out colors) for all the HEVC\4K SDR titles (the main movie & 7 trailers)!

So version .29 did do something different from version .22?

So now I'm not sure if my Mediainfo (latest version and I used a previous version) is incorrectly reporting the colorspace for some reason, but apparently BDrebuilder does 'something' when encoding to adjust\modify this setting?

I thought perhaps one of my BDrebuilder side programs it needs to work is out of date or has a setting wrong?

I can't seem to find that original page of BDrebuilder where it would tell you the latest version and give you the instructions and versions to use of Avisynth etc. (Wasn't that here in this section at one time?) So maybe one of my other program versions is outdated\incorrect or set wrong?

[kpic]

Ok, I've done further research and BDrebuilder, I'll say for me at least is having issues with these HEVC SDR BT.2020 files, I won't say M2TS yet as I'm not sure that's where it is having an issue.

Previously I thought that the title played with the proper colors in PowerDVD, that's kind of true. If I play the title and go straight to the 4K 'main movie' it'll look completely washed out w\ very limited color as it was doing before... however if you then go to the menu and play one of the other 4K titles - one of the 7 trailers that are also in HEVC SDR BT.2020 - (it looks fine) and then go back to the main movie at any point (that was previously 'washed out') the movie will play just fine in perfect beautiful colors. Weird huh? But now I don't think it is the actual video file being encoded wrong, my BDrebuilder is doing something to the clipinfo file I think.

Mediainfo (latest version) for these M2TS does not show the correct information, not only does it show them as BT.709 instead of the previous BT.2020 it doesn't even show the audio file for them, so thinking my Mediainfo might be wonky (though its fine showing all correct info for the untouched M2TS) I needed to dig further... I instead went into one of the playlists of one of those M2TS clips and pulled the Mediainfo for it and surprise, surprise it showed it as 'HDR-10' looking over at the original playlist and no mention of that at all, so now I'm getting somewhere...

If you donate to Pel and get his great little BDedit program they (since 0.51 I think) will look at UHD titles. So using the latest version(0.56) I pulled up the BDrebuilder & original versions to compare, going to one of the HEVC SDR BT.2020 'Playlist' tab...



BDrebuilder version on left & Original on right... you'll see the time-stamps are off, not sure why it does that but I'm pretty sure I've noticed that before after BDRB with other titles and they've been ok. But everything else matches; note video is HEVC and SDR BT.2020, so both are ok there...

Here is the 'Clipinfo' tab for that file that registers to that playlist...



Ok, again with the mixed up timestamps, but more importantly... BDRB version on left shows the clip information as HEVC 'HDR-10' while the Original version on the right shows as it is supposed to be, no 'HDR-10', so for some reason BDRB added in the 'HDR-10' to the Clipinfo file for this video even though it most definitely is not HDR-10?

One final test... I sent the entire title through DVDFab UHD Copy to shrink it down to 50gb and no problem at all... All those 'problem' HEVC SDR BT.2020 tracks came back as there were supposed to be; SDR BT.2020... no overexposed or washed out colors, Mediainfo had no problem picking up that there were just as the original still SDR BT.2020... So why don't I just use DVDFab to shrink it to 50gb then? Cause it sucks, it broke one of the menu navigation buttons, it caused one of the titles to go out of sync and who knows what else that I never had an issue with BDRB.

If you're still reading at this point, not sure where to go from here... for my BDRB at least (not sure if it's a 'bug' or not until someone can confirm) it maybe?? assumes BT.2020 material cannot be SDR it has to be HDR (or at least that's the way it's acting). Unfortunately to test this you'd need a title that has SDR BT.2020 video on it. Until then either it's everyone's BDRB (though you may never encounter it), or it's one of my settings in BDRB, or it's one of my 'helper' programs being out of date or a setting in there somewhere but it'll probably take someone smarter than me to figure all that out.

In the meantime I have this title I guess I may be able to see how my stand alone player deals with it, or if acts like PowerDVD did with it. If it does perhaps I can cobble something together using BDRB for all the 'non SDR BT.2020' parts and DVDFab for those that are SDR BT.2020?

Oh and when I get done with this title I have another title sitting here just like that one, yes with SDR BT.2020 again.

EDIT: For some reason my images did not show up, I believe they need to be approved to go in if I recall, but anyway trust me that info is there, I can provide the screenshots if anyone wants to take this investigation further.

[Emulgator]

Well, SDR BT.2020. Table 3-6 allows that indeed. Maybe you found a bug worth reporting.
I will have to search if I have a disc with that combination.
BTW, I was playing back HDR SMPTE ST2086, BT.2020 in MPC-BE 1.6.10, there it is rendered ok.
(MPC-HC 2.1.0 rendered that washed out, I did not care to fix any rendering pipeline for now)

[kpic]

Just a further update to this... I wanted to take a few things out of the equation; this specific title I worked with and my hardware.

So I mentioned above the other title I have from the same company also used 'SDR BT.2020' for all of the 4K titles (the main movie & the trailers). I moved that untouched BD-ROM folder to my HP laptop (not sure of the model # but it's a 2022 and somewhat powerful but not full-on gaming laptop, so it should be fine with BDRB) - This laptop never had BDRB installed on it so I downloaded the latest version - Again I suppose that original BDRB page from a while ago is no longer here at Doom that told you what specific other program versions are needed so when I went to install BDRB on the laptop I needed Avisynth so I used the latest version (assuming this is now ok to use?).

With that I had a different title than used above (though still with SDR BT.2020) and a fresh install of BDRB along with the latest Avisynth (an inspection of tools came back OK). With that I set BDRB to shrink the original 84gb version to a full backup of 45gb w\quicker encode for the extras and set my "UHD_MAXRATE=40000" and ran it overnight.

And the final result was the same exact thing I experienced above; the BDRB version had a full functioning UHD so that was good, but all of the HEVC SDR BT.2020 titles were now converted over to BT.709 - I think that's about as much testing as I can do, I was able to remove the specific title & hardware out of the equation. Just to be sure I took this title and sent it through DVDFab and just as before it broke the chapters in the main movie (though I was able to fix it using BDedit) but it did keep the titles as SDR BT.2020.

I'm not sure if it is a specific bug in BDRB or one of those helper programs but one of them doesn't like that specific combination of SDR BT.2020, and though I suppose it would be rare to encounter a title with those specs in case anyone does and hits the same thing I did be aware of this issue.

[CV91913]

@kpic...if you go to the bug report thread and bring up page 1, the instructions are still there.

[kpic]

Originally Posted by CV91913

@kpic...if you go to the bug report thread and bring up page 1, the instructions are still there.

Thank you for that! I must have forgot where I saw it!