UHD & Blu-ray Database / KEYDB.cfg

[Relight]

Originally Posted by Snap

Would be nice if there was a user-contributable list/database for disc hashes with comprehensive version info...

This thread was originally meant for the development of a new database for Blu-ray and UHD data with the purpose being new (and improved) KEYDB.cfg output.

(Starbuck2010's database is offline and he indicated that he doesn't have time for it anymore.)

However, since FindVUK operates a database, this really isn't needed.

The following links were meant to be temporary pending the new database, circa January 22, 2018, and are no longer maintained:

Latest KEYDB.cfg for Blu-ray and UHD: http://s000.tinyupload.com/index.php...96669669839157

Latest KEYDB.cfg for UHD ONLY: http://s000.tinyupload.com/index.php...19705707482529

Latest so-called "hashed keys" (ONLY work with MakeMKV, frequently updated): http://makemkv.com/forum2/viewtopic.php?f=12&t=16959

[DavVador]

Hello,
I cannot submit keys but i got a bunch of UHD (from France) BD so i can offer UPC and SHA1
Thanks for your work !

Here are my 3 first contributions :

VOLUME_NAME | Title | Country | UPC/EAN | SHA1 | AACS VER
Passengers_2016 | Passengers | FR | 3333298300314 | b0785adfc37c757b55eea9ab801b0fc1bf26158a | AACS 2.0  MKB v60
VALERIAN CITE MILLE PLANETES | Valérian et la Cité des Milles Planètes | FR | 3700724902952 | 0a75f67a59afa4e04fdb4706cb004eeb1d51a814 | AACS VER
THE_REVENANT | The Revenant | FR | 3344428062699 | 232e860cf4d0d28296de9c99a3aecca87f904ea1 | AACS ver

Edit : I submited more on the website

And there are a few EAN on the excel file available here :
https://forum.redfox.bz/threads/uhd-...%BCssel.74136/

[nalor]

I noticed you already offer to select 'Bluray' as type when entering new data directly on your website and you've also added a field for the VUK.
Do you have plans to make your website the new central-aacs-database site?
If so it would be great if you could add support for all fields that are specified for the keydb.cfg file and a few more.

Here's an example of a MetaXML that I've created for FindVUK to upload to a 'to be developed' online database

<?xml version="1.0" encoding="UTF-8"?>
<Bluray>
<FileType>BlurayMetaXML</FileType>
<DiscId Date="2017-08-22">16ABFD83376D41E8CD26D1438FF02146059663BD</DiscId>
<VolumeId>F18B48F2BC9719E471BED63EA43BDC25</VolumeId>
<MediaKey>D2BF8DAC6A288A80B84CA5E8924D76CB</MediaKey>
<VolumeUniqueKey>E7E516328BAFAE8D8D91C28530695EE8</VolumeUniqueKey>
<VolumeLabel>War for the Planet of the Apes</VolumeLabel>
<BDplus Date="2017.09.07">1</BDplus>
<BusEncryptionEnabled>1</BusEncryptionEnabled>
<MKBrev>50</MKBrev>
<MainPlaylist>00800.mpls</MainPlaylist>
<UnitKeys>
<UnitKey Nr="1">80CA0B712E1FD110E8B9287B9F3C4FAC</UnitKey>
</UnitKeys>
<MetaTitles>
<MetaTitle Language="eng">War for the Planet of the Apes</MetaTitle>
<MetaTitle Language="fra">La Planète des Singes : Suprématie</MetaTitle>
<MetaTitle Language="spa">La guerra del planeta de los simios</MetaTitle>
<MetaTitle Language="nld">War for the Planet of the Apes</MetaTitle>
<MetaTitle Language="deu">Planet der Affen: Survival</MetaTitle>
<MetaTitle Language="ita">The War - Il Pianeta delle Scimmie</MetaTitle>
<MetaTitle Language="jpn">War for the Planet of the Apes</MetaTitle>
<MetaTitle Language="cat">La guerra del planeta de los simios</MetaTitle>
</MetaTitles>
</Bluray>

(small note - there can be multiple UnitKey entries)

Would be great if you could add all the fields to your db and finally allow to download a language-specific keydb.cfg created from the data.

(and I think it would be easy to add a 'postmeta' mode to findvuk that just grabs all easily available information from the bluray/uhd disc and submits it automatically to the onlinedb.)

[candela]

Originally Posted by nalor

(small note - there can be multiple UniqueKey entries)

they are actually called UnitKey for BD or TitleKey for HDDVD. Uniquekey can be general though but they aren't actually unique and shared by many discs

being able to upload BD+ tables to the database would also be useful (but they are ~1MB/disc on average, 5-10 GB of storage will go a long way , tables for many discs are also identical)

[nalor]

Originally Posted by candela

they are actually called UnitKey for BD or TitleKey for HDDVD. Uniquekey can be general though but they aren't actually unique and shared by many discs

oh man - you're absolutely right .... (changed the post above already)
Also corrected it immediatly in FindVUK

[BRC85sYrg]

Originally Posted by Relight

http://www.uhddb.com

I don't know what the old key database was like, since I never saw it and it's down now. But I'm more than happy to maintain and develop this. Suggestions, discussions, bug reports, etc. and most importantly submissions into the database are welcome.

Big thanks for your efforts!

The database page ran by Starbuck2010 which unfortunately is down right now looked like this:
https://web.archive.org/web/20170705...abdv.com/aacs/

manual updates were done at the following page:
https://web.archive.org/web/20170629...ydb-upload.php

I hope this is what you were looking for.

[candela]

ohter possible fields are:
- Region info (A,B,C)
- comment field for additional info

and not to discourage you but a database like this requires a proper design and serious thought in advance. For example:
- 1 DiscID can be linked to many UPC codes and have different VolumeLabels
- 1 UPC code can be linked to many countries
- people can upload fake data (prevention mechanisms needed such as detecting duplicate VUK, impossible keylengths, confirming VUK from MK/VID calculation, etc )
- how to deal with duplicate entries where the key cannot be confirmed at the moment (there are several in the old AACSupdater database)
- prevent re-upload from previously deleted entries
- correction of existing entries
- ...

[Relight]

I think all suggestions posted so far (thank you!) can be easily implemented - the only question is what is the best way to proceed.

What data is really necessary? Should the database just use FindVUK as the primary input method?

It seems there is at least some interest in what I have done so far. However, since I am not yet very knowledgable on the decryption side (for example candela mentions MK/VID calculation which I don't know), I would welcome to work in collaboration with any of the experts in the community.

Now that there are some entries as test data (thank you DavVador!), I will hold the submission form until the database is developed a little bit more, hopefully with help from anyone interested in advising on the data it should have, possible integrations, integrating previous data from KEYDB.cfg, etc.

I think it would be great to have users simply upload an XML submission from FindVUK - does that seem like a good idea? The whole UPC/Country thing really isn't required I think - I got into that because that's what others are doing with their spreadsheets - the SHA1 should be the identification method for people wondering if their particular UHD has had a VUK released yet.

I know it's more disc-related than decryption, but I like the region suggestion by candela, and I also wouldn't mind seeing info such as disc size and number of layers.

[Relight]

Updated first post with latest KEYDB.cfg files. I will keep those files updated.

I have some questions about some of the content in KEYDB.cfg, if anyone is willing to chat with me, please send me a PM - thanks!

[DavVador]

From my point of view, UPC/EAN is very interesting, it's an easy way to check if your bluray can be decrypted without the need to put the bluray in your burner or even before buying.
Region code or country is an extra even if it's less important imo.

I don't have much free time to be honest, but would be glad to help in any way.
I'm not an expert at all in crypto but i can help in the webcode (html/php) if needed
Or if you need some testing, i'm ok (I got 2 drives, 1 UHD friendly and 1 official UHD) !

And being able to post an XML generated with FindVUK would help you a lot i think if you wish to get more people submitting fields in your DB.

[ChaosKing]

You could even combine it with a simple app to scan BDs in a store.

[candela]

Data in your database does not have to be limited to data in keydb. You can build an entire Bluray movie database if you want, even IMDb codes can be useful.

Keydb specification can be found here. Even though VUK is sufficient to play a certain disc (ignoring bd+, bus encryption, playlist obfuscation and other DRM crap), I have explained here why it is a mistake to collect only VUK

Even if keydb does not have support for some data, it should still be collected as separate data fields in the database (e.g. BD+, MKBrev, Region) and can be written in the keydb comment field. Keydb should be regarded as just 1 kind of output format. There should be for example CSV or SQL output which is much easier to work with for data analysis.

For example, a FindVUK XML

<?xml version="1.0" encoding="UTF-8"?>
<Bluray>
	<FileType>BlurayMetaXML</FileType>
	<DiscId Date="2011-06-06">113B873574ABF236895BE54376AA14649DA0152E</DiscId>
	<VolumeId>A56C5A7AAAA8CBDE3D83834C836D8830</VolumeId>
	<MediaKey>DF991ABB947C503E4BC18DC09F4F5579</MediaKey>
	<VolumeUniqueKey>73C153CC28B32B650F23F8A9891331B9</VolumeUniqueKey>
	<VolumeLabel>RETURN_OF_THE_JEDI</VolumeLabel>
	<BDplus Date="2011-05-04">1</BDplus>
	<BusEncryptionEnabled>0</BusEncryptionEnabled>
	<MKBrev>19</MKBrev>
	<MainPlaylist>00800.mpls</MainPlaylist>
	<UnitKeys>
		<UnitKey Nr="1">04B80421D00B2C68134BCE7807421287 </UnitKey>
		<UnitKey Nr="2">85581552DC02679109AF39A4EA5077EB </UnitKey>
		<UnitKey Nr="3">013BE88A01E408202A1994CD0252184C </UnitKey>
		<UnitKey Nr="4">C2E9C5B3D4D68C80EBD4982C2D0B0584 </UnitKey>
		<UnitKey Nr="5">BA966BE272D3AD16EAF4F2FF2A596B38 </UnitKey>
		<UnitKey Nr="6">9571943B2E4CBA7CDADF6EA2D70141A1 </UnitKey>
		<UnitKey Nr="7">6FB81616BF05A6A554CF99B396FF260D </UnitKey>
	</UnitKeys>
	<MetaTitles>
		<MetaTitle Language="eng">Return of the Jedi</MetaTitle>
		<MetaTitle Language="fra">Le retour du Jedi</MetaTitle>
	</MetaTitles>
</Bluray>

can become a keydb.cfg entry similar to this:

0x113B873574ABF236895BE54376AA14649DA0152E = RETURN_OF_THE_JEDI (Return of the Jedi) 
| D | 2011-06-06 
| V | 0x73C153CC28B32B650F23F8A9891331B9 
| U | 1-0x04B80421D00B2C68134BCE7807421287 | 2-0x85581552DC02679109AF39A4EA5077EB | 3-0x013BE88A01E408202A1994CD0252184C 
     | 4-0xC2E9C5B3D4D68C80EBD4982C2D0B0584 | 5-0xBA966BE272D3AD16EAF4F2FF2A596B38 | 6-0x9571943B2E4CBA7CDADF6EA2D70141A1 | 7-0x6FB81616BF05A6A554CF99B396FF260D 
| M | 0xDF991ABB947C503E4BC18DC09F4F5579 
| I | 0xA56C5A7AAAA8CBDE3D83834C836D8830 
; FindVUK 1.02/MKBv19/BD+/Region-B-/Playlist00800

If you only take input from "trusted" sources like FindVuk you should not be too concerned with validation of the data or calculation of MK/VID to VUK (although without encryption anyone can fake entries). I just wanted to point out that there will be submissions for the same DiscId containing different data, for a multitude of reasons. You need to take into account how to deal with such scenarios as without access to the actual disc, the data is hard or impossible to verify.

With support for regular Bluray, we are not talking about a database of 200 UHD discs, easily maintainable manually. This will be in the region of 80.000 known DiscIds just to start with (and several 100.000 UPC if you want to track those)

Also it should be possible to submit from sources other than FindVUK, e.g. a program similar to AACSUpdater (this could even be built in to FindVUK) can harvest DiscID/VUK/VID keys from VLC AACS cache directories

[nalor]

I already added a 'synchronize' mode to FindVUK that downloads a keydb.cfg from a website, compares it to the local file and finally:
1) adds new entries from the online file to the local file
2) apply changes of entries that are different between both files to the local file (the online file is the 'master')
3) uploads new entries from the local file to the online db also as xml in a different format.

Here's an example:

<?xml version="1.0" encoding="UTF-8"?>
<Bluray>
	<FileType>BlurayLegacyXML</FileType>
	<LegacyEntries>
		<LegacyEntry>
			<DiscId Date="">C0A58017FAD25AC92C3B9F19D74FC1E5BFC1B025</DiscId>
			<Title>MARAUDERS (Marauders)</Title>
			<VolumeUniqueKey>1A741C903EB21F024F94B2F3F3F19D2A</VolumeUniqueKey>
			<VolumeId/>
			<MediaKey/>
			<Comment>MKBv62/FindVUK 0.96</Comment>
		</LegacyEntry>
		<LegacyEntry>
			<DiscId Date="">E20E7A19466208B7C58D0A87B8050425564E0F49</DiscId>
			<Title>Bikini Destination triple fantasy</Title>
			<VolumeUniqueKey>0E0BCB9F4F66870166F4C6FC23A3020F</VolumeUniqueKey>
			<VolumeId/>
			<MediaKey/>
			<Comment/>
		</LegacyEntry>
	</LegacyEntries>
</Bluray>

And I also thought how it's possible to prevent invalid entries in the database created by super funny script kiddies .. and decided that it's necessary to sign the xml in order to decide if it's trustworthy or not.

I know there's an XML signature specification but it's not included in Purebasic and after reading the spec I decided that it would definitely be too much work to reimplement it in Purebasic (and I also thought that I could use an external library - but it's also too complicated or I'm too stupid..).

In the end I created my own format that looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<Main>
	<Data>
		<UUID>fa9c7bdf-00d5-49e6-9c27-ba4d39d9c9f8</UUID>
		<Timestamp>2018-01-13T10:00:44+0000</Timestamp>
		<Base64>PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiPz48Qmx1cmF5PjxGaWxlVHlwZT5CbHVyYXlNZXRhWE1MPC9G
		aWxlVHlwZT48RGlzY0lkIERhdGU9IjIwMTctMDgtMjIiPjE2QUJGRDgzMzc2RDQxRThDRDI2RDE0MzhGRjAyMTQ2MDU5NjYzQkQ8L0Rp
		c2NJZD48Vm9sdW1lSWQ+RjE4QjQ4RjJCQzk3MTlFNDcxQkVENjNFQTQzQkRDMjU8L1ZvbHVtZUlkPjxNZWRpYUtleT5EMkJGOERBQzZB
		Mjg4QTgwQjg0Q0E1RTg5MjRENzZDQjwvTWVkaWFLZXk+PFZvbHVtZVVuaXF1ZUtleT5FN0U1MTYzMjhCQUZBRThEOEQ5MUMyODUzMDY5
		NUVFODwvVm9sdW1lVW5pcXVlS2V5PjxWb2x1bWVMYWJlbD5XYXIgZm9yIHRoZSBQbGFuZXQgb2YgdGhlIEFwZXM8L1ZvbHVtZUxhYmVs
		PjxCRHBsdXMgRGF0ZT0iMjAxNy4wOS4wNyI+MTwvQkRwbHVzPjxCdXNFbmNyeXB0aW9uRW5hYmxlZD4xPC9CdXNFbmNyeXB0aW9uRW5h
		YmxlZD48TUtCcmV2PjUwPC9NS0JyZXY+PE1haW5QbGF5bGlzdD4wMDgwMC5tcGxzPC9NYWluUGxheWxpc3Q+PFVuaXF1ZUtleXM+PFVu
		aXF1ZUtleSBOcj0iMSI+ODBDQTBCNzEyRTFGRDExMEU4QjkyODdCOUYzQzRGQUM8L1VuaXF1ZUtleT48L1VuaXF1ZUtleXM+PE1ldGFU
		aXRsZXM+PE1ldGFUaXRsZSBMYW5ndWFnZT0iZW5nIj5XYXIgZm9yIHRoZSBQbGFuZXQgb2YgdGhlIEFwZXM8L01ldGFUaXRsZT48TWV0
		YVRpdGxlIExhbmd1YWdlPSJmcmEiPkxhIFBsYW7DqHRlIGRlcyBTaW5nZXMgOiBTdXByw6ltYXRpZTwvTWV0YVRpdGxlPjxNZXRhVGl0
		bGUgTGFuZ3VhZ2U9InNwYSI+TGEgZ3VlcnJhIGRlbCBwbGFuZXRhIGRlIGxvcyBzaW1pb3M8L01ldGFUaXRsZT48TWV0YVRpdGxlIExh
		bmd1YWdlPSJubGQiPldhciBmb3IgdGhlIFBsYW5ldCBvZiB0aGUgQXBlczwvTWV0YVRpdGxlPjxNZXRhVGl0bGUgTGFuZ3VhZ2U9ImRl
		dSI+UGxhbmV0IGRlciBBZmZlbjogU3Vydml2YWw8L01ldGFUaXRsZT48TWV0YVRpdGxlIExhbmd1YWdlPSJpdGEiPlRoZSBXYXIgLSBJ
		bCBQaWFuZXRhIGRlbGxlIFNjaW1taWU8L01ldGFUaXRsZT48TWV0YVRpdGxlIExhbmd1YWdlPSJqcG4iPldhciBmb3IgdGhlIFBsYW5l
		dCBvZiB0aGUgQXBlczwvTWV0YVRpdGxlPjxNZXRhVGl0bGUgTGFuZ3VhZ2U9ImNhdCI+TGEgZ3VlcnJhIGRlbCBwbGFuZXRhIGRlIGxv
		cyBzaW1pb3M8L01ldGFUaXRsZT48L01ldGFUaXRsZXM+PC9CbHVyYXk+</Base64>
	</Data>
	<Signature>
		<Checksum cipher="SHA256">0da227c5c0419892e196a8fff53f6f43e94a378e67359ddeba51416f5bfa6ed6</Checksum>
		<Signature>1D783940EFA594B7955B21F353336A3F2228265A44F445B18DB9029B585CD2A17EE68CD1C7
C3C2D69C534C78040130F073CC1075D55A63FCA62843BD0EBD4A09</Signature>
	</Signature>
</Main>

(usually Base64-Data and the Signature do not include linebreaks - just added them here to reduce the length of the lines)

The original XML (BlurayMetaXML or BlurayLegacyXML) is Base64 encoded - a SHA256 checksum is created from the data, a unique uuid and the timestamp - and the checksum is signed with libsodium and a private key that is 'embedded' into findvuk.
On the other end I've created a php-script that checks if the checksum matches the data and if the signature validates against the checksum with the help of a public key.
I know embedding a key into an application isn't really a secure solution - but I think it serves the purpose (if someone has a better idea just tell me).

So this is everything that is already working on my side

[Relight]

Originally Posted by nalor

So this is everything that is already working on my side

Cool

I need to think a bit more about Legacy entries. I sent you a couple messages, let's make things happen

[Snap]

Neato. Pity ol' vBulletin doesn't support notifications or I would have noticed it sooner

Lately I've been checking this list maintained by a MyCE user, as recommended to me by a user perusing the keys thread. What might be time saving is if their spreadsheet could be used as a source to populate the site perhaps (given permission/credit of course), as it's already a pretty solid set of info per disc, including which releases are compatible with DeUHD (which at this point has become the only actively updated decryption method).

One thought on the UHDDB site is the Submission section asks for the type of release and AACS version. Seeing as the domain and home page suggest its for UHD releases only maybe it could be more straightforward to focus submissions to just UHD? Or perhaps creating a separate browsing category to clearly separate them and simplifying the submission to just UHD or Blu-Ray as an option. Just a thought.

Looking forward to how this progresses.

[DavVador]

Originally Posted by nalor

I already added a 'synchronize' mode to FindVUK...

Is there a place where we can download your latest version of FindVUK ?
The only one i can find is 1.02.
Unless it's not for public release.

[Relight]

New development and new keys here: http://makemkv.com/forum2/viewtopic.php?f=12&t=16959

Anyone have any thoughts about what these "hashed keys" are?

[candela]

Originally Posted by Relight

New development and new keys here: http://makemkv.com/forum2/viewtopic.php?f=12&t=16959

Anyone have any thoughts about what these "hashed keys" are?

At first glance, these hashed keys are just a way to force people into using makemkv. Why else would he also not release the specification for this fantastic new file format in this announcement. He even states it is possible to encode the keys in both formats but then only releases the hashed keys. Also, no info on how these keys are better. The format itself is obviously not superior since it contains no info except a volumelabel and some kind of unknown hash which provides no way to identify a disc you own like the SHA-1 hash keydb uses. For all we know he can simply take the discid+vuk, encrypt them and then apply something similar to base64 encoding

[datman]

Originally Posted by candela

At first glance, these hashed keys are just a way to force people into using makemkv. Why else would he also not release the specification for this fantastic new file format in this announcement. He even states it is possible to encode the keys in both formats but then only releases the hashed keys. Also, no info on how these keys are better. The format itself is obviously not superior since it contains no info except a volumelabel and some kind of unknown hash which provides no way to identify a disc you own like the SHA-1 hash keydb uses. For all we know he can simply take the discid+vuk, encrypt them and then apply something similar to base64 encoding

Well all due respect can't you just copy them into your KEYDB.cfg file. Me it contains keys I need so all smiles here.

[Snap]

Originally Posted by datman

Well all due respect can't you just copy them into your KEYDB.cfg file. Me it contains keys I need so all smiles here.

The VUK and the 'Hashed Keys' (maybe I'll abbreviate it to 'HK') are different formats, and HK is only compatible with MakeMKV currently.

The announcement post did however state 'no other software can read files in this format so far' (emphasis mine) so I'm not sure if that's a challenge to other devs or if they'll open it up (though currently this feature does give MakeMKV an edge vs the outrageously expensive DeUHD).

The part in the announcement about the key being able to be read from a keydb.cfg or HK format is just regarding how MakeMKV can read it, afaik. In other words, now that MakeMKV supports reading two formats the user can use either one, but as of now only MakeMKV can read and convert to the HK format.