Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
|
|
Thread Tools | Search this Thread | Display Modes |
|
22nd September 2008, 17:55 | #1 | Link |
Registered User
Join Date: Jul 2008
Posts: 96
|
.bdmv, .clpi, .mpls and .m2ts - Exact specs of AVCHD/BD structure and files
Okay, since I was simply unable to find anything specific on the structure and binary contents of the files on a BD/AVCHD, i thought it warrants its own thread (sorry if there already is one, I couldn't find anything upon searching!).
On making my AVCHD menu generator I learned a few things about the structure and the files, but the majority of the bytes remain a mystery to me. Since it's near impossible to get roman76 to react to queries we also don't know how he generates his files in tsMuxer when using the BD mux option. The official AVCHD Info site doesn't tell you shit about exact specs, all they give you is some vague info how the video streams should look like.. So here's my findings: 1) Structure: Root is always the folders BDMV and CERTIFICATE. On FAT32 this needs to be within a folder called AVCHD and on optical media BDMV and CERTIFICATE need to be on the root level. CERTIFICATE seems to be mandatory (probably for keyfiles) and contains an (empty) BACKUP folder for redundancy, AFAIK it is always empty for self-authored discs. BDMV is where it's at, and in there we have: AUXDATA: Auxilliary Data judging by the name. Probably bonus material for your computer like screensavers etc goes here. BACKUP: Contains backups of the CLIPINFO and PLAYLIST directories along with the two .bdmv files in the BDMV directory. For redundancy BDJO: Probably has to do with the Bluray Java, but I'm not sure in how far. This NES emulator for PS3 has a .bdjo file in there CLIPINF: This contains one .clpi file for each .m2ts stream (also split ones) of the same name with exact specs about the stream JAR: The Java Applets probably go in here. META: Dunno. Metadata? D'oh, what kind of Metadata? PLAYLIST: Contains a .mpls file for every track (not stream!) on the disc. So if a disc made with my script has 4 tracks, there's gonna be 4 mpls files in there, one for each, no matter into how many chunks the stream was split up (plus there must be -also with m2ts and clpi- 00000.mpls, which is what the BD/AVCHD starts with, in my case the menu track!) STREAM: All .m2ts streams go in here. These are the actual content! This structure must be followed 100% as far as I know, so even though they might be empty all these directories need to be there in this structure, with the exact names in caps! 2) Differences between AVCHD FAT32/UDF: The only difference between AVCHD stored on a FAT32 volume and optical media lies in the filenames, because 8.3 naming is used on FAT32, and it's all caps: *.clpi (UDF) becomes *.CLP (FAT32), *.mpls becomes *.MPL, *.m2ts becomes *.MTS, MovieObject.bdmv becomes MOVIEOBJ.BDM and index.bdmv becomes INDEX.BDM Even though the .mpls files reference the .m2ts files internally, it seems there is some fault tolerance, probably required by the spec, so there's no need to change the string in the .mpls files accordingly from M2TS to MTS and vice versa. 3) File contents: Unfortunately I was only able to find out something about the structure of .mpls playlist files by looking at it in a binary editor: First there's a header, "MPLS0200", which is probably a file identifier with version number. Some bytes later (not always the same offset!), the M2TS file that is played first is referenced, (aways with a P infront).. Around 60 bytes later seems to be the language code, preceded with an "a" (so e.g. it's aeng for english). All of this minus the header is repeated in the same mpls file if one track has more chunks to be played sequentially. The rest of the bytes is mostly zeroes, so reverse-engineering it should be possible... .clpi files don't give much away, there's a header again (HDMV0200) and further down after lots of zeroes there's that language code with 'a' infront again.. index.bdmv is the same thing: Header (INDX0100) and mostly zeros... MovieObject.bdmv: Header (MOBJ0100) and mostly zeros... So. Does anyone know where to get further specs about this? I'd especially like to know how the menu m2ts (00000.m2ts) is structured so I could inject my own graphics in the menu generator! ;-) I would guess it is rather similar to the structure of those VTSx_0.VOBs on a DVD that contain the track's menu gfx!... Last edited by kaid; 22nd September 2008 at 22:50. |
24th September 2008, 14:14 | #2 | Link | |
Registered User
Join Date: Jul 2008
Posts: 96
|
Something else that would be of great interest are the contents of .clpi files, as I would like to be able to generate my own, so all that's needed for my script is m2ts files as input...
I just found this, a patent for recordable BDs that tells a lot about the filestructure, very very interesting: Quote:
Last edited by kaid; 24th September 2008 at 14:24. |
|
26th September 2008, 15:00 | #3 | Link |
Registered User
Join Date: May 2008
Posts: 1,840
|
The way I understand and recently experienced is AVCHD is a digital camera format which always uses the FAT32 structure changes you mention, it doesn't have the FAT32 4GB restriction. It can be burned onto a DVD or BD-R and work on most BD players. Also from what I gather AVCHD has different specs then BD, for example no DTS support and different max video bitrate.
BD is the format that has BDMV\CERTIFICATE in root, what tsmuxer outputs and what your program is intended for. I know it's easy to get these 2 confused because some programs output BD and call it AVCHD, and some players display AVCHD, when its technically BD. |
27th September 2008, 17:43 | #4 | Link | |||
Registered User
Join Date: Jul 2008
Posts: 96
|
Quote:
From what i gathered AVCHD isn't always on FAT32, when it's a mini-DVD-R based camcorder it isn't, because then it uses UDF and the different naming scheme (atleast that's what the spec says!).. But if the camcorder is using SD it HAS to use FAT32 (NTFS on Flashmedia isn't supported by anything!) and FAT32 *always* has the 4GB-limit! Quote:
Quote:
Or is there some other differences I don't know about yet? |
|||
3rd October 2008, 10:58 | #6 | Link |
Banned
Join Date: Oct 2001
Location: https://t.me/pump_upp
Posts: 811
|
I want to clarify:
AVCHD headers index.bdmv - INDX0100 MovieObject.bdmv - MOBJ0100 PLAYLIST *.mpls - MPLS0100 CLIPINF *.clpi - HDMV0100 So NERO does it right. Blu-ray headers on BD-RE index.bdmv - INDX0200 MovieObject.bdmv - MOBJ0200 PLAYLIST *.mpls - MPLS0200 CLIPINF *.clpi - HDMV0200 ATTENTION! tsMuxeR mixed up headers, it sets mpls and clpi headers as Blu-ray = WRONG! You have to patch *.mpls and *.clpi to AVCHD compliant headers. Last edited by frank; 3rd October 2008 at 11:01. |
29th January 2009, 04:29 | #7 | Link |
Registered User
Join Date: Jan 2009
Location: By the Yellow Brick Road
Posts: 58
|
@kaid
I know its been a while since anyone has contributed to this thread... and I am just looking for info about the layout of the *.clpi file(s). My issue is that when using tsMuxeR to create a BD structure, it has on many occasions messed up the AR info found in the .clpi file. I have muxed source files which had original ARs 2.35 or 2.4 and tsMuxeR has it set to 4:3 in the .clpi file. When these are burned to disc and played on a standalone, I get a "good" picture on the left half of my 16:9 HDTV and a ghost-like similar image on the right, as tho it was 2 tracks running side-by-side. I have tried using BDedit but can not edit the .clpi file. I am hoping that you, in your research, have found where in the .clpi the AR info is stored and could help me with the off-set so I could try hexing that to see whether I can get the correct AR without having to re-encode the source x264 file with borders. TIA, James
__________________
Vista SP1 32-bit w/4GB RAM AMD Athlon 64 X2 4400+ @ 2.23GHz nVidia GeForce 8400GS w/512MB RAM |
30th January 2009, 01:41 | #9 | Link | |
Registered User
Join Date: Jan 2009
Location: By the Yellow Brick Road
Posts: 58
|
Quote:
I took a look at the .clpi from a movie-only rip, using HxD and it shows a value of 30h at offset 109. MediaInfo tells me that the movie has original AR of 2.4:1. Where can I find which hex value equals 16:9 and which equals 4:3? Thanks in advance for any pointers. James
__________________
Vista SP1 32-bit w/4GB RAM AMD Athlon 64 X2 4400+ @ 2.23GHz nVidia GeForce 8400GS w/512MB RAM |
|
31st January 2009, 23:16 | #11 | Link |
Registered User
Join Date: Jan 2009
Posts: 1
|
to deank
I have mailed you before time links to download samples of AVCHD structure made by your (great) program and made by Panasonic HDWriter 2.5e for the same stream. I hope you can see differences and this will helpful in your researche. Your structure is on PanasonicTV unfortunately functionless, from HDWriter is OK. Has Panasonic in his structure some one secret byte or what ? |
1st February 2009, 07:33 | #12 | Link | |
Registered User
Join Date: Jan 2009
Location: By the Yellow Brick Road
Posts: 58
|
Quote:
I understand the limitation on AR but it seems that after I have backed up the movie, the 16:9 info in the stream is missing and gets replaced by an AR calculated from the actual pixels. The non-AVCHD/BD AR appears after I have run the movie through tsMuxeR. Thanks again. I will get back here and let you know whether the AR change is successful. James
__________________
Vista SP1 32-bit w/4GB RAM AMD Athlon 64 X2 4400+ @ 2.23GHz nVidia GeForce 8400GS w/512MB RAM |
|
1st February 2009, 22:38 | #13 | Link | |
Registered User
Join Date: Jan 2009
Location: By the Yellow Brick Road
Posts: 58
|
Quote:
I am running a re-encode now to see whether x264 and/or tsMuxeR will accept the change and re-encode the stream as 16:9. Thanks again. James
__________________
Vista SP1 32-bit w/4GB RAM AMD Athlon 64 X2 4400+ @ 2.23GHz nVidia GeForce 8400GS w/512MB RAM |
|
28th July 2018, 16:29 | #14 | Link |
Registered User
Join Date: Jul 2013
Posts: 60
|
I have ton's of BMDV folders coming from a Panasonic video camera. I would like to parse the timestamp of each recorded clip from the clpi files. (I know the date of each recording, but not time.)
I'm trying to identify possible byte addresses of that possibly stored timestamp within the clpi files by comparing clpi files from the same day versus comparing clpi files from different days vs from different years. (I'm using "vbindiff" to make the differences visible) Addresses that look more different the farther apart two clips are should be good candidates for timestamps. I'm documenting my findings here, maybe they will be useful for others too: 0x17 & 0x1B: single bytes that always have different values, 0x1B seems to be equal to the value of 0x17 plus 4. 0x39 - 0x3B: 3 bytes always different, didn't find a pattern yet. 0xE2 - 0xEA: Most clips: ASCII "00001M2TS" where the number is counting (and is always the number in the filename+1), Those clips have a 0x01 at address 0xDD. Clips with a 0x00 at 0xDD don't have this ASCII string. Some of the clips without ASCII string have value 0x00F6 instead of 0x0106 @ 0xE & 0xF. Those files are missing 0x10 bytes between 0xC3 & 0xEF. So for those clips 0x10 must be subtracted for any address higher than 0xEF for the value to match the discription. 0x100 - 0x105: Clips on the same day mostly only vary on 0x101 & 0x103 - 0x105, clips from different days also vary in 0x100. 0x119 contains the 0x30 described by others above as coding for aspect ratio at address 0x109 (which is the address of this 0x30 for clpi files without the ASCII string mentioned 3 lines above.) 0x146 - 0x147, 0x151 - 0x153, 0x15B: more changes. I can't see any patterns here.. Last edited by kaefert; 28th July 2018 at 22:03. |
28th July 2018, 16:55 | #15 | Link | |
Registered User
Join Date: Sep 2013
Location: Czech Republic
Posts: 321
|
Quote:
|
|
5th August 2018, 15:27 | #17 | Link | |
Registered User
Join Date: Apr 2004
Location: London, England
Posts: 168
|
Quote:
According to the official specs site, I need to have these books and a license just to use the perishing logo (see Authoring House Logo Usage) section at the BDA Licensing office It doesn't stop there either, as I also need the BD-RE2 specs to cover any Java issues - most of the good information on BD-J is long since taken down (probably because Netflix & Amazon use a very similar system on their pay per view models) and even HD Cookbook site is history these days. So if you could kindly point me in the right direction it would be greatly appreciated
__________________
www.opusproductions.com Multichannel Audio Productions DVD-Audio/Video Authoring specialists |
|
28th July 2018, 22:04 | #18 | Link |
Registered User
Join Date: Jul 2013
Posts: 60
|
oh thanks, I'll check it out.
update: hmm, this seems to be windows only.. doesnt work with mono.. And I did search for specs of those clpi files, this thread was the only thing I found. UPDATE: yess! that worked! With wine AVCHDInfo.exe can extract Timestamps for each frame of an m2ts file, so it doesn't need the clpi files to get that info but only the video file itself UPDATE2: ahh, and the MPLS2JSON.exe tool shows me that I've been looking for a non existent needle in the haystack, no timestamp infos in there Last edited by kaefert; 28th July 2018 at 22:21. |
Tags |
avchd, bluray, clpi, m2ts, structure |
Thread Tools | Search this Thread |
Display Modes | |
|
|