Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
16th June 2008, 13:49 | #1 | Link |
Registered User
Join Date: Jun 2008
Posts: 13
|
MKV to M2TS on Linux
The point of this post is to discuss converting HD content in MKV files to M2TS files for streaming to a PS3, via Mediatomb, on an Ubuntu Linux server.
The PS3 can be quite fussy on what video and audio formats are acceptable when it comes to streamed HD content. The main problem I have remaining is when the Video stream resolution isn’t 1920x1080 or 1280x720 - my PS3 won’t show any video on playback. I’ve read a lot on various forums and most solutions involve using Windows GUI applications (E.g. avisynth, MeGUI). As I’m using a Linux server to store my files for streaming - I would really like to create a script to do the job in one hit if possible and just leave my server to chug away. (And thus Windows apps aren’t an option for me.) Before I go too far I would say I’m novice in the field, (Novice = Noob + '3weeks reading forums like mad' + 'tinkering at home'), so I’m drawing together items from various threads to help other people starting out. Therefore I am happy to accept constructive criticism here to improve the content of this thread as that will only serve to help others down the line. Here’s what I found so far - PS3 Valid Video Criteria:
PS3 Valid Audio Criteria:
I must credit sticky123 for a great blog post which is the foundation to this posting. I am hoping to take his/her work a step or two further. There are three scenarios I would like to discuss and really need help with number 3: Scenario 1) the MKV file contains video and audio streams which are both PS3 valid. As you can see the (fantastic) application tsMuxer does the repackaging job without the need to re-encode any data stream. It can also force the Video profile level to 4.1 to ensure the PS3 is happy. Scenario 2) here the Audio stream isn’t PS3 compliant. i.e. stored in a DTS format. We can ‘break’ the DTS stream out of the MKV file into a separate file on it’s own and re-encode it to AC3 format, finally re-muxing (via tsMuxer) the new AC3 file and Video stream into a M2TS file. Scenario 3) this time the Video stream isn’t PS3 compliant. i.e. the resolution isn’t quite right. Now this is where my knowledge/experience lets me down and why I’m looking for help from the rest of you. As far as I’m aware it should be, (he says laughably), a simple case of add black borders to the video stream to bring it up-to the correct resolution. (Obviously ensure the aspect ratio is maintained.) I would want to keep the Video data at the same quality/codec as the original source. From what I’ve read so far there are some powerful (free) tools available such as mencoder, ffmpeg and x264. However all of them have more command line options than you can shake a stick at and finding the correct route to take has been rather daunting. Thus I’m asking for help. Scenario 4) obviously it’s possible for Scenario 2 & 3 to apply to one MKV file, but I didn’t think I needed to go over that individually. I’m hoping some of you will find this useful, and I’m hoping more that someone will be able to help me with Scenario 3. Once I’ve got a reasonable solution for that I will post my script for all to share. Best Regards Chuckle Worthy Other things to discuss at a later date: - subtitles. (I don’t want to discuss subtitles at the moment, maybe once we’ve got the video and audio sorted then we can take things a step further.) - ff/rewind with HD resolutions doesn’t work very well. |
16th June 2008, 14:13 | #3 | Link |
Registered User
Join Date: Jun 2008
Posts: 13
|
Isn't Avisynth a Windows GUI application?
A pal of mine gave me a DVD with a load of film trailers on. I've just got a HD TV and I'm trying to be flash and show off the TV and video streaming etc. But as per - I'm not had too much success. Last edited by ChuckleWorthy; 16th June 2008 at 14:19. |
16th June 2008, 18:14 | #8 | Link |
Registered User
Join Date: Jun 2008
Posts: 13
|
Ohhh. Heh. I'll turn off my dullard setting...
I used Microsoft Visio 2003 Standard Edition - I always find a couple of pictures help explain things far easier then paragraphs of waffle. Last edited by ChuckleWorthy; 17th June 2008 at 12:43. |
23rd June 2008, 15:12 | #9 | Link |
Registered User
Join Date: Feb 2003
Posts: 22
|
I have posted my script which does this in another thread. Feel free to use it.
EDIT: Sorry, I did not realize you already have most of the scenarios covered. Just ignore my post. Last edited by Grimmi Meloni; 23rd June 2008 at 15:15. |
25th June 2008, 21:21 | #10 | Link |
Registered User
Join Date: Jun 2008
Posts: 13
|
Okay I've been answering my own question as best as I can. This is my first adventure in this world, so after lots and lots of tinkering later - 'mencoder' seemed like the best way forward.
Like I said here's what I've come up with so far. It has produced some really good results IMO. Again I'm happy for comments/suggestions/help etc. Code:
#!/bin/bash #** #** m2tsEx.sh - a script for converting MKV files to M2TS for streaming to a PS3 #** from a DLNA Media server. #** Uses: #** - tsMuxer: http://www.videohelp.com/tools/tsMuxeR #** (I use the Win32 version of tsMuxer as I found the Linux version unreliable) #** - wine: http://www.winehq.org/ #** - mkvtoolnix: http://www.videohelp.com/tools/MKVtoolnix #** - dcadec: http://www.videolan.org/developers/libdca.html #** - aften: http://aften.sourceforge.net/ #** - mencoder: http://www.mplayerhq.hu/ #** - x264: http://www.videolan.org/developers/x264.html #** #** v0.1 initial version - 24th June 2008 #** #** Usage: m2tsEx.sh filename.mkv #** #** Functions ******************************************************************************** RED=$( echo -e "\e[31;40m" ) GREEN=$( echo -e "\e[32;40m" ) YELLO=$( echo -e "\e[33;40m" ) BLUE=$( echo -e "\e[34;40m" ) MAGENTA=$(echo -e "\e[35;40m" ) CYAN=$( echo -e "\e[36;40m" ) RESET=$( echo -e "\e[0m" ) dmpstart() { echo "--------------------------------------------------------" > "./process.log"; } dmp() { echo "`date`: $*" >> "./process.log"; } #** colorful echo functions msg() { echo ""; echo ">> $*"; dmp "$*"; } rmsg() { echo ""; echo ">> ${RED}$*${RESET}"; dmp "$*"; } gmsg() { echo ""; echo ">> ${GREEN}$*${RESET}"; dmp "$*"; } ymsg() { echo ""; echo ">> ${YELLO}$*${RESET}"; dmp "$*"; } bmsg() { echo ""; echo ">> ${BLUE}$*${RESET}"; dmp "$*"; } mmsg() { echo ""; echo ">> ${MAGENTA}$*${RESET}"; dmp "$*"; } cmsg() { echo ""; echo ">> ${CYAN}$*${RESET}"; dmp "$*"; } #** Constants ******************************************************************************** BIN_DIR="/usr/bin" LOCAL_BIN_DIR="/usr/local/bin" WORKING_DIR="./wrking/" SEVEN20="1280:720" TEN80="1920:1080" SEVEN20_WIDTH="1280" TEN80_WIDTH="1920" PRIMARY_LANGUAGE="eng" #** Used in the mux.meta file, change this if you want... (obviously) SECONDARY_LANGUAGE="und" #** Check the passed parameter is a MKV file ************************************************ dmpstart cmsg "Starting..." SRC_FILE="$1" EXTENSION=`$BIN_DIR/basename $SRC_FILE | awk -F . '{print $NF}' | tr 'A-Z' 'a-z'` BASENAME=$($BIN_DIR/basename "$SRC_FILE" .mkv) DEST_FILE=$BASENAME.m2ts if [ "$EXTENSION" != "mkv" ] then rmsg "Not a Matroska File, exiting..." exit 1 fi #** EXTRACT DETAILS OUT OF THE MKV FILE ****************************************************** #** Get the info from the MKV file cmsg "Extracting MKV Information..." MKV_INFO=`$BIN_DIR/mkvinfo "$SRC_FILE"` #** Check there are Streams within the MKV we're interested in cmsg "Determine Track Id's..." VIDEO_TRACK_NO=`echo "$MKV_INFO" | grep "V_MPEG4/ISO/AVC" -B10 | grep "Track number:" | awk '{ print $5 }'` AC3_TRACK_NO=`echo "$MKV_INFO" | grep "A_AC3" -B10 -C3 | grep -E "Language: $PRIMARY_LANGUAGE"\|"Language: $SECONDARY_LANGUAGE" -B13 | grep "Track number:" | awk '{ print $5 }' | head -1` DTS_TRACK_NO=`echo "$MKV_INFO" | grep "A_DTS" -B10 -C3 | grep -E "Language: $PRIMARY_LANGUAGE"\|"Language: $SECONDARY_LANGUAGE" -B13 | grep "Track number:" | awk '{ print $5 }' | head -1` if [[ -z $VIDEO_TRACK_NO ]] then rmsg "No Video Stream found, exiting..." exit 1; fi if [[ -z $AC3_TRACK_NO ]] && [[ -z $DTS_TRACK_NO ]] then rmsg "No AC3 or DTS Streams found, exiting..." exit 1; fi #** AUDIO TRACK INVESTIGATION **************************************************************** #** Okay, lets find the lowest value Track Number, as that generally is the Main audio stream #** I'll cheat by setting any null values to a high number, so the non-nulls will win the test cmsg "Sort Out Audio Tracks..." if [[ -z $AC3_TRACK_NO ]] then AC3_TRACK_NO=999 fi if [[ -z $DTS_TRACK_NO ]] then DTS_TRACK_NO=999 fi if [[ $AC3_TRACK_NO -lt $DTS_TRACK_NO ]] then AUDIO_TRACK_NO=$AC3_TRACK_NO AUDIO_TRANSCODE=1 #FALSE gmsg "AC3 Audio Stream Found - Track Id: $AC3_TRACK_NO." else AUDIO_TRACK_NO=$DTS_TRACK_NO AUDIO_TRANSCODE=0 #TRUE gmsg "DTS Audio Stream Found - Track Id: $DTS_TRACK_NO." fi #** VIDEO TRACK INVESTIGATION **************************************************************** #** Get the Video Resolution etc cmsg "Sort Out Video Resolution..." SRC_WIDTH=`echo "$MKV_INFO" | grep "Pixel width:" | awk '{ print $5}'` SRC_HEIGHT=`echo "$MKV_INFO" | grep "Pixel height:" | awk '{ print $5}'` SRC_RESOLUTION="$SRC_WIDTH:$SRC_HEIGHT" SRC_FPS=`echo "$MKV_INFO" | grep "V_MPEG4/ISO/AVC" -A3 | grep "fps" | awk '{ print $6 }' | cut -c2-100` #** Calculate the Average Bit Rate SRC_SECONDS=`echo "$MKV_INFO" | grep Duration | awk '{ print $4}' | sed 's/s//' | awk -F . '{ print $1 }'` SRC_SIZE=`echo "$MKV_INFO" | grep Segment, | awk '{ print $4}'` SRC_AVG_BITRATE="$[ ( ($SRC_SIZE * 8) / ($SRC_SECONDS ) / 1024 ) ]" cmsg "Source Video Details - Resolution: $SRC_RESOLUTION, Duration: $SRC_SECONDS seconds, Average Bitrate: $SRC_AVG_BITRATE, Frames Per Second: $SRC_FPS." #** Calculate the desired Resolution if [[ -z $2 ]] then if [[ "$SRC_WIDTH" != "$SEVEN20_WIDTH" ]] && [[ "$SRC_WIDTH" != "$TEN80_WIDTH" ]] then rmsg "Odd Resolution: $SRC_RESOLUTION. Please specify manually: 1920:1080 or 1280:720, exiting..." exit 1; else if [ "$SRC_WIDTH" == "$SEVEN20_WIDTH" ] then DEST_RESOLUTION=$SEVEN20; fi if [ "$SRC_WIDTH" == "$TEN80_WIDTH" ] then DEST_RESOLUTION=$TEN80; fi fi else #** Use the passed paramter for the new Resolution DEST_RESOLUTION=$2 fi #** See if we need to transcode the video if [[ $SRC_RESOLUTION == $DEST_RESOLUTION ]] then VIDEO_TRANSCODE=1 #FALSE mmsg "Video Stream - Track Id: $VIDEO_TRACK_NO. No Transcoding Required." else VIDEO_TRANSCODE=0 #TRUE mmsg "Video Stream - Track Id: $VIDEO_TRACK_NO. Transcoding To $DEST_RESOLUTION Resolution." fi #** START THE CONVERSION ****************************************************** #** Create the working temporary directory mkdir -p ${WORKING_DIR} #** Create the tsMuxer config file echo "MUXOPT --no-pcr-on-video-pid --new-audio-pes --vbr " > ${WORKING_DIR}mux.meta if [[ $VIDEO_TRANSCODE -eq 1 ]] then #** No Video Transcoding - just reference the source MKV file echo "V_MPEG4/ISO/AVC, \"$SRC_FILE\", level=4.1, insertSEI, contSPS, track=$VIDEO_TRACK_NO, fps=$SRC_FPS, lang=$PRIMARY_LANGUAGE" >> ${WORKING_DIR}mux.meta else #** Video Transcoding - break out the source stream cmsg "Extracting H264 Stream..." $BIN_DIR/nice -n20 \ $BIN_DIR/mkvextract tracks "$SRC_FILE" $VIDEO_TRACK_NO:"${WORKING_DIR}$BASENAME.h264" #** Start Video Transcoding X264PASS1="" X264PASS1="${X264PASS1}pass=1:threads=1:turbo=1:bitrate=$SRC_AVG_BITRATE:frameref=1:bframes=3:" X264PASS1="${X264PASS1}b_pyramid:direct_pred=auto:weight_b:me=dia:mixed_refs:bime:" X264PASS1="${X264PASS1}trellis=1:nopsnr:nossim:subq=1:level_idc=41" #** This sets the Profile Level to 4.1 X264PASS2="" X264PASS2="${X264PASS2}pass=2:threads=1:turbo=0:bitrate=$SRC_AVG_BITRATE:frameref=1:bframes=3:" #** I found a frameref of 3 was a no-no X264PASS2="${X264PASS2}b_pyramid:direct_pred=auto:weight_b:me=umh:mixed_refs:bime:" X264PASS2="${X264PASS2}partitions=all:8x8dct:" X264PASS2="${X264PASS2}trellis=1:nopsnr:nossim:subq=6:level_idc=41" cmsg "Video Transcoding - First Pass..." $BIN_DIR/nice -n20 \ $BIN_DIR/mencoder \ "${WORKING_DIR}$BASENAME.h264" \ -vf expand=$DEST_RESOLUTION \ -ofps $SRC_FPS \ -ovc x264 \ -x264encopts $X264PASS1 \ -nosound \ -of rawvideo \ -o /dev/null; # -o "${WORKING_DIR}${BASENAME}-pass1.h264" > ${WORKING_DIR}pass1.txt; # Second pass cmsg "Video Transcoding - Second Pass..." $BIN_DIR/nice -n20 \ $BIN_DIR/mencoder \ "${WORKING_DIR}$BASENAME.h264" \ -vf expand=$DEST_RESOLUTION \ -ofps $SRC_FPS \ -ovc x264 \ -x264encopts $X264PASS2 \ -nosound \ -of rawvideo \ -o "${WORKING_DIR}${BASENAME}-pass2.h264"; # -o "${WORKING_DIR}${BASENAME}-pass2.h264" > ${WORKING_DIR}pass2.txt; #** Get rid of the first Pass statistics wosname rm -f "divx2pass.log" echo "V_MPEG4/ISO/AVC, \"${WORKING_DIR}${BASENAME}-pass2.h264\", level=4.1, insertSEI, contSPS, track=1, fps=$SRC_FPS, lang=$PRIMARY_LANGUAGE" >> ${WORKING_DIR}mux.meta fi if [[ $AUDIO_TRANSCODE -eq 1 ]] then #** No Audio Transcoding - just reference the source MKV file echo "A_AC3, \"$SRC_FILE\", track=$AUDIO_TRACK_NO, lang=$PRIMARY_LANGUAGE" >> ${WORKING_DIR}mux.meta else #** Audio Transcoding - break out the source stream cmsg "Extracting DTS Stream..." $BIN_DIR/nice -n20 \ $BIN_DIR/mkvextract tracks "$SRC_FILE" $AUDIO_TRACK_NO:"${WORKING_DIR}$BASENAME.dts" cmsg "Transcoding DTS To WAV..." $LOCAL_BIN_DIR/dcadec -r -o wavall "${WORKING_DIR}$BASENAME.dts" > "${WORKING_DIR}$BASENAME.wav" cmsg "Transcoding WAV To AC3..." $BIN_DIR/aften "${WORKING_DIR}$BASENAME.wav" "${WORKING_DIR}$BASENAME.ac3" echo "A_AC3, \"${WORKING_DIR}$BASENAME.ac3\", track=1, lang=$PRIMARY_LANGUAGE" >> ${WORKING_DIR}mux.meta fi #** Get tsMuxer on the case... cmsg "tsMuxing To M2TS File..." $BIN_DIR/wine $LOCAL_BIN_DIR/tsMuxeR.exe ${WORKING_DIR}mux.meta $DEST_FILE cmsg "Tidying Up..." #rm -r -f $WORKING_DIR cmsg "Done." #*********************************************************************************** #** Big Thanks To The Following: #** #** http://tinyurl.com/3e9x7n - This is the foundation to my script above - Huge thanks #** http://tinyurl.com/4prnev - I got the x264encopts from here #** http://tinyurl.com/6k6lsb - How to compile x264 and ffmpeg via Ubuntu #** http://tinyurl.com/6k5opc #** Plenty of other sites too... #** #*********************************************************************************** |
6th August 2008, 11:03 | #11 | Link |
Registered User
Join Date: Jan 2003
Location: Parma (Italy)
Posts: 11
|
Very good work!
I can help you improve your script if you want. For example, a lot of MKV in HD are not standard 1280x720 or 1920x1080....you can find many situations, like 1280x5xx etc etc. From what i've seen your script strictly check and strictly wants to use those 2 standard definitions but PS3 can play also different ones without problems.
__________________
By RyLoS |
23rd August 2008, 15:35 | #12 | Link |
Registered User
Join Date: Jun 2008
Posts: 13
|
We'll I've been tinkering and reading even more Doom9 forum posts. Therefore I've tweaked my original script quite alot to make life easier - I hope...
I think I may have cracked the whole FF/Rewind problem thanks to other threads. Anyhow - as before - feel free to comment. Cheers Chuckle |
Tags |
linux, mediatomb, ps3, tsmuxer |
|
|