Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > DV

Reply
 
Thread Tools Search this Thread Display Modes
Old 13th March 2008, 22:23   #1  |  Link
mattsmith321
Registered User
 
mattsmith321's Avatar
 
Join Date: Mar 2008
Location: Austin, TX
Posts: 14
Encoding DV for backup

Hi all!

I've avoided writing this post for a while, but I've kind of reached a point where I need a push in the right direction. This post is actually very similar to the recent Max Quality & Archival Purposes thread in the DivX forum but I wanted to give my specific requirements and not hijack that thread.

I've got ~25 mini DV tapes worth of family home video (I know, boring!) that I've recorded over the past ~10 years that I would like to:
  • Get off of tape and onto my computer in a format that
    • Works with my backup strategy - I have my primary hard drive in the computer that I sync with two external hard drives that I rotate on/off site every couple of weeks.
    • Is easily shareable - I'd like to be able to burn the files to a DVD (not necessarily make a DVD) or be able to take one of my external backup drives to my extended family and copy locally.
    • Can be encoded to other formats - Every now and then I'll have a clip that I either want to upload to YouTube or encode for my iPod.
    • Works in consumer video editing software - Every now and then I'll go the extra mile to actually edit the clips into a video with a soundtrack, etc.

So, the first part is easy. I use WinDV to get the video from the tape and onto my computer into individual timestamped clips based on scene change/discontinuity threshholds. What I end up with is ~12GB of data and ~30 - 60 files (per hour / tape). Doing the math for ~25 tapes, we end up with ~300GB of data and 750 - 1500 files. While I do agree with Ron Spencer that keeping the data in DV format is ideal and hard drives are fairly cheap, it just isn't practical to replicate 300GB of video + 10GB of images + 50GB of music. Yes, I know that 500GB drives are common now, but I'm fine with losing some quality at the expense of convenience. We do that with images (JPG vs RAW) and music (MP3 vs FLAC), etc.

What I need is a process or tool that can batch encode a list or directory of files to about 20% of the size (2.4GB/hour) or rate (6000kbps) of the original format. I came up with the 20% number through a little trial and error and it seems reasonable: 12GB * 0.20 = 2.4GB per hour which would allow about 2 hours (4.5GB) on a DVD.

Every few months for the past couple of years I've taken a stab at this only to give up in frustration. I've learned a lot in the past two years. I went from just pulling everything with MovieMaker into one semi-decent sized WMV file that looked like crap to finding out about all of these great open source tools. Unfortunately, I'm only able to grok the GUI tools (AutoGK, AutoMKV, RipBot, MediaCoder) but can't quite make the leap to CLI (or even MeGUI).

At this point, after having experimented with H.264 (or x264), I don't think that I want to go that route primarily because it is slightly less shareable (I know, they all require a codec, but most of my family would not be on H.264 yet) but mainly because the editor support was pretty bad with MovieMaker and Premiere Elements. Sidenote: Sure editors will get there and I should probably be using better editors but we're not there and I'd like to let me kids use the easy tools also.

At this point, I'm down to the following:
  • MediaCoder - I like the fact that it supports batch and I could use one tool to encode to this archive/backup format and then also use it to go to iPode format, etc. However, I am having serious de-interlacing issues that I haven't been able to get past.
  • AutoMKV - I really like this tool but I'm having trouble getting some of the profiles to come out like I expect.
  • AutoGK - Consistently easy but doesn't do batch.

Sorry for the long post. It's kind of a rant or outlet for my frustrations for not being able to move past this. As a programmer, I tend to get stuck and can't move past certain things. It seems like there should be something that is easy like Windows Media/Movie Maker/Premiere Elements/Quick Time but with better quality; but not as complicated as some of the existing options.

I think what I want is the equivalent of WinDV (minimal options) that can encode a directory of DV home video to a near-lossless quality (that the experts here agree is ideal for quality vs. space) that is easier to backup and use and share. Alas, I know that as soon as you hide the options, someone will want the ability to tweak things which gets us right to where we already are.

Anyone have any profiles for MediaCoder or AutoMKV that they use to backup their DV?
__________________
Matt Smith
mattsmith321 is offline   Reply With Quote
Old 14th March 2008, 03:14   #2  |  Link
scharfis_brain
brainless
 
scharfis_brain's Avatar
 
Join Date: Mar 2003
Location: Germany
Posts: 3,653
I'd suggest you to use MPEG2 for archiving your DV-Videos.
Procoder has very nice quality and also a built in batch processing feature.
__________________
Don't forget the 'c'!

Don't PM me for technical support, please.
scharfis_brain is offline   Reply With Quote
Old 14th March 2008, 03:46   #3  |  Link
Video Dude
Senior Member
 
Join Date: Apr 2004
Posts: 1,053
I would do all of the below to preserve home movies that can't be recreated:

Encode high bitrate MPEG-2 (DVD). Burn two copies: Verbatim 16x MCC-004 (PAPA) and TY Premium 8x DVD+R. Give one copy to a family member or friend offsite.

Store DV files on external WD or Seagate 400 or 500 GB. Cost less than $100 and would take an only an hour to two to copy. (A lot easier than all that encoding and lets you access the the raw files later on to convert to future formats.)

Keep all the DV tapes.
Video Dude is offline   Reply With Quote
Old 14th March 2008, 06:44   #4  |  Link
mattsmith321
Registered User
 
mattsmith321's Avatar
 
Join Date: Mar 2008
Location: Austin, TX
Posts: 14
Thanks guys. MPEG-2 hasn't really even been on my radar until today. I think I was caught up in the hype around DivX, Xvid and x264. Obviously those are the better formats if I'm looking for smaller file sizes, but since I've got some flexibility...

Off to do some more research and testing!
__________________
Matt Smith
mattsmith321 is offline   Reply With Quote
Old 14th March 2008, 15:00   #5  |  Link
mattsmith321
Registered User
 
mattsmith321's Avatar
 
Join Date: Mar 2008
Location: Austin, TX
Posts: 14
Great. The more I read, the less I know. The thread, Is it harder to encode DV into MPEG2?? was a very enlightening read. It seems I have another learning curve in front of me.

I have yet to look at ProCoder since I would have to purchase it. It feels odd going from all open-source for the Xvid and x264 tools to having to pay to do something in an older format.
__________________
Matt Smith
mattsmith321 is offline   Reply With Quote
Old 14th March 2008, 15:13   #6  |  Link
Southstorm
20yr Videographer
 
Join Date: Jan 2008
Location: The Great Northwest
Posts: 272
ProCoder is good, but so is the free HCenc. My brother and I have been making amature movies from a DV camcorder with an Mpeg2 output for years. With the proper bitrate and settings, you can achieve quality results. We didn't get it right the first time, it took some experimenting and tweaking to say the least.
Southstorm is offline   Reply With Quote
Old 14th March 2008, 16:21   #7  |  Link
smok3
brontosaurusrex
 
smok3's Avatar
 
Join Date: Oct 2001
Posts: 2,392
there is a difference between pal DV and ntsc DV in terms of how chroma is sampled, afaik both matrixes are incompatible with how mpeg2 stores chroma (but pal fits in a bit better).
__________________
certain other member
smok3 is offline   Reply With Quote
Old 14th March 2008, 17:15   #8  |  Link
scharfis_brain
brainless
 
scharfis_brain's Avatar
 
Join Date: Mar 2003
Location: Germany
Posts: 3,653
There is no other format than DV AFAIK, that stores the weird chroma sampling natively.
__________________
Don't forget the 'c'!

Don't PM me for technical support, please.
scharfis_brain is offline   Reply With Quote
Old 14th March 2008, 17:58   #9  |  Link
ron spencer
DVD Magistrate
 
Join Date: Dec 2003
Location: Sodor
Posts: 991
hard drives are cheap....cheap...just store DV...simple easy


autogk will batch

Last edited by ron spencer; 14th March 2008 at 18:00.
ron spencer is offline   Reply With Quote
Old 14th March 2008, 18:09   #10  |  Link
JohnnyMalaria
Registered User
 
Join Date: Sep 2006
Posts: 602
Quote:
Originally Posted by smok3 View Post
there is a difference between pal DV and ntsc DV in terms of how chroma is sampled, afaik both matrixes are incompatible with how mpeg2 stores chroma (but pal fits in a bit better).
Yes - PAL uses 4:2:0 and NTSC uses 4:1:1.
JohnnyMalaria is offline   Reply With Quote
Old 14th March 2008, 19:31   #11  |  Link
mattsmith321
Registered User
 
mattsmith321's Avatar
 
Join Date: Mar 2008
Location: Austin, TX
Posts: 14
Quote:
Originally Posted by ron spencer View Post
autogk will batch
Can you provide a little more detail on that. I haven't run across that so far.
__________________
Matt Smith
mattsmith321 is offline   Reply With Quote
Old 14th March 2008, 19:54   #12  |  Link
mattsmith321
Registered User
 
mattsmith321's Avatar
 
Join Date: Mar 2008
Location: Austin, TX
Posts: 14
Quote:
Originally Posted by ron spencer View Post
hard drives are cheap....cheap...just store DV...simple easy
I don't disagree with you about that. I do have an issue about maintaining that much data when it probably wouldn't be used that much.

I think I'm going to back off my requirement about needing to work with the encoded videos in editing software. I think at this point in time, it is more important to me to have encoded/compressed versions of the files so that I can a) keep a copy of everything on my main hard drive so that my wife and kids have easy access, b) import into Adobe Photoshop Elements so that the clips can be organized/tagged/categorized with my pictures,and c) easily keep backup copies on my current 250GB backup hard drives.

If and when I ever have a need to put a project together and I find that the encoded versions aren't working or are too lossy, then I can always go back to tape. Yes, it will be a little bit of a pain to re-import, but that is a trade-off I can live with.

So, I will probably end up with the following:
- Import with WinDV from tape to date-stamped clips
- Encode/compress clips to ~6-8Mbps (still need to decide MPEG-2/XviD/DivX)
- Delete original clips
- Import encoded clips into Adobe Photoshop Elements
- For iPod or lo-fi (youtube, online, etc.) needs, use the encoded clips.
- For projects where quality is a concern, re-import the original clip.
- At some point, I will probably copy the DV tapes to new tapes to keep them fresh.

I'm not a videophile so I have no interest in obsessing over the differences between chroma, quants, mux, etc. My family did lose everything when I was growing up so I am especially cognizant of keeping separate physical backups at different locations. That, and being able to have better access to the clips is important.

Thanks for everyone's help so far. I really do appreciate it and admire those of you that understand and have to deal with all of the nuances associated with video and audio.
__________________
Matt Smith
mattsmith321 is offline   Reply With Quote
Old 14th March 2008, 23:54   #13  |  Link
Southstorm
20yr Videographer
 
Join Date: Jan 2008
Location: The Great Northwest
Posts: 272
You already know my vote is for Mpeg2, and as far as editing goes, Cuttermaran is awesome for Mpeg2.
Southstorm is offline   Reply With Quote
Old 15th March 2008, 00:40   #14  |  Link
scharfis_brain
brainless
 
scharfis_brain's Avatar
 
Join Date: Mar 2003
Location: Germany
Posts: 3,653
yeah, MPEG2 should be your choice, if you need to compress your videos.

it is compatible and standardised.
not interlacing issues etc.
for divx/h.264 there is no good throughout support over all sorts of applications.
also mpeg2 is easily being 'reformatted' for DVD-Player compatible DVD-Video-Discs.
divx and h.264 will need re-encoding.
__________________
Don't forget the 'c'!

Don't PM me for technical support, please.
scharfis_brain is offline   Reply With Quote
Old 15th March 2008, 15:01   #15  |  Link
ron spencer
DVD Magistrate
 
Join Date: Dec 2003
Location: Sodor
Posts: 991
there is one BIG problem with MPEG-2...suppose in 5 years you want to convert them to HD? You will need to go back to your tapes if you have them....MPEG-2 at 740x480 to HD just does not work....you compress twice.

for autogk....just add job, then go back and load another video file...click add job...over and over....there you batch
ron spencer is offline   Reply With Quote
Old 15th March 2008, 16:13   #16  |  Link
JohnnyMalaria
Registered User
 
Join Date: Sep 2006
Posts: 602
Quote:
Originally Posted by mattsmith321 View Post
So, I will probably end up with the following:
- Import with WinDV from tape to date-stamped clips
- Encode/compress clips to ~6-8Mbps (still need to decide MPEG-2/XviD/DivX)
- Delete original clips
- Import encoded clips into Adobe Photoshop Elements
- For iPod or lo-fi (youtube, online, etc.) needs, use the encoded clips.
- For projects where quality is a concern, re-import the original clip.
- At some point, I will probably copy the DV tapes to new tapes to keep them fresh.
About the two things I have emboldened above:

What do you mean by date-stamped? If you mean the date information is in the file but not burned onto the video itself, *any* DV capture application will do that. If you want the date on the video permanently, WinDV won't do that. You have to use it in conjunction with another program to add the date afterwards or use a separate program that does both simultaneously. Which is better depends on exactly what you are after.

Copying tapes to new DV tapes - everything on the tape is duplicated EXCEPT for the timecode. So, if your original captured information is in any way dependent on the timecode, you may have a problem down the road. e.g., if your editing program has created clips and noted the timecode so you can recapture them at a later date then the copy of the tape won't work properly.
__________________
John Miller
Enosoft DV Processor - Free for personal use
JohnnyMalaria is offline   Reply With Quote
Old 15th March 2008, 21:21   #17  |  Link
mattsmith321
Registered User
 
mattsmith321's Avatar
 
Join Date: Mar 2008
Location: Austin, TX
Posts: 14
Quote:
Originally Posted by scharfis_brain View Post
yeah, MPEG2 should be your choice, if you need to compress your videos.

it is compatible and standardised.
not interlacing issues etc.
for divx/h.264 there is no good throughout support over all sorts of applications.
also mpeg2 is easily being 'reformatted' for DVD-Player compatible DVD-Video-Discs.
divx and h.264 will need re-encoding.
After do some more reading and thinking about it, I definitely do agree that going to MPEG2 is probably the best approach. Now, I'm in a bit of a quandary on how to go about that. The MPEG2 Encoder Test thread was very informative regarding my options. Everybody seems to really like ProCoder. However, $200 is a little more than I was wanting to spend. I have no doubt that it is a quality tool but much of it's functionality would be lost on me. I thought I had found the perfect solution at $60 in ProCoder Express (this review was great) but alas, that isn't sold anymore and the functionality has been integrated into ProCoder 3.

I will be taking a look at the tools listed here and elsewhere to see what I can come up with.
__________________
Matt Smith
mattsmith321 is offline   Reply With Quote
Old 15th March 2008, 21:33   #18  |  Link
mattsmith321
Registered User
 
mattsmith321's Avatar
 
Join Date: Mar 2008
Location: Austin, TX
Posts: 14
Quote:
Originally Posted by ron spencer View Post
there is one BIG problem with MPEG-2...suppose in 5 years you want to convert them to HD? You will need to go back to your tapes if you have them....MPEG-2 at 740x480 to HD just does not work....you compress twice.
I'll take your word on that. However, that would still be an issue if I went to the other formats (Xvid, x264, etc.), correct? By going MPEG-2 I would at least get the benefits that scharfis_brain laid out. Regardless, hopefully I am still out five years or so before I get on the HD bandwagon.

Quote:
Originally Posted by ron spencer View Post
for autogk....just add job, then go back and load another video file...click add job...over and over....there you batch
Yes, I guess that does quality as batch but was looking for something batch-ier. But I guess I can't use AutoGK now that everyone has convinced me to go MPEG2.
__________________
Matt Smith
mattsmith321 is offline   Reply With Quote
Old 15th March 2008, 21:40   #19  |  Link
mattsmith321
Registered User
 
mattsmith321's Avatar
 
Join Date: Mar 2008
Location: Austin, TX
Posts: 14
Quote:
Originally Posted by JohnnyMalaria View Post
What do you mean by date-stamped? If you mean the date information is in the file but not burned onto the video itself, *any* DV capture application will do that. If you want the date on the video permanently, WinDV won't do that. You have to use it in conjunction with another program to add the date afterwards or use a separate program that does both simultaneously. Which is better depends on exactly what you are after.
I just meant that I let WinDV name each clip according to the tape date info. I use the following filename formatting: %Y-%m-%d. Most of the stuff in my head is organized chronologically, so I like having all of my clips named according to the date of the clip. The fact that WinDV could do that and split the clips based on a discontinuity threshold were exactly what I was looking for.

Quote:
Originally Posted by JohnnyMalaria View Post
Copying tapes to new DV tapes - everything on the tape is duplicated EXCEPT for the timecode. So, if your original captured information is in any way dependent on the timecode, you may have a problem down the road. e.g., if your editing program has created clips and noted the timecode so you can recapture them at a later date then the copy of the tape won't work properly.
That is interesting (and will probably be annoying). Thanks for the heads-up. If it's not one thing, it's another. You just gave Ron another reason for why I should keep the original clips. He must work for one of the hard drive manufacturers. Just kidding.
__________________
Matt Smith
mattsmith321 is offline   Reply With Quote
Old 16th March 2008, 03:21   #20  |  Link
Southstorm
20yr Videographer
 
Join Date: Jan 2008
Location: The Great Northwest
Posts: 272
Have you checked out HCenc? Its a very good Mpeg2 encoder, and there are a few gui's that work with it for batch converting.
Southstorm is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 06:17.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.