Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
3rd August 2007, 07:23 | #1 | Link |
Registered User
Join Date: Oct 2001
Location: Germany
Posts: 7,277
|
Why does bobbing double the filesize?
Take an interlaved source and compare a 'normal' yadif()/tdeint() to Tdeint(mode=1) encode and the filesize doubles (with a fixed quantizer). (1pass cq x264)
This might be 'clear' at first view since bobbing (tdeint(mode=1)) doubles the framerate and framecount -> doubling the data doubles the filesize, but shouldn't bobbing add a lot of redundant information which should be easyly compressable with MPEG compression? Do I miss something, shouldn't MPEG compression compensate the framerate/framecount doubling at least a bit? Cu Selur Last edited by Selur; 3rd August 2007 at 11:01. |
3rd August 2007, 12:58 | #2 | Link | |
Freevo Developer
Join Date: May 2007
Location: Waterloo, Ontario
Posts: 118
|
Quote:
Let's suppose your video at frame rate R has X I-frames and Y P-frames. (We'll ignore B-frames for sake of argument but the thinking applies just as well there.) If you encode such that there is a key frame every 250 frames, then encoding at 2R means you'll have twice as many I-frames due to doubling frame rate, so roughly 2X I-frames. The number of I-frames introduced by scene changes will be constant, but if there are many more I-frames due to reaching the keyframe interval, then in 2R we can expect nearly 2X I-frames, less some percentage. Intuitively it seems that the average I-frame size at R will be roughly the same at 2R, so that means roughly twice the space needed for I-frames at 2R. Similarly, I see no reason to believe why encoding a P-frame would use, on average, much fewer bits at 2R than at R. The number of bits needed to store the motion vectors and residual for each macroblock for P-frame Fn at R is probably pretty close to the space needed for frame F2n+1 at 2R. Therefore again, with double the framerate, you'd have roughly 2Y, or twice the number of P-frames and twice the number of bits required for them. Someone who is more clueful with encoding may well contradict me. Last edited by Tack; 3rd August 2007 at 13:03. |
|
3rd August 2007, 13:30 | #3 | Link | ||
x264 developer
Join Date: Sep 2004
Posts: 2,392
|
yadif supports both half-fieldrate and full-fieldrate modes, so your comparison would be more accurate if you used the same deinterlacer for both. yadif and tdeint might have different compressability.
But I'll ignore the fact that there is deinterlacing involved, and just discuss e.g. 25fps vs 50fps content assuming a perfect source. Encode the 25fps content, using whatever settings you want. Now upgrade any disposable B-frames to B-refs, and insert 1 new disposable B-frame between each original frame to get 50fps. B-frames are smaller on average than P-frames, and the new B-frames are smaller even than the B-refs from the 25fps content due to being closer to their references, so the increase in bitrate was less than a factor of 2. In certain high motion scenes this might not be the optimal way to code 50fps - you might get better compression by using only P-frames. But that just further improves the potential bit savings. Why might this not work in practice: x264's B-adapt algorithm was tuned for 24-30fps, so it might not be using enough B-frames even if you increase the max appropriately. iirc xvid got significant compression gains with some tuning for 50fps. Also, x264 doesn't yet support multilevel B-pyramid. Quote:
Quote:
The bits needed for motion vectors are approximately constant per frame, but most of the bits in P-frames go to the residual (otherwise you couldn't change bitrate by changing QP). The magnitude of coefficients in the residual is directly proportional to the amount of change between the reference frame and the current frame. Bit cost is not directly proportional to the magnitude of the coefficients, but it is monotonic increasing. This is why P-frames get bigger as you put more B-frames between them, which is in turn why you don't always use a zillion consecutive B-frames. Last edited by akupenguin; 3rd August 2007 at 13:32. |
||
3rd August 2007, 20:21 | #4 | Link | ||
Registered User
Join Date: Oct 2001
Location: Germany
Posts: 7,277
|
Thanks for all for clearing this a bit up for me.
Quote:
btw. here are the settings I used: Code:
--qp 22 --keyint 500 --ref 6 --mixed-refs --no-fast-pskip --bframes 6 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 7 --analyse all --8x8dct --b-bias 30 --me umh --threads auto --thread-input Code:
SSIM Mean Y:0.9712650 PSNR Mean Y:41.587 U:41.884 V:42.379 Avg:41.686 Global:41.405 kb/s: 4666.06 encoded 1660 frames, 3.83 fps, 4666.15 kb/s Filesize : 38.729.051 Bytes Code:
SSIM Mean Y:0.9780961 PSNR Mean Y:42.743 U:43.639 V:44.221 Avg:43.019 Global:42.477 kb/s: 4227.34 encoded 1660 frames, 5.40 fps, 4227.43 kb/s Filesize : 35.087.634 Bytes Code:
SSIM Mean Y:0.9728272 PSNR Mean Y:42.147 U:42.898 V:43.533 Avg:42.397 Global:42.131 kb/s: 3615.35 encoded 1660 frames, 5.67 fps, 3615.43 kb/s Filesize : 30.008.098 Bytes Code:
SSIM Mean Y:0.9718138 PSNR Mean Y:41.976 U:42.446 V:43.147 Avg:42.145 Global:41.878 kb/s: 7261.71 encoded 3320 frames, 6.34 fps, 7261.81 kb/s Filesize : 60.273.048 Bytes Code:
SSIM Mean Y:0.9756015 PSNR Mean Y:42.387 U:42.803 V:43.377 Avg:42.533 Global:42.193 kb/s: 7052.65 encoded 3320 frames, 0.93 fps, 7052.75 kb/s Filesize: 58.537.819 Bytes Code:
SSIM Mean Y:0.9719284 PSNR Mean Y:41.848 U:42.589 V:43.271 Avg:42.091 Global:41.735 kb/s: 8384.43 encoded 3320 frames, 6.45 fps, 8384.53 kb/s Filesize: 69.591.596 Bytes Code:
SSIM Mean Y:0.9709189 PSNR Mean Y:41.720 U:42.393 V:42.987 Avg:41.948 Global:41.640 kb/s: 7768.99 encoded 3320 frames, 1.87 fps, 7769.08 kb/s Filesize: 64.483.403 Bytes Quote:
Cu Selur |
||
3rd August 2007, 20:29 | #5 | Link |
*Space Reserved*
Join Date: May 2006
Posts: 953
|
What's this? Your modified version of mcbob.
--- nvm, I found out what it was http://forum.gleitz.info/showthread.php?t=15979 Last edited by Terranigma; 3rd August 2007 at 20:32. |
16th March 2008, 01:53 | #6 | Link | |
Registered User
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,197
|
Quote:
I recall fleetingly reading somewhere that the Nero HD profiles are tuned for 50fps. Never tried Nero AVC myself, but just wondering if any of Sharktooths x264 HD profiles are also? If so, can they be used for SD sources as well? All of my 50fps archives are 720 x 544
__________________
Nostalgia's not what it used to be |
|
21st March 2008, 03:09 | #7 | Link |
Registered User
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,197
|
OK, no takers there
Basically my interest stems from the fact that I have accumulated more than a TB of bob-deinterlaced (MVBob and MCBob) 50p footage archived in HuffYUV-YV12 format, but can no longer afford the HDD storage space. I’ve already encoded the footage to XviD (2-pass Q2 equivalent) with good results, but before pressing delete I’d like to archive in at least one more compressed format and preferably one that can be converted and playable in Blu-Ray format in the future. I’ve looked at HDV-HD1 (MPEG2, MP@H-14, 720/576, 50p, 19.7Mbps), based on one of the HD formats available in TMPGenc 4.0 Xpress, but x264 encoded (MeGUI) with ST’s Blu-Ray profile at 20Mbps (roughly equivalent to q20) simply blows it away. Not surprisingly, the x264 encodes struggle a bit on my P4 CPU (3 GHz, 500MB RAM, XP), even with CoreAVC, but I’m thinking they’ll do just fine for playing on a dual-core PC and compatible AVCHD/Blu-Ray set-top in the future. Several ‘burning’ questions now arise: 1. What container should I use for the x264 encoding – mkv, mp4 or raw? 2. Should I resize from 720 x 544 back to 720 x 576 before encoding, for Blu-Ray compatibility? 3. Are there any applications out there that can author the x264 files to Blu-Ray format? I’ve tried a method described over in VideoHelp: http://forum.videohelp.com/topic346069.html It works perfectly well with my HDV-HD1 test clips but I can’t get TSRemux to open the x264 files, at least in the available container formats. From what I’ve read, I think I need to first convert the files into a Blu-Ray compatible (BDAV) MPEG-2 Transport Stream, but cannot find any information on how to do that. Would appreciate any help on the subject. Cheers. Edit: maybe should have posted this over in the Blu-Ray authoring forum but it seemed more relevant to x264. Edit2: Ha, just tried the same method using TSMuxer instead of TSRemux and it works fine; TSMuxer accepts both mkv and rawAVC formats. The Blu-Ray UDF2.5 images burned onto DVD disc play perfectly well on my PC DVD drive, and using 720x544 or 720x576 50p x264 streams. I guess the acid test for permissible resolutions will be test on a Blu-Ray AVCHD set-top player.
__________________
Nostalgia's not what it used to be Last edited by WorBry; 21st March 2008 at 05:03. |
18th May 2009, 23:05 | #9 | Link |
Software Developer
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
|
What do you expect? You give the answer yourself in your very first post: If you double the number of frames, then you must also double the framerate (roughly) to retain the same level of quality. On the one hand you could expect that the bobbed version contains more redundant data, so the bitrate may not need to double exactly. Probably you can get a way with a bit less than the double bitrate. On the other hand deinterlacing/bobbing tends to amplify noise and to create random artifacts -- which is hard to compress. Maybe MDegrain() after bobbing will help here! Last but not least CRF isn't exact as a quality measure! So if the filesize doubles in the bobbed version at the same CRF value, this doesn't mean that you got the exactly same quality in both encodes! That makes it almost impossible to compare the sizes in a useful way. So instead of doing CRF encodes and judging the file sizes, you should do 2-Pass encodes and compare them visually. For example start with giving the bobbed version 200% bitrate of the "normal" one. Then try lowering that successively (e.g. to 175%, 150%, 125%) and check whether these still looks okay or not. This would give some more useful results, I think...
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊ Last edited by LoRd_MuldeR; 18th May 2009 at 23:37. |
19th May 2009, 09:01 | #10 | Link | ||
Registered User
Join Date: Oct 2001
Location: Germany
Posts: 7,277
|
like akupenguin posted:
Quote:
Quote:
Using 2pass and compare the quality would be a good way if I would know a way to objectively judge the quality, but when it comes to small noise differences it seems my perception is changing a bit an a daily basis. -> even if I would have the time to try I would probably change the final conclusion every view days. Cu Selur |
||
19th May 2009, 09:38 | #11 | Link | |
Software Developer
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
|
Quote:
Well, there is no better way. Looking at file sizes isn't helpful. If you see that the file size has (nearly) doubled, what info do you get from that? Twice the file size is perfectly okay, if the you also get twice the quality. But how do you define twice the quality? You can't do that visually. Also CRF (or even worse QP) as an objective quality measure is far from being accurate enough! And metrics like PSNR or SSIM are useless anyway. So what you actually can do is comparing two 2-Pass encodes and decide which one looks better. This can be done visually. Also to judge the effect of bobbing, you'll need to look at it in normal playback speed, I guess. Frame by frame comparison may be misleading here...
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊ Last edited by LoRd_MuldeR; 19th May 2009 at 09:40. |
|
19th May 2009, 16:25 | #13 | Link |
Software Developer
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
|
I think the question was more how much bitrate is required for a "bobbed" (double framerate) encode, compared to a "deinterlaced" (keep framerate) one, and how that bitrate can be lowered. That the bitrate doubles in QP mode isn't very surprising. But only because the bitrate doubles when encoding at a constant quantizer, this doesn't necessarily mean that you actually need that much bitrate to get visually pleasant results. So in the "double framerate" version you could probably get away with a higher quantizer, because the loss will be less noticeable at the higher framerate...
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊ Last edited by LoRd_MuldeR; 19th May 2009 at 19:02. |
19th May 2009, 19:04 | #14 | Link |
Derek Prestegard IRL
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,989
|
I'm always interested in coding double-rate progressive, versus single-rate interlaced. Any chance you could do some tests with encoding both versus a "good" bobber, with whatever tuning you're doing for double rate?
That would be quite interesting ~MiSfit
__________________
These are all my personal statements, not those of my employer :) |
19th May 2009, 20:56 | #15 | Link |
Registered User
Join Date: Oct 2001
Location: Germany
Posts: 7,277
|
My question was: "shouldn't MPEG compression compensate the framerate/framecount doubling at least a bit?"
I was confused since I did some test and saw that I ended up with 98%+ bit rate when bobbing compared to deinterlacing. |
19th May 2009, 21:01 | #16 | Link |
Software Developer
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
|
That's because you encoded at a constant quantizer. If you had encoded at the bitrate that gives similar subjective quality to the non-bobbed version, you'd probably required much less bitrate.
Judging the bitrate (or filesize) without comparing it to the resulting quality is pretty useless. For example: I'd assume that your bobbed version has better visual quality. And in that case +98% bitrate in relation to quality improvement is less than +98% in fact. So you must ask yourself: Is +98% bitrate worth the quality improvement or not ???
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊ Last edited by LoRd_MuldeR; 19th May 2009 at 21:10. |
19th May 2009, 21:58 | #17 | Link |
Registered User
Join Date: Sep 2007
Posts: 5,374
|
I did some quick tests a while back on various deinterlacers and bobbers with x264 crf mode. The results were averaged and normalized to TGMC(select even). These results varied by source material, but the relative trends the same. Disclaimer: this was completely unscientific, just some quick and dirty observations. So don't think these observations would apply to every source material or case...
Personally, I think the extra bitrate required for bobbing is worth it - everything is super smooth and makes single rate deinterlacing look jerky in comparison. Subjectively, TempGaussMC_beta1 and MCBob destroyed the others in image quality. The penalty of course is high CPU usage, and watching paint to dry as the encode completes My hypothesis for these results was that the lower quality deinterlacers left more ugly "jaggies" and deinterlacing remenants that were effectively adding extra noise, increasing the frame complexity and therefore the bitrate required. This was clear from looking the results frame-by-frame later |
20th May 2009, 13:11 | #18 | Link |
Registered User
Join Date: Oct 2001
Location: Germany
Posts: 7,277
|
@LoRd_MuldeR: I think you still don't get what may question was. It was: "Why does x264 double the filesize?" I never asked about quality,...
Since one of the main ideas about MPEG compression (and compression in general) is to benefit from similarities I assumed that using a bobber to double the framerate should not produce 98+% larger files since the bobber should produce really similar images. I know that the images are not identical, but the differences should not result in a size doubling. Since x264 output size is doubled to reasons spring to mind: a. my assumption that a bobber produces really simliar frames during bobbing is wrong or b. x264 doesn't take full advantage of these similarities. Judging from what has been said in the thread so far I assumed: a. the frames a bobber produces are not as similar as I suspected and b. x264 wasn't capable of really taking advantage of such similarities really well, so my post one year later was ment to ask if anything changed regarding x264s ability to take better advantage of these similarities (e.g. through multi-level b-pyramid support). Cu Selur Cu Selur |
2nd July 2009, 10:16 | #19 | Link | |
Professional Lemming
Join Date: Dec 2003
Location: Stuttgart, Germany
Posts: 359
|
Quote:
So 576p25 at 2Mbit had similar visual quality per frame than 576p50 at 2.5Mbit. But the 50p version was clearly the winner due to its motion smoothness I conducted these test with xvid at the time and the (GPL) source must be still floating around on the net as "xvid50p". If not I'll have to dig in my backups of 2006. bis besser, T0B1A5
__________________
projects page: ELDER, SmoothD, etc. |
|
Thread Tools | Search this Thread |
Display Modes | |
|
|