Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 26th May 2010, 11:31   #1  |  Link
Dyomich
Codec Analysis Expert
 
Join Date: Feb 2006
Location: Moscow
Posts: 37
MPEG-4 AVC/H.264 video codecs comparison

Dear doom9 experts,

Moscow State University Graphics & Multimedia Laboratory has finished 6-th H.264 codecs comparison.
It is intended for practical researchers and developers in the field of high-end video compression.



We have tested newest implementations of MPEG-4 AVC/H.264 video codecs and compare with XviD (MPEG-4 ASP) and Theora encoders.
One of the main targets for this comparison was to test H.264 encoders for transcoding tasks for Movies and HDTV video content.

Codec that were tested:
  • DivX H.264
  • Elecard H.264
  • Intel® MediaSDK AVC/H.264
  • MainConcept H.264
  • Microsoft Expression Encoder
  • Theora
  • x264
  • XviD (MPEG-4 ASP codec)
  • VP8

Summary report topics:
  • Objective measurements (SSIM, PSNR, Average Advantage and etc.)
  • Encoding speed
  • Analysis of averaged objective results
  • Leaders in different areas (Movies, HDTV)
  • Options analysis for codecs
  • Additional subjective analysis for video codes
    • For psycho-visual enhancement in codecs analysis
    • For fade processing analysis
    • For animation movie compression analysis
  • Codecs encoding quality progress over years

Enhancements in comparison to Previous H.264/AVC Comparison:
  • Subjective comparison
  • New codecs
  • New sequences
  • New type of special analysis for codecs
  • Using natural sequences' special modifications
  • Using synthetic sequences
  • Not only H.264 Codecs (but also XviD, Theora) were tested

Some examples from the comparison report:

This figure depicts RD-curve for bitrate/quality. Higher curve corresponds better encoding quality. This graph shows quality drop for Theora encoder at 1000kbps.

This figure depicts bitrate handling graph – encoders with good bitarte handling methods has horizontal lines close to 1.0 value. This graph shows strange bitrate handling methods for MS Expression encoder. The more information for it could be found by other graphs analysis.

This figure depicts the progress of the x264 encoder over several years. Y-axis shows encoding quality – encoders with its mark higher than other have better quality. X-axis shows encoding time – encoders with its marks placed to left are faster than other. Therefore encoders in upper-left corner are best – faster and have higher quality than competitors.

More detailed analysis could be found at next page



Best regards,
Dr. Dmitriy Kulikov,
Head of Video Codec Testing team,
Graphics&Media Lab,
Moscow State University

Last edited by Dyomich; 16th June 2010 at 08:19.
Dyomich is offline   Reply With Quote
Old 26th May 2010, 15:51   #2  |  Link
mbohupa
Fanático del DVD-RB
 
Join Date: Jun 2006
Location: Paraguay
Posts: 1
Wow! Great article!!
mbohupa is offline   Reply With Quote
Old 26th May 2010, 20:58   #3  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,988
Thanks for posting this as always, Dr. Kulikov. MSU's comparisons are always thorough and well written. I personally think it would have been interesting to compare some extreme cases with much lower speed requirements, so x264's extra special sauce could be used.

Still, I'm totally unsurprised by x264's dominance. Thank you again.

Derek
__________________
These are all my personal statements, not those of my employer :)
Blue_MiSfit is offline   Reply With Quote
Old 27th May 2010, 07:08   #4  |  Link
Shevach
Video compressionist
 
Join Date: Jun 2009
Location: Israel
Posts: 126
Dr. Kulikov

According to your report only encoders were compared, i.e. Rate Control and Choose Mode.
My question concerns the assessment of decoders, namely the robustness of decoders under bit-stream errors.
Have you any methods to compare H.264 decoders?
Shevach is offline   Reply With Quote
Old 27th May 2010, 14:05   #5  |  Link
Keiyakusha
契約者
 
Keiyakusha's Avatar
 
Join Date: Jun 2008
Posts: 1,576
Quote:
Originally Posted by Shevach View Post
Have you any methods to compare H.264 decoders?
There is nothing to compare except speed. And thats pretty much clear, just read some CoreAVC, DiAVC threads.
Keiyakusha is offline   Reply With Quote
Old 27th May 2010, 14:10   #6  |  Link
nm
Registered User
 
Join Date: Mar 2005
Location: Finland
Posts: 2,641
Quote:
Originally Posted by Keiyakusha View Post
There is nothing to compare except speed. And thats pretty much clear, just read some CoreAVC, DiAVC threads.
Robust handling of corrupted streams is quite important for digital television reception, for example. That's why many people prefer lavc for MPEG-2 decoding.
nm is offline   Reply With Quote
Old 28th May 2010, 06:13   #7  |  Link
PhrostByte
Grand Fruitioner
 
PhrostByte's Avatar
 
Join Date: Mar 2004
Location: Chicago, IL
Posts: 115
Nice article, great job!
PhrostByte is offline   Reply With Quote
Old 28th May 2010, 11:05   #8  |  Link
Dyomich
Codec Analysis Expert
 
Join Date: Feb 2006
Location: Moscow
Posts: 37
Quote:
Originally Posted by Shevach View Post
Dr. Kulikov

According to your report only encoders were compared, i.e. Rate Control and Choose Mode.
My question concerns the assessment of decoders, namely the robustness of decoders under bit-stream errors.
Have you any methods to compare H.264 decoders?
Yes, we have a methodlogy to comapre and analyze H.264 decoders, also we had a project on private MPEG-2 and H.264 decoders comparison with codec developer company.
Dyomich is offline   Reply With Quote
Old 28th May 2010, 11:08   #9  |  Link
Dyomich
Codec Analysis Expert
 
Join Date: Feb 2006
Location: Moscow
Posts: 37
Quote:
Originally Posted by nm View Post
Robust handling of corrupted streams is quite important for digital television reception, for example. That's why many people prefer lavc for MPEG-2 decoding.
Yes, you are right. Error concelament algorithm is very improtant point of any decoder.
You have pointed to one of our previous decoders comparison. Now we have better methodology for decoder comparison.
Dyomich is offline   Reply With Quote
Old 28th May 2010, 11:08   #10  |  Link
Dyomich
Codec Analysis Expert
 
Join Date: Feb 2006
Location: Moscow
Posts: 37
Quote:
Originally Posted by PhrostByte View Post
Nice article, great job!
Thank you
Dyomich is offline   Reply With Quote
Old 28th May 2010, 12:12   #11  |  Link
Raptus
heretic nuB
 
Join Date: Sep 2003
Posts: 71
Quote:
The leader in this comparison is x264—its quality difference (according to the SSIM metric) could be explained by the special encoding option ("tune-SSIM"). Interestingly, using the PSNR metric for MainConcept yielded results comparable with or better than those of x264. This means that no encoder can achieve the best results for both SSIM and PSNR when using the same parameters.
Of course. The requirements for optimizing for each of those metrics (or visual quality for that matter) are not orthogonal. Why omit that x264 can also be tuned for PSNR?

I'm also missing a downloadable PDF version of the free report.

Last edited by Raptus; 28th May 2010 at 12:18.
Raptus is offline   Reply With Quote
Old 28th May 2010, 14:40   #12  |  Link
dapperdan
Registered User
 
Join Date: Aug 2009
Posts: 201
Quote:
Originally Posted by Raptus View Post
Why omit that x264 can also be tuned for PSNR?
I read that quote as meaning that when both encoders were tuned for PSNR, MainConcept won (though it not's clear that's what it actually means).

It seems the for-pay versions have the PSNR graphs included, so if anyone has access to those they could confirm one way or the other.
dapperdan is offline   Reply With Quote
Old 28th May 2010, 16:27   #13  |  Link
Dyomich
Codec Analysis Expert
 
Join Date: Feb 2006
Location: Moscow
Posts: 37
Quote:
Originally Posted by Raptus View Post
I'm also missing a downloadable PDF version of the free report.
We plan to include downloadable version of pdf in few days.
Dyomich is offline   Reply With Quote
Old 28th May 2010, 18:48   #14  |  Link
bob0r
Pain and suffering
 
bob0r's Avatar
 
Join Date: Jul 2002
Posts: 1,337
The moment you posted the results, they are already outdated and x264 has already moved further ahead!
bob0r is offline   Reply With Quote
Old 29th May 2010, 01:41   #15  |  Link
CruNcher
Registered User
 
CruNcher's Avatar
 
Join Date: Apr 2002
Location: Germany
Posts: 4,926
also a more important contender entered the stage after the call was made VP8
__________________
all my compares are riddles so please try to decipher them yourselves :)

It is about Time

Join the Revolution NOW before it is to Late !

http://forum.doom9.org/showthread.php?t=168004

Last edited by CruNcher; 5th July 2010 at 12:38.
CruNcher is offline   Reply With Quote
Old 29th May 2010, 09:17   #16  |  Link
Astrophizz
Registered User
 
Join Date: Jul 2008
Posts: 184
They plan to add VP8 in an appendix.
Astrophizz is offline   Reply With Quote
Old 16th June 2010, 08:19   #17  |  Link
Dyomich
Codec Analysis Expert
 
Join Date: Feb 2006
Location: Moscow
Posts: 37
VP8 has recently attracted a lot of interest after it was owned by
Google. As you know, on May 2010, the WebM Project was launched,
featuring contributions from "Mozilla, Opera, Google and more than
forty other publishers, software and hardware vendors" in a major
effort to use VP8 as the codec for HTML5.

As one of appendixes to the annual H.264 comparison report
an additional VP8 encoder vs. x264 encoder comparison was presented.

We have tested VP8 encoder and compare its encoding quality and
speed with x264.

The final report contains all RD-curves, bitrate handling analysis and
speed/quality graphs. Six different VP8 presets were tested which
were chosen with the help of VP8 developers (so those were VP8
developers guided settings).

http://www.compression.ru/video/code...8_vs_h264.html

Last edited by Dyomich; 5th July 2010 at 10:37.
Dyomich is offline   Reply With Quote
Old 16th June 2010, 17:55   #18  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,666
You really shouldn't allow misleading comments by developers to be posted like that. They claim that artifacts from MPEG compression "benefit H.264 and MPEG" and bias against VP8, but VP8 has almost the exact same transform scheme as H.264, and the same transform size. The "bias" against VP8 should be practically the same as the bias against any other H.264 encoder.

This kind of lie has been spread constantly when it comes to any non-MPEG video format; the same was repeated over and over with Theora, despite the fact that Theora's transform is practically identical to MPEG-1/2/4.

Last edited by Dark Shikari; 18th June 2010 at 18:31.
Dark Shikari is offline   Reply With Quote
Old 18th June 2010, 17:34   #19  |  Link
fields_g
x264... Brilliant!
 
Join Date: Mar 2005
Location: Rockville, MD
Posts: 167
Exactly what I was thinking when I read that statement. I'm glad someone with a bit more technical knowledge spoke up.
fields_g is offline   Reply With Quote
Old 5th July 2010, 10:39   #20  |  Link
Dyomich
Codec Analysis Expert
 
Join Date: Feb 2006
Location: Moscow
Posts: 37
Quote:
Originally Posted by Dark Shikari View Post
You really shouldn't allow misleading comments by developers to be posted like that. They claim that artifacts from MPEG compression "benefit H.264 and MPEG" and bias against VP8, but VP8 has almost the exact same transform scheme as H.264, and the same transform size. The "bias" against VP8 should be practically the same as the bias against any other H.264 encoder.

This kind of lie has been spread constantly when it comes to any non-MPEG video format; the same was repeated over and over with Theora, despite the fact that Theora's transform is practically identical to MPEG-1/2/4.
Yes, you are probably right, but our comparison rules permit developers to post their comments to comparison.

Last edited by Dyomich; 8th July 2010 at 09:16.
Dyomich is offline   Reply With Quote
Reply

Tags
codec, comparison, h.264, mpeg-4, msu

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 19:56.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.