BPG Image Format (HEVC subset for browsers)

[Nintendo Maniac 64]

It's like asking the same thing about audio formats and MP3 - even though better things have been made, the incumbent was already "good enough".

So much like Vorbis and AAC, the likes of better lossy image formats will likely be limited to specific niches (Vorbis is used in games, AAC, outside of Apple, is used with video).

[gamebox]

You might be right Atak_Snajpera, but that's a personal level of view. Modern websites, for instance, contain more visual material than ever before and in ever increasing resolutions and sizes. Faster hardware today means it's easier to navigate through large websites, and that means even more material served. Traffic load is probably exponentially larger per user than before, and today more people use internet and on various devices too, so total traffic must be even greater. I wonder how much big sites like Facebook, Google, etc would benefit from newer image technologies serving the same quality with a portion of the bitrate.

When talking about JPEG2000 it's a different story. I've tried it (and many others) myself once I needed to cram lots of large pictures on a single DVD disk, and gave up on it quickly. It simply has severe quality issues (some even when not filesize-constrained) and savings in file size are not as big as you would expect from an advanced format coming after such advancements in technology as we've had in recent decades. BPG (HEVC) on the other hand, does live up to expectations.

Another thing I forgot - HEVC seems to have (postprocessing) tools to (finally) get rid of compression artifacts present in all other older image and video formats. It is really a welcome change to see it in video, and the images would really benefit from it too. If I'm not mistaking, some of those artifacts are present at milder forms even when the file is saved in a very high quality in older formats - and that becomes visible when zooming or printing.

[Atak_Snajpera]

Currently streaming media (youtube,vimeo,soundcloud and so on) generate majority of traffic. JPEG images are not a big problem like in 90s.

[pieter3d]

Originally Posted by gamebox

HEVC seems to have (postprocessing) tools to (finally) get rid of compression artifacts present in all other older image and video formats.

What are you referring to here? Deblocking / SAO? This isn't really post processing, although I guess it ends up being the same thing if you just decode a single still image.

Note that HEVC's deblocker is nearly identical to H.264.

[foxyshadis]

So far not much else has stepped up to replace PNG for lossless & alpha, though. PNG files are not only huge, zlib is very slow to decode and PNG's basic filtering wasn't designed for parallelism; all together, lossless HEVC can actually be faster on large images. WebP wasn't actually designed to improve JPEG, it was more of an engineering project to justify buying On2. If any company other than Google was behind it, it'd have been forgotten after the first press release.

JPEG XR is slowly gaining traction, built into Windows and Flash Player and some editing apps now, and actually supports everything from lossless to fancy colorspaces. We'll see how it evolves. It may not have the extreme compression that HEVC's excellent I-frame format gives, but being a real royalty-free standard is a big advantage.

Originally Posted by gamebox

I don't understand something. If HEVC (and H264 before) is developed as the most advanced visual compression and aimed at widest device support, why is it not "diversified" into still-image use as well - by the creators of the standard themselves? They could just define a subset of compression techniques used for that purpose (only intra tools), and (video) decoder chips could (I guess) easily be adapted to still-image use too without any limits on the resolution, as processing speed, decoded image buffer size and data throughput (bitrate) wouldn't be critical there.

They did, finally. The MP in MPEG is for "Moving pictures;" the video engineers who developed early standards weren't considering their use for anything else, and up until H.264 the I-frame formats were basically JPEG anyway, so there wasn't a point. It wasn't until Google and Microsoft turned the I-frames of their codecs into actual image formats that MPEG & ITU finally started to think about that need. (JPEG XR is from WMV and WebP is from VP8.) HEVC had a "Still Picture" format almost from the beginning thanks to this, but it's relatively limited compared to competing standards like JPEG XR (more like WebP). They punted all the higher-fidelity stuff to the Range Extensions group, which finished its standard this year, adding full fidelity 4:4:4 16-bit in either RGB or YUV.

The Still Image flag doesn't mean anything to the bitstream, it just tells the program not to treat it like a video. (No play buttons or any of that junk.) That doesn't mean hardware gets to ignore size limitations or DPB, but at least it's very relaxed compared to video, and fallback to software is easy.

Originally Posted by gamebox

MPEG-2 decoding chips easily handled audio (MP3), JPEG files too (it probably used quantization/dct techniques similar to H261/H262). With HEVC it seems the only way to use the technology for still-images on widespread hardware is to create some formal video clip containing all intra frames (as with still-image "galleries" on a DVD), which is an unnecessarily harder and impractical way to do the same thing (that way all the pictures have to be scaled to the same resolution, the maximum resolution is limited (perhaps even the available bitrate/macroblock count), the device needs to have frame-advance commands for the user, etc.).

With BPG and similar efforts it seems to me we just create another "unofficial" format, unsupported, not well known, without hardware support whatsoever, and as such it seems bound to fail right from the start despite all the quality and advancements.

Dunno why you'd think that the only way to use HEVC is as a video of intra frames. They'd be used in separate files or in a larger container document just like any other image format. BPG literally is HEVC with a few reversible tweaks to reduce the signalling overhead and add third-party extensibility, so any images saved in a hardware-compatible format will still be hardware-compatible. (Assuming hardware compatibility is a speed win; JPEG isn't accelerated anymore because it decodes too fast on the CPU.)

I think this is more of a "get people talking" format than a real contender, unless some company adopts it and pushes it into international standardization. The patent issues will dog it otherwise, since JPEG, JXR, and PNG are all royalty-free. It's mainly of interest to bandwidth- or size-constrained programs, and the mobile world is still full of that.

[xooyoozoo]

Originally Posted by easyfab

Yes I also use back-forth ( Xnview or Irfanview ) but for an area comparison this split method ( with mouse mouvement ) from xooyoozoo page is really interesting.

I would like something like this http://exp.martres.me/splitview/ but for image where you can load your own local file.

I made an attempt with drag-n-drop. It probably has some problematic corner cases, but at least reloads are easy.

[gamebox]

Originally Posted by foxyshadis

up until H.264 the I-frame formats were basically JPEG anyway

Hm, I thought H263 quantization of MPEG-4 ASP was somehow different than the ones used in previous formats, and that quantization itself was actually responsible for better compression (keeping less space-consuming high contrast/frequency details than MPEG-2, but still more than only the "softest" ones MPEG-1 keeps).

Originally Posted by foxyshadis

Dunno why you'd think that the only way to use HEVC is as a video of intra frames. They'd be used in separate files or in a larger container document just like any other image format.

That's because I thought there was not going to be Still-image use of HEVC, at least planned by the creators of the standard. Perhaps the thing we need then is the right container format itself and a widely recognized extension - JPEG files have their (recognizable) extension and format, JPEG2000 does too, etc. Perhaps we need something like MKV or PNG here.

Originally Posted by foxyshadis

any images saved in a hardware-compatible format will still be hardware-compatible.

BPG having H265 quantization is HEVC hardware-compatible, but the container (and file extension) itself has to be supported and handled by the hardware too, right? PNG is a worldwide format, MP4 is an universal container for video, MKV is well-known too and supported likewise, but BPG seems to me like one-man (and novel) effort which will not gain worldwide hardware support (for the time being at least).

As an offtopic, I also hoped company behind (new) WinZIP format would create some worldwide format/extension for "single additionally compressed JPEG", like ZJP or so, so you could view the image directly instead of handling the archive. That would make their compression really usable in everyday life - currently it is only good for creating image archives for rare and "offline" use.

@pieter3d Yes, I'm refering to Deblocking / SAO, but also 4x4 transform from H264 that removes "mosquito" noise.

[foxyshadis]

Originally Posted by gamebox

Hm, I thought H263 quantization of MPEG-4 ASP was somehow different than the ones used in previous formats, and that quantization itself was actually responsible for better compression (keeping less space-consuming high contrast/frequency details than MPEG-2, but still more than only the "softest" ones MPEG-1 keeps).

H.263 quant is just a little better for much lower bitrates, being softer than blockier. But true, MPEG-4p2 did offer very rudimentary block prediction, allowing predicting the block average from the top or left block, instead of just the left block like MPEG-1 & 2, as well as a much less useful first-row AC predictor. Against JPEG, there would only be noticeable gain at very low bitrates (but you could losslessly convert a JPEG to MPEG-4 ASP's I-frame and back, with a few tweaks to the matrix, because it's all frequency domain prediction).

MPEG 4p10's arbitrary spatial prediction were the first that made an appreciable dent in I-frame sizes, which is why they were expanded further for HEVC.

BPG having H265 quantization is HEVC hardware-compatible, but the container (and file extension) itself has to be supported and handled by the hardware too, right? PNG is a worldwide format, MP4 is an universal container for video, MKV is well-known too and supported likewise, but BPG seems to me like one-man (and novel) effort which will not gain worldwide hardware support (for the time being at least).

Players rarely send the container to hardware, they read it, split out the raw video, and upload that. All H.264 hardware I know of works that way. Even if the hardware can read some containers, only the laziest players wouldn't accept anything but hardware-supported containers, because hardware can always accept raw bitstreams. (Though some old chipsets could only read AVI with Divx and MP3.) TVs and set-top boxes that can't be updated blur the line, but that's consumer electronics for you-- the refresh cycle is king.

And don't forget that PNG and MKV started out as one-man hobby projects. Sometimes they catch on, sometimes they don't.

[gamebox]

Originally Posted by foxyshadis

And don't forget that PNG and MKV started out as one-man hobby projects. Sometimes they catch on, sometimes they don't.

Yes, but I was wondering if there was, for example, something like "MKP" - Matroska Picture container, if we already have MKV for video. I guess hardware support could easily be extended then, as something like that would have that "family of formats" or "sounds familiar" feel to it, and of course be open-source too. Perhaps PNC too (Portable Network Continous-tone image) as a "brother" of PNG?

With BPG (HEVC) for the first time the pictures have that "natural" feel to me. No blocking, no mosquito noise/ringing, no difference between "important" and "less important" parts and details in a picture. Excellent compression is a plus. JPEG-2000 had zones with obviously lowered resolution as a result of wavelet transform, and many "advanced" formats also made distinct visual difference between sharp zones with higher-frequency details and very "softened" ones with low-frequency "background". Sort of like the treatment of information you see in DJVU format - sharp vector-graphic lettering and blurred background. With HEVC loss of details is "gracefull" - affected parts of pictures don't appear blurred or blocky, they seem as they were "simple" and "continuous" to start with. And most of all, HEVC gets rid of "feathery" like look of H264 - that's what I call areas where high and low frequency details mix so H264 uses 4x4 and 16x16 blocks alternatively. You get a sharp edge of something as 4x4 transform always gives you, then just a few pixels away the very same edge gets blurred and dissolved by 16x16 transform, so after applying the deblocking on that the edge it ends up looking like the washed-out material "fleece", also waving left and right and losing shape even if it was actually geometrically precise. That's probably the biggest low-bitrate improvement I noticed in HEVC, and I guess we owe that to SAO.

[easyfab]

Originally Posted by xooyoozoo

I made an attempt with drag-n-drop. It probably has some problematic corner cases, but at least reloads are easy.

Thanks xooyoozoo, It work good here and very easily with drag and drop. Also nice to have flip mode and split mode in same tool .

[foxyshadis]

Originally Posted by gamebox

Yes, but I was wondering if there was, for example, something like "MKP" - Matroska Picture container, if we already have MKV for video. I guess hardware support could easily be extended then, as something like that would have that "family of formats" or "sounds familiar" feel to it, and of course be open-source too. Perhaps PNC too (Portable Network Continous-tone image) as a "brother" of PNG?

I used .mki and .m4i as analogues to .mka and .m4a, but that's just me.

Originally Posted by gamebox

Sort of like the treatment of information you see in DJVU format - sharp vector-graphic lettering and blurred background.

Of topic, but damn, I hate this so much. MrSID seems to be a really terrible encoder; I regularly see text completely blurred out in old maps I request from libraries. Region of interest coding is extremely important, but it seems like a lot of institutions use a perfect/horrible dichotomy instead of gracefully degrading, or at least keeping the background ok instead of horrible, for when it does screw up. Text at 600 DPI and background at 72 or less is just too much.

[ruesberg]

libbpg-0.9.4 is out. Extract of the changelog:

version 0.9.4:

- Modified alpha plane encoding to allow progressive display and
streaming encoding. This change is incompatible, so images
containing alpha from the previous versions of the format cannot be
decoded.

- Added 4:2:2 and 4:2:0 chroma formats with MPEG2 chroma sample position.

http://bellard.org/bpg/libbpg-0.9.4.tar.gz

[Nintendo Maniac 64]

Hey, perfect for comparing to the newest Daala build that just came out today!

Originally Posted by Kurtnoise

A windows binary from today...

The changelog available here.

[djcj]

Static Linux binaries (32 and 64 bit) with x265 support: http://www.mediafire.com/?muwfwcucsglr217
In case someone has trouble compiling them.

[xooyoozoo]

In case this is of use to some, I'll share my personal fork of libbpg with a few quality-of-life additions to encoding flags.

CRF 18 (as opposed to CQP 18) encode with weaker deblocking, higher quality chroma:

-q 18 -deblocking -2 -chroma_offset -3

Try to produce output within 3% of 200kB using at least 2 and up to 8 total passes:

-s 200 -size_tol 3 -max_passes 8

CRF 18 encode. Use output if <200kB (+ default tolerance), else try up to (default # passes) to get 200kB.

-q 18 -s 200 -size_limit

[easyfab]

Hi xooyoozoo,

Thank you for sharing, I made some little tests.
I build under msys/mingw-w64 x64
I changed makefile with
CONFIG_WIN32=y
CROSS_PREFIX:=x86_64-w64-mingw32-
#USE_JCTVC_HIGH_BIT_DEPTH=y
and removed -fno-asynchronous-unwind-tables from CFLAGS for correct build ( don't know why this cause error with x64 ? it's the same with original 0.9.4 ).

Now for your build I have this with -e JCTVC :
***************************************************************************
** WARNING: Transform skip fast is enabled (which only tests NxN splits),**
** but transform skip log2 max size is not 2 (4x4) **
** It may be better to disable transform skip fast mode **
***************************************************************************
and a corrupt image :


With my build of original 0.9.4 it's ok.

With -e x265 all is ok but setting quality -q is now difficult/different between JCTVC/X265 ( your change CQP -> CRF ) for example -q 18 for -e JCTVC and original -e x265 give ~340kB with your -e x265 -q 18 ( crf 18 ) ~170kb therefore same as -q 25 for JCTVC
If possible I think it would be better to keep -q -> CQP and create a new option -crf for crf

I will try multipass latter.

[xooyoozoo]

Originally Posted by easyfab

Now for your build I have this with -e JCTVC :
***************************************************************************
** WARNING: Transform skip fast is enabled (which only tests NxN splits),**
** but transform skip log2 max size is not 2 (4x4) **
** It may be better to disable transform skip fast mode **
***************************************************************************
and a corrupt image :

Whoops missed that. Thanks.

With -e x265 all is ok but setting quality -q is now difficult/different between JCTVC/X265 ( your change CQP -> CRF ) for example -q 18 for -e JCTVC and original -e x265 give ~340kB with your -e x265 -q 18 ( crf 18 ) ~170kb therefore same as -q 25 for JCTVC
If possible I think it would be better to keep -q -> CQP and create a new option -crf for crf

I will try multipass latter.

Hmmm I'll look into it.

I don't see CQP as a serious mode because it also turns off CUTree, but the straightforward solution would be to turn off CUTree at 0 aq_strength. That would automatically put x265 into CQP mode. Even then, however, the original settings still relied on magical, arbitrary offsets (which I've removed) to match JCTVC and x265, and there's no reason not to make our own magic with CRF.

Edit:

I was wrong about CUTree being useful. Either it only works in inter prediction, or intra propagation of quality doesn't extend much more than a couple of CUs. Turning CUTree off now means that x265 CRF matches JCTVC QP a bit better, but it's still not 100% equal.

Also, color-x265 + alpha-jctvc is now possible again. JCTVC's alpha qp is now obtained from x265's output's headers when needed.

[xooyoozoo]

I put up a comparison between default x265-based BPG and the same but with things like Psy-Rd activated. You'd probably prefer pressing Shift to switch between versions.

To my eyes, broadly-textured, "scenic" images like Machu Picchu, Isle of Skye, and Eaglefairy benefitted the most. Localized, finely detailed areas are next in line (the statue surface in Nymph, the girl's outfit in Production, the wings in Swallowtail).

What stops this from being an unmitigated improvement is the occasional artifacts popping up in extremely flat regions (background edges in Fruits and Ballet Exercise). I had to lower psyrd slightly because it was even worse before. Psyrdoq seems to cause less issues, at least at low single digits. Meanwhile, deblocking actually barely made a visual quality dent, and chroma qp was only changed to stop haphazard moments of chroma noise with psyrd+psyrodq is on.

[smok3]

xooyoozoo; looking at your example page, to my eyes psy wins with scenic images, portraits are always worse (But this may be a bit over-compressed for that purpose).

p.s. Trying to compile libbpg on osx .... (did it with some help from homebrew to install libjpeg).

edit: testing testing here as well on the more CGI type of images. Especially interesting is the 2nd image with its 6.8K size, wow (quality sucks thought).

Problems:
- some/most images have some green line on the left/right border < fixed by using the full version of js decoder
- "quantizer parameter" mode is obviously not the way to use this, if you have slightly/extremely varied image types (yes I need to re-read this thread more carefully).
- interesting stuff happening with seemingly simple rotated dark-grey quads on pixelmoon image (4:4:4 encoded).

Revelations:
- The range of stuff this thing can/will compress well is really breath-taking.

Image License:
- You are allowed to use this images in your own compression tests, keep in mind that: certain logos portrait-ed are probably copyrighted to their respective brands.

Questions:
a. How do I use x265 mode? crf?

[easyfab]

Originally Posted by smok3

a. How do I use x265 mode? crf?

If you build bpg with libx265, you can use x265 with -e x265
and crf is default with x265 ( xooyoozoo version )