Crazy family JPEG formats (2017).

[Jamaika]

Anyone can create this family. I took me wile the time so I don't advise to go this route.
Adds files codecs about unknown and the new functions JPEG and other codecs based on the JPEG codec which isn't located in the converter FFmpeg.

Independent JPEG Group's CJPEG, version 9b 17-Jan-2016
Copyright (C) 2016, Thomas G. Lane, Guido Vollbeding

Warning! Without function MaxMemory images 4K aren't encoded or decoded.
Do not ask me which means 'big gamut' and 'YCC' in 'cjpeg.exe': Create big gamut YCC JPEG file
Do not use function 'rgb' and 'bgycc'.

cjpeg_08bit.exe -maxmemory 1000000L -quality 90 -optimize -progressive -verbose RGB48.ppm YCbCr(yuv411)24_bt601.jpg
cjpeg_10bit.exe -maxmemory 1000000L -bgycc -quality 90 -optimize -progressive -verbose RGB48.ppm YCbCr(yuv411)30_HDR.jpg
cjpeg_12bit.exe -maxmemory 1000000L -bgycc -quality 90 -optimize -progressive -verbose RGB48.ppm YCbCr(yuv411)36_HDR.jpg

cjpeg_08bit.exe -maxmemory 1000000L -rgb -quality 100 -optimize -progressive -verbose RGB24.ppm nearlosslessR?G?B?24.jpg {checked decoding in HEX for 8bit}
cjpeg_08bit.exe -maxmemory 1000000L -rgb1 -quality 100 -optimize -progressive -verbose RGB24.ppm R?G?B?24(444)_nearlosslessRGB24.jpg {checked decoding in HEX for 8bit}
cjpeg_10bit.exe -maxmemory 1000000L -rgb -quality 100 -optimize -progressive -verbose RGB48.ppm nearlosslessR?G?B?30.jpg 
cjpeg_10bit.exe -maxmemory 1000000L -rgb1 -quality 100 -optimize -progressive -verbose RGB48.ppm R?G?B?30(444)_nearlosslessRGB30.jpg 
cjpeg_12bit.exe -maxmemory 1000000L -rgb -quality 100 -optimize -progressive -verbose RGB48.ppm nearlosslessRGB36.jpg 
cjpeg_12bit.exe -maxmemory 1000000L -rgb1 -quality 100 -optimize -progressive -verbose RGB48.ppm R?G?B?36(444)_nearlosslessR?G?B?36.jpg

This is the opj_compress utility from the OpenJPEG project.
It compresses various image formats with the JPEG 2000 algorithm.
It has been compiled against openjp2 library Sep-2016 v2.1.2.

cjpeg2000_RGB48bit.exe -i RGB48.ppm -o YCoCg48(444)_lossless.j2k -r 1 -mct 1 -C "" { checked decoding in HEX for 16bit}
cjpeg2000_RGB48bit.exe -i RGB48.ppm -o RGB48.j2k -r 1 -mct 0 -C "" { checked decoding in HEX for 16bit}

JPEG XR Encoder Utility
Copyright 2013 Microsoft Corporation - All Rights Reserved (resigned)

JxrEncApp.exe -v -q 1.0 -d 3 -a 0 -c 10 -i RGB48.tiff -o YCoCg48(444)_lossless.jxr (don't import PPM)

WebP Encoder v0.6 Copyright 7-Feb-2017

cwebp_08bit.exe -q 100 -noalpha -progress -m 6 -metadata all -v RGB48.ppm -o YUV(420)24.webp {no import file metadata .exif/.icc/.xmp}
cwebp_08bit.exe -q 100 -noalpha -lossless -progress -z 9 -m 6 -metadata all -v RGB48.ppm -o RGB24.webp (bugs, no output depth 16bit)

FLIF (Free Lossless Image Format) 0.3 [7 Feb 2017]

exiftool(>v10.40).exe RGB48.tiff(raw) -o RGB48.exif/.icc/.xmp {if is metadata .exif/.icc/.xmp}
flif.exe -e -v -o -k -E100 -N -Q100 -K -W -P512 RGB48.ppm RGB48.exif RGB48.icc RGB48.xmp RGB48_lossless.flif {checked decoding in HEX for 16bit}
flif.exe -e -v -o -k -E100 -N -Q100 -K -A -P512 RGB48.ppm RGB48.exif RGB48.icc RGB48.xmp YCoCg48(444)_lossless.flif { checked decoding in HEX for 16bit}
exiftool(>v10.40).exe RGB48.flif {if is metadata .exif/.icc/.xmp}

DNG_validate (dng2jpeg), version 1.4 (64-bit)
Copyright 2005-2012 Adobe Systems, Inc. (fixed)

Jpeg XT Copyright (C) 2012-2014 Thomas Richter, University of Stuttgartand Accusoft
default is to decode the jpeg input and write a ppm output (only Linux)

I don't think the topic is developed. However, as someone will have some news or for him are new codecs this let writes.
https://www.sendspace.com/file/i8fmap

[Midzuki]

Many thanks for the binaries

I already knew cjpeg, but not its 10-bit and 12-bit "flavors".

The command-line JXR encoder is fresh news to me as well

[Jamaika]

I want to be boast. I was able to compile Adobe DNG Converter for Windows. Adobe pages appear very old source of free Compilers.
At first I had trouble. At github.com users said that supposedly is under Windows, Mac and Linux, but it really is only under Linux. I inquired and was able to compile.
The converter saves files dng / tiff and has a complete database ICC Exif XMP. I even managed to add the latest version of Adobe XMP.
https://www.adobe.com/support/downlo..._eula_win.html
http://www.adobe.com/products/xmp.html
However, the converter has a lot of limitations, as it with free software. DNG(JPEG) can only be created with the DNG (RAW) and not of TIFF. Don't open TIFF files and no libraries TIFF. These files are lossy and not everyone the program shows the correct image size. FFmpeg can't read DNG. I don't know whether DNG(JPEG) has ex. colormatrix HDR. Probably not. Generally, we can create TIFF, but no JPEG files, although the program includes a library of JPEG 9b.
Overall, this is garbage, but sometimes useful to correctly convert the DNG files to TIFF, without installing Photoshop.

dng_validate.exe -v -cs2 -tiff lossless.tiff input.dng
dng_validate.exe -v -cs1 -dng nearlossless(jpeg).dng input.dng
dcraw_emu.exe(libraw 1.9) -v -F -T -o 2 RGB24(AdobeDNG).dng

https://www.sendspace.com/file/1uxspi

Library encoder:  DNG SDK     1.4         [14 Jul 2016]  ftp://ftp.adobe.com/pub/adobe/dng/
                  XMP SDK CC              [12 Jan 2017]  https://github.com/hfiguiere/exempi/commits/master
                  libexpat    2.2.0       [19 Feb 2017]  https://github.com/libexpat/libexpat/commits/master
                  libJPEG     9.2(9b)8bit [22 May 2016]  http://www.ijg.org/
                  zlib        1.2.11      [15 Jan 2017]  https://github.com/madler/zlib/commits/master
Compiled by Jamaika

Edit:
I am changing creator codec jpegXR.
Library jxrlib known as JxrEncApp throw out in the trash. https://jxrlib.codeplex.com/
I intercede JPEG XR software reference 1.47. This codec creates a 16bit photos.
https://jpeg.org/downloads/jpegxr/jpegxr-ref.zip

jpegxr.exe -c -o YC?C?48(444)_lossless.jxr -a 0 -q 0 -f YUV444 -l 0 RGB48.ppm {checked decoding in HEX for 16bit}

https://www.sendspace.com/file/53h64y

[dipje]

jxrencapp creates 16bit files just fine? At least here. All YUV support has been ripped out of it if I look at the source code, maybe something about conversion not working out.

But.. why are you using the YUV444 mode and not just keep it rgb? Just curious, since I'm now confused what you're actually testing.. and you're finding all weird kind of encoders nobody ever seen before :P.

[Jamaika]

Originally Posted by dipje

jxrencapp creates 16bit files just fine? At least here. All YUV support has been ripped out of it if I look at the source code, maybe something about conversion not working out.

JxrEncApp wrongly imports PPM files.

Originally Posted by dipje

But.. why are you using the YUV444 mode and not just keep it rgb? Just curious, since I'm now confused what you're actually testing.. and you're finding all weird kind of encoders nobody ever seen before :P.

Because there is no RGB. The codec converts to lossless YCoCg if it has such an algorithm. It's probably a mistake in the description. For YUV444 the files are larger. Probably it should be YCoCg444, but you need to check. For volunteers.

[Jamaika]

I test additives to Adobe DNG_validate. These are the converters to create movies DNG (RAW) from films RAW. Unfortunately, open source are very old and have many bugs. They end up in the trash.
https://github.com/apertus-open-sour...ster/Movie2DNG
https://github.com/FoxelSA/movie2dng
https://github.com/jmue/dngconvert
https://github.com/yanburman/sjcam_raw2dng

Edit:
Encoders which don't work with gcc 7.0 (Windows 10). They don't open files PPM. Theoretically they should work.
https://github.com/thorfdbg/libjpeg/commits/master
https://github.com/MOXfiles/libmox (only plugin Adobe Premiere)
https://github.com/Netflix/vmaf/commits/master
https://github.com/Fimagena/raw2dng

https://www.sendspace.com/file/robudn

[dipje]

Originally Posted by Jamaika

Because there is no RGB. The codec converts to lossless YCoCg if it has such an algorithm. It's probably a mistake in the description. For YUV444 the files are larger. Probably it should be YCoCg444, but you need to check. For volunteers.

.. in the jxrlib source and jxrencapp I believe I saw no RGB->YUV / YCoCg / whatever conversion at all.
Even more, it appears chroma-subsampling has been ripped out because it only works with it's YUV modes and it has no way of converting RGB to YUV or similar.

This is what I believe though. I didn't look too closely.

Since it has support for embedded ICC profiles and it has lossless in and out (Which, yes, is a bit for bit lossless) it's as much RGB as I can imagine.

[Jamaika]

Originally Posted by dipje

Since it has support for embedded ICC profiles and it has lossless in and out (Which, yes, is a bit for bit lossless) it's as much RGB as I can imagine.

Info "wiki":
The ICC defines the format precisely but does not define algorithms or processing details. This means there is room for variation between different applications and systems that work with ICC profiles. Since late 2010, the current version of the specification is 4.3.
In my test I use file PPM without profile ICC. In file TIFF with profile ICC hasn't info metadata in images JPEG2000, JXR. In WebP with libTIFF(Adobe XMP Specification, January 2004!!!) is info: "Error! Cannot retrieve TIFF ExtraSamples info."

I recently discovered a function for converting tiff to jpx in jpegXR.
-x: encodes images in the ISO box-based file format instead
of the TIFF format tag
This means that a JPEG2000 implementations.

Format                                   : JPEG 2000
Format profile                           : MPEG-4
Codec ID                                 : jpx  (jxrc)
File size                                : 37.0 MiB
Color space                              : RGB

In what is this save container? I don't know, but the file JPX programs don't read.
Info wiki jpeg2000:
1. Irreversible Color Transform (ICT) uses the well known YCbCr color space. It is called "irreversible" because it has to be implemented in floating or fix-point and causes round-off errors.
2. Reversible Color Transform (RCT) uses a modified YUV color space that does not introduce quantization errors, so it is fully reversible. Proper implementation of the RCT requires that numbers are rounded as specified that cannot be expressed exactly in matrix form.
Reversible Colour Transform or the 26-bit YCoCg-R may result in smaller final compressed file size than a "smaller" intermediate representation (such as RGB or YCoCg24).
Info Jpeg2000
3. Multiple Component Transform (MCT) has to be used. 0: no MCT (RCT!!!) (lossless RGB has larger files than YCC); 1: RGB->YCC conversion ; 2: custom MCT. If custom MCT, "-m" option has to be used (see hereunder). By default, RGB->YCC conversion is used if there are 3 components or more, no conversion otherwise. YCC = YCoCg

[Jamaika]

I don't believe that the codec development ideas will be merged, because for years nothing has been done in this direction. In the github is a mess. Nothing can't talk about the pictures based on the HEVC codec. I'm guessing that the development of JPEG XT will be suspended for the new JPEG XS. It is unknown when he will be released or whether it will be free of charge.

[benwaggoner]

Originally Posted by Jamaika

Nothing can't talk about the pictures based on the HEVC codec.

There has been the BPG effort to make a file format for HEVC still images:

https://bellard.org/bpg/

[Jamaika]

I intentionally don't write about BPG. The project stopped and I don't know if it will continue. I also don't see other creators accepting the new BPG format. Still, the JPEG family is growing more and more.

[Jamaika]

Info viewer FFplay:
[jpeg2000 @ 00000000046b6de0] Block with lengthinc greater than 8192 is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[jpeg2000 @ 00000000046b6de0] If you want to help, upload a sample of this file to ftp://upload.ffmpeg.org/incoming/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
[j2k_pipe @ 00000000046a4e20] Stream #0: not enough frames to estimate rate; consider increasing probesize
Input #0, j2k_pipe, from 'RGB48.j2k':
Duration: N/A, bitrate: N/A
Stream #0:0: Video: jpeg2000, rgb48le, 5212x3468, lossless, 25 tbr, 25 tbn, 25 tbc
[jpeg2000 @ 00000000046a62a0] Block with lengthinc greater than 8192 is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[jpeg2000 @ 00000000046a62a0] If you want to help, upload a sample of this file to ftp://upload.ffmpeg.org/incoming/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)

[jpegls @ 0000000004826d40] JPEG-LS that is not <= 8 bits/component or 16-bit gray is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[jpegls_pipe @ 0000000004814d00] Stream #0: not enough frames to estimate rate; consider increasing probesize
[jpegls_pipe @ 0000000004814d00] decoding for stream 0 failed
[jpegls_pipe @ 0000000004814d00] Could not find codec parameters for stream 0 (Video: jpegls, none(bt470bg/unknown/unknown), lossless): unspecified size
Consider increasing the value for the 'analyzeduration' and 'probesize' options
Input #0, jpegls_pipe, from 'rgb48.jls':
Duration: N/A, bitrate: N/A
Stream #0:0: Video: jpegls, none(bt470bg/unknown/unknown), lossless, 25 tbr, 25 tbn, 25 tbc
[jpegls @ 000000000481d760] JPEG-LS that is not <= 8 bits/component or 16-bit gray is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.

[webp @ 0000000002df6d40] skipping unsupported chunk: ICC
[webp @ 0000000002df6d40] skipping unsupported chunk: XMP

[LigH]

Today libschroedinger was removed from ffmpeg sources.

[Jamaika]

Thanks for the info.
The library has stopped being developed and Debian has removed it
from its repositories citing security issues.
The native Dirac decoder supports everything the library has and basic
encoding support is still provided via the native vc2 (Dirac Pro, intra
only version of Dirac) encoder. Hence, there's no reason to still support
linking to the library and potentially leading users into security issues.

This is the policy of the creators. JpegXR or Adobe DNG are no longer supported as old open source. Interesting how it is with OpenExr because the versions codecs are unofficial and a few creators. Jpeg codec is also very old.
I wonder if the openjpeg 2.2.0 version will be implemented?
What creators ffmpeg choose jpegXS or maybe some HEVC?
What about deleting old libogg or mxflib containers for improved security?

[LigH]

GCC 8.0.0 already? I believe 7.1.0-2 was not so long ago...

[nevcairiel]

Originally Posted by LigH

GCC 8.0.0 already? I believe 7.1.0-2 was not so long ago...

The development branch is renamed instantly after a release. Basically, he is using a very early pre-release version. Personally, I would stay away from those.

[Jamaika]

Originally Posted by LigH

GCC 8.0.0 already? I believe 7.1.0-2 was not so long ago...

You're right. GCC stable versions should be used. However, you will notice that the creators are losing on github and the programs are rarely updated.
From time to time I add open source to be able to compile codecs in any gcc. This is my fun and exploration rather than the interest of the users.

[Jamaika]

Recently I tried to compile non-standard codecs jpeg for the portals store photos type dropbox(lepton) or google(guetzli).
GCC 7.1-8.0 unfortunately it can't cope with the build. Even when adjusting the function under GCC std::thread or std::mutex from files mingw.thread.h or mingw.mutex.h encoder doesn't work.
https://github.com/meganz/mingw-std-threads
I adds my open source codecs. I don't think that there will be a quick fix for dropbox errors.
https://www.sendspace.com/file/6qs8qd

[LigH]

Off-topic remark: I wonder if there is any "JPEG optimizer" which is able to re-quantize JPEG images like xat.com MagiCompress™ can: with different ratios per slice/MB, adaptive to the detail amount. Or is that approach patent-protected?

[Jamaika]

Perhaps, but this MagiCompress™ is archaic programs (2009). I don't see new pages.
https://wazapdark.wordpress.com/programs/
http://www.softsea.com/review/JPEG-Optimizer.html