Lossless Encoding Comparison 2024

[benwaggoner]

Originally Posted by MoSal

Using the encoder's CLI directly is indeed the way to go. But keeping a possibly huge y4m lossless file around is maybe what stops people from trying this (big hindrance, or completely impossible).

Having done hours of 8K 120 fps video, perhaps my sense of "huge" is a bit skewed. But a half-hour of 10-bit 420 doesn't seem particularly big.

This is compounded by some tools seemingly not supporting y4m piping, and insisting on an input file with a y4m extension.

I wrote "seemingly" above because a simple solution exists, mkfifo :

mkfifo /tmp/t.y4m
# first terminal
ffmpeg -i <input_file> -f yuv4mpegpipe - > /tmp/t.y4m
# second terminal
x265 <options> /tmp/t.y4m <output_file>

(Using two terminals to keep things simple, and see progress from both tools.)

Nice trick!

There is no external library versions "built into" FFmpeg. There is no x264 or x265 specific version "in" ffmpeg (compile-time preprocessor checks may exist to support APIs of older library versions alongside newer ones).

I am speaking of the version of x265 that is compiled into ffmpeg. I don't know if any of the common builds include x265 3.6 yet as it only came out a few weeks ago.

Unless you rolled your own, of course.

I don't know that 3.6 did anything impactful for lossless, however. I've not tested it for that. At a minimum there would be some performance improvements (more than 2x on ARM, more modest on Intel). On a current CPU --avx512 might improve performance some, particularly in --preset placebo.

The concern is not the external encoder library version. The concern is the possibility that FFmpeg may set/override/ignore/not-support some encoding parameters passed by users, and there is no way to ensure that the result you get will match what you get when you use the external encoder CLI directly.

There are new parameters and syntax added in 3.6, for example --rskip used to be a boolean, and now can take a 0, 1, or 2 as a parameter. I don't think any of those changes would impact lossless.

[benwaggoner]

Originally Posted by huhn

my point is 10 and 12 bit HEVC have the same size from 16 bit.

Yeah, with the same 10-bit input I would expect Main10 and Main12 to have essentially the same file size, as they both use the same internal 32-bit precision.

this is so utterly unlikely they have to be the "same" internally and are massively bigger then AVC 10 bit but if i pipe 10 bit directly HEVC and AVC are now comparable slashing size by about a lot down to 65%.

Pipe versus file shouldn't make any difference! Very strange. I find piping and reading from .y4m to provide identical results in x265.

[rwill]

In this thread: confused people.

Not really something OpenAI can train its LLM with...

[huhn]

Originally Posted by benwaggoner

Yeah, with the same 10-bit input I would expect Main10 and Main12 to have essentially the same file size, as they both use the same internal 32-bit precision.

with 16 bit input and lossless output. there is no way lossless 10 bit has the same size as lossless 12. lossless 10 bit needs less information to recreate.
and 10 bit input and 16 bit input should have about roughly the same size or 16 should be lower with lossless. the dithering option was not used so this is rounded 10 bit it is supposed to output.

for lossy i can understand atleast in part the result.

[rwill]

Originally Posted by benwaggoner

Yeah, with the same 10-bit input I would expect Main10 and Main12 to have essentially the same file size, as they both use the same internal 32-bit precision.

Can you please provide a source for that "internal 32-bit precision", I am willing to learn.

[rwill]

I don't know what you are all going on about when its this simple to test:

./x264 foreman_cif.yuv --profile high444 --input-depth 8 --preset veryslow --input-res 352x288 --fps 25 --qp 0 -o test.264 --output-depth 8 -A p8x8,b8x8,i8x8
-> encoded 300 frames, 157.56 fps, 10722.74 kb/s

./x264 foreman_cif.yuv --profile high444 --input-depth 8 --preset veryslow --input-res 352x288 --fps 25 --qp 0 -o test.264 --output-depth 10 -A p8x8,b8x8,i8x8
-> encoded 300 frames, 93.17 fps, 18207.77 kb/s

./x265 --input foreman_cif.yuv --profile main --input-depth 8 --preset veryslow --input-res 352x288 --fps 25 --lossless -o test.265
-> encoded 300 frames in 54.74s (5.48 fps), 10633.94 kb/s, Avg QP:4.00

./x265 --input foreman_cif.yuv --profile main10 --input-depth 8 --preset veryslow --input-res 352x288 --fps 25 --lossless -o test.265
-> encoded 300 frames in 54.97s (5.46 fps), 18111.93 kb/s, Avg QP:4.00

./x265 --input foreman_cif.yuv --profile main12 --input-depth 8 --preset veryslow --input-res 352x288 --fps 25 --lossless -o test.265
-> encoded 300 frames in 52.86s (5.67 fps), 26905.68 kb/s, Avg QP:4.00

[poisondeathray]

It looks like the -x265-params "output-depth=10" switch is not passed with ffmpeg and libx265 . This happens regardless of lossless=1 vs. lossy . The output encode is 12bit from 16bit input

But it works ok with x265 cli => --output-depth 10 switch is honored

I didn't test the latest git , my ffmpeg binary is about a month old, maybe it's fixed by now

[huhn]

@rwill
this make sure the input is 8 bit: --input-depth 8
right?

so why would the output be massively bigger?
is there well defined behaviour for these cases?
like adding the first 4 bit of the 8 bit input to the end to create accurate 12 bit?
does x264 even have an output flag: i tried to find the commit but that seem to be to old:
https://git.videolan.org/?p=x264.git...45522e57127db4

[rwill]

Originally Posted by huhn

@rwill
this make sure the input is 8 bit: --input-depth 8
right?

It DEFINES that the input raw .yuv has 8 bit depth.

Originally Posted by huhn

so why would the output be massively bigger?

For 10bit output bit depth? Because the pictures are encoded at 10bit precision and not with 8.

Originally Posted by huhn

is there well defined behaviour for these cases?
like adding the first 4 bit of the 8 bit input to the end to create accurate 12 bit?

What? Given 8bit input depth and an output depth of 10 the 8 significant bit are just shifted up by 2 I guess. Maybe there is even correct scaling... I don't know.

Originally Posted by huhn

does x264 even have an output flag: i tried to find the commit but that seem to be to old:
https://git.videolan.org/?p=x264.git...45522e57127db4

Output flag? Have you taken a look at the CLI fullhelp options yet? Your git link 404's btw. I see no reason to look up something like an CLI option in git anyway.

[huhn]

padding should cast barely anymore space.
scaling quite a bit.
and if it does stacking to 16 bit followed by dithering the space needed should explode.

[tormento]

Time for another comparison:

This commit implements a standard, compliant, version 3 and version 4 FFv1 encoder, entirely in Vulkan. The encoder is written in standard GLSL and requires a Vulkan 1.3 supporting GPU with the BDA extension.

[Z2697]

--output-depth is a CLI only option. To "control" the internal bitdepth used in FFmpeg libx265 you need to convert it. I suggest using a combination of zscale and format in filter (e.g. -vf zscale,format=yuv420p10le) because the default swscale conversion may do something weird. The format filter after zscale only sets output format for zscale filter and does nothing else, the whole conversion is done by zscale.

Same thing with FFmpeg libx264, actually. This is not a bug. It's by design.

--input-depth tells the reader the bitdepth of the input YUV file because it has no metadata, the reader will read anything according to what you told it.

The actual "pad/truncate the bits" stuff happens when --output-depth (it also sets internal depth) is different from input depth, specified by hand or determined by reader (Y4M/AVS/VPY/etc.).