Command for fast decoding and lossless encoding

[vigan1]

Hello.
I want to use x264 or x265 to capture a lossless 1080p video.
If I understand correctly x265 CRF0 is not lossless, while x264 CRF0 is lossless ?

I want to have a x264 file that is the most easy to decode.
How can I achieve that with ffmpeg ?

Here what I think I will use :
- x264 (easier to decode than x265, while still being kind of small)
- preset ultrafast (for fast encode)
- pix_fmt RGB24 (lossless)
- tune fastdecode
- tune zerolatency

I understand that zerolatency disable b-frames, which also make the file easier to decode.
Can someone make me an FFmpeg script for the best decoding speed for x264 and x265 ? Is it possible to have a 264 video without interframe (all intra) ?

I want to get the lowest CPU usage while decoding.
Thank you very much.

[FranceBB]

Originally Posted by vigan1

Hello.
I want to use x264 or x265 to capture a lossless 1080p video.
If I understand correctly x265 CRF0 is not lossless, while x264 CRF0 is lossless ?

Why do you wanna use crf when there's --lossless in the command line?
Just use

x265.exe --preset medium --lossless -o "raw_video.hevc"

Rate control is disabled as well as all quality metrics (they would only waste CPU cycles), DCT is bypassed and no quantization occurs, but normal predictions are still allowed, so the encoder will find optimal inter or intra predictions and then losslessly encode the residual. You can use the preset you want, but of course you have to keep in mind that if you're capturing something like a digital source coming from an SDI, and HDMI, a Display Port, a Desktop through DirectShow and so on and your CPU can't cope with the input framerate, you're gonna lose frames!
TL;DR if your source is 25p and your encoding speed falls to 20fps, you're gonna drop frames, so test the presets and choose wisely according to what your source is and what your CPU can sustain as a load. If x265 is too heavy for real time encoding, you can try x264 but if they're both too heavy, I'd suggest you to go for something else. There are many other lossless codecs out there like HuffYUV, Lagarith, UTVideo, FFV1 etc. You might even capture as lossless HuffYUV and eventually re-encode as x265 lossless later if you wanna keep it as lossless, thus greatly reducing the file size compared to uncompressed.

Cheers,
Frank.

[vigan1]

Thank you very much, now that is very interesting !
I did not now about the lossless command and was thinking the crf0 would mean no prediction and lower cpu. But in fact, it is not the case, so thank you for pointing it, i did not know.

So Rate control is disabled as well as all quality metrics (they would only waste CPU cycles). That is why when I capture with ShareX program (only x265 CRF0 available) I can't sustain 30fps recording, while I can when I use virtualdub (x265 lossless command).

I have a very slow cpu (laptop 2cores 4threads) and I can't get 60fps with x264, so I think ffhufyuv will be my best performer at recording. I can upgrade my SSD for bigger files, but I can't upgrade mu CPU so I will go with this.

EDIT : It seems UTvideo is faster to encode than Huffyuv

How can I "benchmark" utvideo vs x264lossless for fps and file size/bitrate ?

[FranceBB]

Originally Posted by vigan1

I have a very slow cpu (laptop 2cores 4threads) and I can't get 60fps with x264, so I think ffhufyuv will be my best performer at recording. I can upgrade my SSD for bigger files, but I can't upgrade mu CPU so I will go with this.

EDIT : It seems UTvideo is faster to encode than Huffyuv

How can I "benchmark" utvideo vs x264lossless for fps and file size/bitrate ?

UTVideo faster than huffyuv? Well it's better in terms of parallelization as it's newer but it should be slightly more computational complex than huffyuv. Anyway you should be easily able to encode at 60p with them.
As to the benchmark, you can encode a sample of a few minutes with them both and check the average fps, but if you really care about space, you can easily re-encode your captured file with x264 or x265. In terms of space they should give you better results. Lagarith is kinda old and I wouldn't use it, but you could also try FFV1 for lossless encoding. It's computationally complex, so I think you could try to see the file size of x264, x265 and FFV1 vs encoding times and see what's viable for you. Don't forget to post your findings, it would be interesting to see how FFV1 compares to x265.

[vigan1]

Thank you I will post my results later when I fix everything.
It seems I am locked at 30fps with all encoder. When I do a gdigrab with ffmpeg. Do you know what I can do I tested in virtualdub and it's the same I am at 30fps max even when I set the -framerate 60 in ffmpeg.


I will update my drivers and see if it is fixed.

On my system UTvideo and hufyuv have the same speed when encoding with ffmpeg. But utvideo is faster when I encode with virtualdub.

EDIT : I updated my drivers, it wasn't it.
It is GDIgrab that capture at half the refresh rate, i set my display to 40hz and it records at 20fps!
By using DXGI in virtualdub it works at 60fps and it use less cpu. Now I will test tommorow. Thank you for the help

EDIT2 : RESULTS are here on an amd ryzen set to 100% min cpu:
- UTvideo v21 T2 is 361fps / 1min=1.35GB
- UTvideo v21 T1 is 275fps / 1min=1.75GB
- x264rgb ultrafast 130fps / 1min=235MB
- MagicYuv v2 (trial) 288fps / 1min= 1.68GB
- Huffyuv is 145fps (I deleted the file, don't have the size)
- FFV1 is 32fps but 1min is under 700MB very slow

The easiest to decode is UTvideo T2 (11%cpu) and x264 (14%cpu)

On my personal computer (laptop) I can't use x264 (max recording is 26fps) while utvideo T2 record at 58fps max (so close to 60fps)
The wierd thing is the codecs do not use 100% cpu while encoding only 30 to 40% all of them.

[FranceBB]

Very interesting findings.
I expected huffyuv, UTVideo and magicyuv to perform faster than x264 and x265 although slower.
I also expected FFV1 to perform better than huffyuv, magicyuv, lagarith and UTVideo, but at a way slower speed. Apparently the tests confirm this but also show that FFV1 is worse than x264, and x265, so now we know for sure.
As to the CPU utilization, it's right that they shouldn't be using your CPU at the very maximum, but there are different reasons. Codecs like Lagarith and Huffyuv are poorly parallelized due to their age: they were invented many years ago back when monocore CPUs were ruling the market, so they don't scale up that well on a multithread environment. As to UTVideo, it's more recent and should be able to top up your CPU at a relatively high frame rate. If you're recording at 60p, it should capture at 60p without dropping any frames. If it does, it might not be your CPU but your storage instead. Keep in mind that saving lossless files can be a strain for mechanical drives 'cause they have to constantly write a huge amount of data and perhaps they're not using contiguous cells or perhaps there are things going on in the background or perhaps they overheat and slow down... Particularly in laptops, manufacturers are more prone to install some crappy Toshiba hard drives that run at 5400 instead of 7200 with a bad hit on speed.
Do you have a mechanical drive or an SSD? If you have an SSD, is the SSD connected via Sata I, Sata II, Sata III, mSata / NVME?

[vigan1]

You should forget FFV1 for live recording, unless you have a supercomputer.
But the compression is the best of all the intra frames codec, it's compression is truly remarkeable when you understand it's all intra.

x264 lossless ultrafast is not intra frame, it uses P-frames (i disabled B-frames that is why it is so fast). But the "ultrafast" setting is good, don't go "superfast" it's 30% slower and use more cpu, and the filesize is NOT 30% smaller. Not worth it.

On my laptop, I know I have a no name ssd but it's very slow at writing but fast at reading (i tested it a while back) it may be the bottleneck of utvideo caping at around 58fps on my laptop.

I did not expect UTvideo to be this fast, I was shocked at how fast a low cpu it record, but it's probably the ssd that was the limit, not the ryzen cpu, but it's a 550read & 520write it should be enough.

If you want better benchmark look here but no Huffyuv or FFV1 :
http://umezawa.dyndns.info/wordpress/?p=6934

I will test UTvideo this week and see if I can improve to get a steady 60fps, if not I will record 30fps !!!

Maybe you should go to UTvideo, it's also open-source like huffyuv.

[FranceBB]

Originally Posted by vigan1

Maybe you should go to UTvideo, it's also open-source like huffyuv.

Well, I do actually have it installed and I've been using it from time to time, however my workflows are all based on HuffYUV as a temporary mezzanine file, so I'm gonna stick with that for now for compatibility purpose, but yeah, I might switch to UTVideo one day or another.

Originally Posted by vigan1

On my laptop, I know I have a no name ssd but it's very slow at writing but fast at reading (i tested it a while back) it may be the bottleneck of utvideo caping at around 58fps on my laptop.

Yeah, that's what I thought...

Originally Posted by vigan1

x264 lossless ultrafast is not intra frame, it uses P-frames (i disabled B-frames that is why it is so fast). But the "ultrafast" setting is good, don't go "superfast" it's 30% slower and use more cpu, and the filesize is NOT 30% smaller. Not worth it.

Well of course it's not intra, it uses motion compensation built into x264 to reduce the file size. You could technically force it to be intra with --keyint 1 but that would incredibly affect compressibility.

Originally Posted by vigan1

I will test UTvideo this week and see if I can improve to get a steady 60fps, if not I will record 30fps !!!

30fps is gonna be safe, but if you wanna risk it, you could try with 50p.

[Asmodian]

Originally Posted by vigan1

Maybe you should go to UTvideo, it's also open-source like huffyuv.

Have you tried MagicYUV? I use it now, it is faster than UTvideo. Decoding is supported by ffmpeg and it comes with codecs VirtualDub can use so compatibility is good. I thought there was a free version that didn't support >8 bit but it looks like even the 8 bit version costs money now, though it is very cheap ($9 USD). There is still the older version that is free too.

[vigan1]

Yes, it's faster than UTvideo T1 but not like the graph on the website shows. But it's not faster than UTvideo T2.

Now UTvideo T1 is in version 21 and it's the same speed (almost) as magicyuv and it's opensource and FREE. UTvideo T2 v21 is faster than magicyuv.

There is a free trial version of magicYUV if you want to test, I was thinking it was the fastest codec, but it seems the graphs are outdated, as it is compared to UTvideo v17.

I respect ignus from magicYUV but my test showed me that UTvideo is as good, and free. Sorry Ignus.
EDIT : There is a free version of magicYUV the v1.2.

Decoding is supported, but encoding is not in FFmpeg. Because it is not opensource, and it was reverse engineered I think, but they failed to implement "multithreaded per-frame decoding" which make ffmpeg decode magicyuv slower than the vfw codec.
(This info might be outdated... maybe ffmpeg is right now).

EDIT : NEW TEST with 1080p Jellyfish 30 seconds video. I wanted to try on 10BIT DSLR videos, not screen shots.
Results are completely different than 8bit RGB !!!

Cineform 10bit422 Filmscan1 = 551MB
FFV1 8bit422 = 634MB
FFV1 10bit422 = 922MB
MagicYUV2 10bit422 =912MB
UTvideoPRO 10bit422 = 1180MB
x264 8bit422 SLOW --keyint1 =895MB
x264 8bit422 Ultrafast --keyint1 =932MB
x264 10bit422 Ultrafast with P-frames =1565MB
x264 10bit422 Ultrafast --keyint1 =1589MB