x265 HEVC Encoder - Page 287

Ma · 19th November 2017, 14:38

Quote:

Originally Posted by x265_Project

Or contribute a patch

It's not always enough -- the patch https://patches.videolan.org/patch/15688/ is forgotten, x265cli.h has changed and now this patch doesn't apply cleanly.

19th November 2017, 19:10

I'm just trying to encourage participation. Sure, ideally we'd love patches to be in the right format, able to be applied cleanly to the latest development tip. But for our documentation, we can certainly handle an email to the developer mailing list with your suggested update to the documentation for a particular feature. Our team can figure out how to turn this into the official patch. Developing x265 is a big job... but this is open source, so we want to encourage participation.

Ma - I'm sorry if something was missed. We'll check into it.

hajj_3 · 19th November 2017, 21:02

are there going to be any nice quality improvements in the near future in x265 or are you getting towards x265's limit?

LigH · 20th November 2017, 08:57

Not speaking for the developers, just my personal guess: I would not expect a revolution anymore, at least in the home consumer area (8-10 bit depth), rather evolution - at most finetuning. "Miracle time" is over, the HEVC specs indicate the range of algorithms which can be used (how to search for redundancies to spare and how to encode the video stream efficiently), and "revolutionary" improvements may not be supported by HEVC specs anymore (but there is e.g. AOMedia AV1). The biggest change I hope for would be integration of libav and AviSynth input modules.

But I could imagine room for more features for professional and specific "niche" usage cases. And of course, steady improvements of speed-up technologies, from more assembly to smarter algorithms in depth.

pradeeprama · 20th November 2017, 10:04

Quote:

Originally Posted by Ma

It's not always enough -- the patch https://patches.videolan.org/patch/15688/ is forgotten, x265cli.h has changed and now this patch doesn't apply cleanly.

My bad - sorry. This patch is now pushed in and the outdated commit message has finally been updated!

LigH · 20th November 2017, 11:50

x265 2.5+65-a7c2f80c18af

Disable opt-qp-pps and opt-ref-list-length-pps by default; use AVC CU analysis data in anlysis-reuselevel 7 and 8; update an outdated help message for --bframes

LigH · 21st November 2017, 12:43

Should have waited one more day...

x265 2.5+66-dae558b40d99

merge with stable + fixed compiler warning (parentheses)

Milestone v2.6 is in sight!
_

P.S.:

The compilation of x265 will switch from YASM to NASM (min. v2.13); I wonder which were the main reasons: Instruction set support, compilation efficiency, bugs?

Ma · 21st November 2017, 13:56

Quote:

Originally Posted by ligh

the compilation of x265 will switch from yasm to nasm (min. V2.13); i wonder which were the main reasons: Instruction set support, compilation efficiency, bugs?

Only nasm supports AVX-512.

nevcairiel · 21st November 2017, 14:05

For the record, x264 did the same switch some weeks ago for the same reason.

Aurelio · 23rd November 2017, 10:44

Yasm is more or less EOL. No real development anymore. I also read once that YASM would need a rewrite to support AVX-512. So not going to happen anytime soon.

Some quotes from the x264 mailing list:

"Yes, but I don't see anyone willing to add support for AVX-512. the entire instruction syntax is different so it's a lot of work.
I have 0 hope of yasm ever getting avx-512 support since the instruction encoding is completely different from the old stuff, so it'd be tons of work to add it.
Yes. Yasm is essentially dead. Nasm is actively maintained.
Intel adds support to all new instruction sets to nasm themselves."

nevcairiel · 23rd November 2017, 10:47

Its unfortunate that nasm still suffers from a bunch of shortcomings that yasm didn't have. But maybe there is hope they'll fix it .... after all the years?

jlpsvk · 24th November 2017, 22:12

any benefit of avx-512 in x265? i got i7-7820X, co can't wait.

LigH · 25th November 2017, 19:02

Again, and again, and again ... there is no AVX-512 source code yet ... so there is no AVX-512 supporting binary yet ... so nobody could have measured an advantage yet.

burfadel · 25th November 2017, 19:32

I wouldn't be surprised if the most useful instructions for encoding in x265 aren't in the supported feature sets of the mainstream processors. AVX-512 is broken down into several feature sets, only required one is foundation. Heavy AVX-512 use may also be am issue thermal wise but that remains to be seen.

25th November 2017, 21:35

You'll need to be patient.

nevcairiel · 26th November 2017, 00:42

Quote:

Originally Posted by burfadel

I wouldn't be surprised if the most useful instructions for encoding in x265 aren't in the supported feature sets of the mainstream processors. AVX-512 is broken down into several feature sets, only required one is foundation.

While in theory that is true, in practice there is currently only two "sets" of AVX512 features, one which is used in Purley Xeons and in Skylake-X, and one in Knights Landing - which is entirely uninteresting outside of HPC. (and the one in Skylake-X/Purley is the one we want for video)
Who knows why they split it up like this, maybe in the future there will be CPUs with smaller subsets, but right now the one "mainstream" CPU core that supports it supports all of the relevant things. But its also the first SIMD instruction set that was built "extensible" like this, previously we got AVX2 to extend AVX1 with integer instructions, now we get AVX512-BW/DQ

On that note, its already confirmed that Cannonlake and Icelake will only extend the current AVX512 instructions, and not take away features again. Of course no-one knows what happens if AMD ever implements AVX512.

The bigger problem with AVX512 is the thermal behavior and as such the downclocking it entails. So for AVX512 to be worthwhile to be used, you need to spend a significant amount of time doing AVX512 things, or the downclocking will slow down other things more then the AVX512 helps.

I believe the x265 developers have had experience with this with AVX2 in the past, which also causes a downclock to some degree, although with more optimized nodes that effect has gone down in newer CPUs.

26th November 2017, 03:34

Quote:

Originally Posted by nevcairiel

The bigger problem with AVX512 is the thermal behavior and as such the downclocking it entails. So for AVX512 to be worthwhile to be used, you need to spend a significant amount of time doing AVX512 things, or the downclocking will slow down other things more then the AVX512 helps.

That's right.

Quote:

I believe the x265 developers have had experience with this with AVX2 in the past, which also causes a downclock to some degree, although with more optimized nodes that effect has gone down in newer CPUs.

Yes, we do.

benwaggoner · 28th November 2017, 05:11

Quote:

Originally Posted by LigH

Not speaking for the developers, just my personal guess: I would not expect a revolution anymore, at least in the home consumer area (8-10 bit depth), rather evolution - at most finetuning. "Miracle time" is over, the HEVC specs indicate the range of algorithms which can be used (how to search for redundancies to spare and how to encode the video stream efficiently), and "revolutionary" improvements may not be supported by HEVC specs anymore (but there is e.g. AOMedia AV1). The biggest change I hope for would be integration of libav and AviSynth input modules.

A revolution gets built out of lots of little evolutions. 20% improvement a year is still reasonable for HEVC.

Quote:

But I could imagine room for more features for professional and specific "niche" usage cases. And of course, steady improvements of speed-up technologies, from more assembly to smarter algorithms in depth.

And performance improvements turn into quality improvements, because instead of encoding faster you can choose to encode better.

LigH · 28th November 2017, 08:27

Quote:

Originally Posted by benwaggoner

And performance improvements turn into quality improvements, because instead of encoding faster you can choose to encode better.

Partially ... more efforts don't guarantee better quality retention in every case (objective vs. subjective metrics). I would prefer increasing the bitrate over passing a "threshold of sanity" (which would be somewhere near preset "slower" for me, personally; provided I had potent hardware).

pradeeprama · 29th November 2017, 06:41

x265 version 2.6 is now out! The key new improvements include support for segmented encoding, and introducing the ability of x265 to reuse and refine analysis from previously done HEVC and AVC encodes. The tarball of this release can be downloaded from the downloads page.

Release date - 29th November, 2017.
============================

New features
------------------
1) x265 can now refine analysis from a previous HEVC encode (using options --refine-inter, and --refine-intra), or a previous AVC encode (using option --refine-mv-type). The previous encode’s information can be packaged using the x265_analysis_data_t data field available in the x265_picture object.
2) Basic support for segmented (or chunked) encoding added with --vbv-end that can specify the status of CPB at the end of a segment. String this together with --vbv-init to encode a title as chunks while maintaining VBV compliance!
3) --force-flush can be used to trigger a premature flush of the encoder. This option is beneficial when input is known to be bursty, and may be at a rate slower than the encoder.
4) Experimental feature --lowpass-dct that uses truncated DCT for transformation.

Encoder enhancements
--------------------------------
1) Slice-parallel mode gets a significant boost in performance, particularly in low-latency mode.
2) x265 now officially supported on VS2017.
3) x265 now supports all depths from mono0 to mono16 for Y4M format.

API changes
-----------------
1) Options that modified PPS dynamically (--opt-qp-pps and --opt-ref-list-length-pps) are now disabled by default to enable users to save bits by not sending headers. If these options are enabled, headers have to be repeated for every GOP.
2) Rate-control and analysis parameters can dynamically be reconfigured simultaneously via the x265_encoder_reconfig API.
3) New API functions to extract intermediate information such as slice-type, scenecut information, reference frames, etc. are now available. This information may be beneficial to integrating applications that are attempting to perform content-adaptive encoding. Refer to documentation on x265_get_slicetype_poc_and_scenecut, and x265_get_ref_frame_list for more details and suggested usage.
4) A new API to pass supplemental CTU information to x265 to influence analysis decisions has been added. Refer to documentation on x265_encoder_ctu_info for more details.

Bug fixes
-------------
1) Bug fixes when --slices is used with VBV settings.
2) Minor memory leak fixed for HDR10+ builds, and default x265 when pools option is specified.
3) HDR10+ bug fix to remove dependence on poc counter to select meta-data information.

Happy compressing!

20th November 2017, 08:57	#5724 \| Link
LigH German doom9/Gleitz SuMo Join Date: Oct 2001 Location: Germany, rural Altmark Posts: 6,782	Not speaking for the developers, just my personal guess: I would not expect a revolution anymore, at least in the home consumer area (8-10 bit depth), rather evolution - at most finetuning. "Miracle time" is over, the HEVC specs indicate the range of algorithms which can be used (how to search for redundancies to spare and how to encode the video stream efficiently), and "revolutionary" improvements may not be supported by HEVC specs anymore (but there is e.g. AOMedia AV1). The biggest change I hope for would be integration of libav and AviSynth input modules. But I could imagine room for more features for professional and specific "niche" usage cases. And of course, steady improvements of speed-up technologies, from more assembly to smarter algorithms in depth. __________________ New German Gleitz board MediaFire: x264 \| x265 \| VPx \| AOM \| Xvid

20th November 2017, 11:50	#5726 \| Link
LigH German doom9/Gleitz SuMo Join Date: Oct 2001 Location: Germany, rural Altmark Posts: 6,782	x265 2.5+65-a7c2f80c18af Disable opt-qp-pps and opt-ref-list-length-pps by default; use AVC CU analysis data in anlysis-reuselevel 7 and 8; update an outdated help message for --bframes __________________ New German Gleitz board MediaFire: x264 \| x265 \| VPx \| AOM \| Xvid

21st November 2017, 12:43	#5727 \| Link
LigH German doom9/Gleitz SuMo Join Date: Oct 2001 Location: Germany, rural Altmark Posts: 6,782	Should have waited one more day... x265 2.5+66-dae558b40d99 merge with stable + fixed compiler warning (parentheses) Milestone v2.6 is in sight! _ P.S.: The compilation of x265 will switch from YASM to NASM (min. v2.13); I wonder which were the main reasons: Instruction set support, compilation efficiency, bugs? __________________ New German Gleitz board MediaFire: x264 \| x265 \| VPx \| AOM \| Xvid Last edited by LigH; 21st November 2017 at 13:37.

21st November 2017, 14:05	#5729 \| Link
nevcairiel Registered Developer Join Date: Mar 2010 Location: Hamburg/Germany Posts: 10,347	For the record, x264 did the same switch some weeks ago for the same reason. __________________ LAV Filters - open source ffmpeg based media splitter and decoders

23rd November 2017, 10:47	#5731 \| Link
nevcairiel Registered Developer Join Date: Mar 2010 Location: Hamburg/Germany Posts: 10,347	Its unfortunate that nasm still suffers from a bunch of shortcomings that yasm didn't have. But maybe there is hope they'll fix it .... after all the years? __________________ LAV Filters - open source ffmpeg based media splitter and decoders

19th November 2017, 19:10	#5722 \| Link
x265_Project Guest Posts: n/a	I'm just trying to encourage participation. Sure, ideally we'd love patches to be in the right format, able to be applied cleanly to the latest development tip. But for our documentation, we can certainly handle an email to the developer mailing list with your suggested update to the documentation for a particular feature. Our team can figure out how to turn this into the official patch. Developing x265 is a big job... but this is open source, so we want to encourage participation. Ma - I'm sorry if something was missed. We'll check into it.

19th November 2017, 21:02	#5723 \| Link
hajj_3 Registered User Join Date: Mar 2004 Posts: 1,126	are there going to be any nice quality improvements in the near future in x265 or are you getting towards x265's limit?

23rd November 2017, 10:44	#5730 \| Link
Aurelio Registered User Join Date: Dec 2013 Posts: 8	Yasm is more or less EOL. No real development anymore. I also read once that YASM would need a rewrite to support AVX-512. So not going to happen anytime soon. Some quotes from the x264 mailing list: "Yes, but I don't see anyone willing to add support for AVX-512. the entire instruction syntax is different so it's a lot of work. I have 0 hope of yasm ever getting avx-512 support since the instruction encoding is completely different from the old stuff, so it'd be tons of work to add it. Yes. Yasm is essentially dead. Nasm is actively maintained. Intel adds support to all new instruction sets to nasm themselves."

24th November 2017, 22:12	#5732 \| Link
jlpsvk Registered User Join Date: Dec 2014 Posts: 240	any benefit of avx-512 in x265? i got i7-7820X, co can't wait.

25th November 2017, 19:02	#5733 \| Link
LigH German doom9/Gleitz SuMo Join Date: Oct 2001 Location: Germany, rural Altmark Posts: 6,782	Again, and again, and again ... there is no AVX-512 source code yet ... so there is no AVX-512 supporting binary yet ... so nobody could have measured an advantage yet. __________________ New German Gleitz board MediaFire: x264 \| x265 \| VPx \| AOM \| Xvid

25th November 2017, 19:32	#5734 \| Link
burfadel Registered User Join Date: Aug 2006 Posts: 2,229	I wouldn't be surprised if the most useful instructions for encoding in x265 aren't in the supported feature sets of the mainstream processors. AVX-512 is broken down into several feature sets, only required one is foundation. Heavy AVX-512 use may also be am issue thermal wise but that remains to be seen.

25th November 2017, 21:35	#5735 \| Link
x265_Project Guest Posts: n/a	You'll need to be patient.

29th November 2017, 06:41	#5740 \| Link
pradeeprama Registered User Join Date: Sep 2015 Posts: 48	x265 version 2.6 is now out! The key new improvements include support for segmented encoding, and introducing the ability of x265 to reuse and refine analysis from previously done HEVC and AVC encodes. The tarball of this release can be downloaded from the downloads page. Release date - 29th November, 2017. ============================ New features ------------------ 1) x265 can now refine analysis from a previous HEVC encode (using options --refine-inter, and --refine-intra), or a previous AVC encode (using option --refine-mv-type). The previous encode’s information can be packaged using the x265_analysis_data_t data field available in the x265_picture object. 2) Basic support for segmented (or chunked) encoding added with --vbv-end that can specify the status of CPB at the end of a segment. String this together with --vbv-init to encode a title as chunks while maintaining VBV compliance! 3) --force-flush can be used to trigger a premature flush of the encoder. This option is beneficial when input is known to be bursty, and may be at a rate slower than the encoder. 4) Experimental feature --lowpass-dct that uses truncated DCT for transformation. Encoder enhancements -------------------------------- 1) Slice-parallel mode gets a significant boost in performance, particularly in low-latency mode. 2) x265 now officially supported on VS2017. 3) x265 now supports all depths from mono0 to mono16 for Y4M format. API changes ----------------- 1) Options that modified PPS dynamically (--opt-qp-pps and --opt-ref-list-length-pps) are now disabled by default to enable users to save bits by not sending headers. If these options are enabled, headers have to be repeated for every GOP. 2) Rate-control and analysis parameters can dynamically be reconfigured simultaneously via the x265_encoder_reconfig API. 3) New API functions to extract intermediate information such as slice-type, scenecut information, reference frames, etc. are now available. This information may be beneficial to integrating applications that are attempting to perform content-adaptive encoding. Refer to documentation on x265_get_slicetype_poc_and_scenecut, and x265_get_ref_frame_list for more details and suggested usage. 4) A new API to pass supplemental CTU information to x265 to influence analysis decisions has been added. Refer to documentation on x265_encoder_ctu_info for more details. Bug fixes ------------- 1) Bug fixes when --slices is used with VBV settings. 2) Minor memory leak fixed for HDR10+ builds, and default x265 when pools option is specified. 3) HDR10+ bug fix to remove dependence on poc counter to select meta-data information. Happy compressing!