Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 12th July 2020, 18:48   #7701  |  Link
theincognito
Registered User
 
Join Date: May 2014
Posts: 15
Quote:
Originally Posted by Selur View Post
Can someone tell me how to properly use the 2pass options 'analysis-save' and 'analysis-load' ?

....
-> Can someone tell me how to properly use the 2pass options 'analysis-save' and 'analysis-load' ?

Cu Selur
This is what I use and it works perfectly for me:

pass 1:
Code:
--analysis-save "E:\Temp\analysis.dat" --analysis-save-reuse-level 10 --analysis-reuse-file "E:\Temp\analysis.dat"
pass 2:
Code:
--analysis-load "E:\Temp\analysis.dat" --analysis-load-reuse-level 10 --analysis-reuse-file "E:\Temp\analysis.dat"
theincognito is offline   Reply With Quote
Old 12th July 2020, 19:43   #7702  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
...after some tests with Sol Levante, I've got a couple of observations about anime encoding, at least with this pure digital 4K HDR anime.
  • --tskip offers a big reduction in bitrate and improvement in quality
  • --rskip 1 works better than --rskip 2
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 12th July 2020, 20:08   #7703  |  Link
microchip8
ffx264/ffhevc author
 
microchip8's Avatar
 
Join Date: May 2007
Location: /dev/video0
Posts: 1,843
Quote:
Originally Posted by benwaggoner View Post
...after some tests with Sol Levante, I've got a couple of observations about anime encoding, at least with this pure digital 4K HDR anime.
  • --tskip offers a big reduction in bitrate and improvement in quality
  • --rskip 1 works better than --rskip 2
I got the reverse for live action. rskip=2 reduces onion artifacts and is faster. Haven't tested tskip
__________________
ffx264 || ffhevc || ffxvid || microenc
microchip8 is offline   Reply With Quote
Old 12th July 2020, 20:18   #7704  |  Link
theincognito
Registered User
 
Join Date: May 2014
Posts: 15
Quote:
Originally Posted by benwaggoner View Post
...after some tests with Sol Levante, I've got a couple of observations about anime encoding, at least with this pure digital 4K HDR anime.
  • --tskip offers a big reduction in bitrate and improvement in quality
  • --rskip 1 works better than --rskip 2
Thats cool What about SDR?
theincognito is offline   Reply With Quote
Old 12th July 2020, 20:29   #7705  |  Link
Asmodian
Registered User
 
Join Date: Feb 2002
Location: San Jose, California
Posts: 4,406
Quote:
Originally Posted by benwaggoner View Post
...after some tests with Sol Levante, I've got a couple of observations about anime encoding, at least with this pure digital 4K HDR anime.
  • --tskip offers a big reduction in bitrate and improvement in quality
  • --rskip 1 works better than --rskip 2
I did some testing with a few SDR 1080p anime series (the common lower budget contemporary Japanese style) and --tskip allowed a large reduction in bitrate while also improving quality. It is not just 4K HDR, tskip is huge for 1080p anime too! I was using --rskip 1 and didn't test --rskip 2. I was also using --tu-inter 4, --tu-intra 4, and --limit-tu 4, which might help for 1080p more than 4K?

tskip really is amazing and it makes x265 much better for anime than x264. I care more about transparency than size but tskip seems to improve both. A must enable option for anime, IMO.

Thank you!

Edit:
An example command line I used targeting playback on a tablet. It runs really well on my new 3950X.
vspipe.exe --y4m "Flying Witch - 05.vpy" - | x265.exe --input - --y4m -o "Flying Witch - 05.265" --crf 22 -p 8 -F 1 --pools 32 --lookahead-slices 0 --gop-lookahead 23 --min-keyint 24 --hme --hist-scenecut --fades --selective-sao 2 --tskip --tu-inter 4 --tu-intra 4 --limit-tu 4 --pmode --rect --amp --rc-lookahead 60 --range limited --colorprim 1 --transfer 1 --colormatrix 1
Edit2: --limit-tu 4 is a speedup with a minor quality hit. Testing with it off does not slow down my 3950X very much with minor quality improvements. I think I will leave it off.
__________________
madVR options explained

Last edited by Asmodian; 13th July 2020 at 00:19.
Asmodian is offline   Reply With Quote
Old 12th July 2020, 22:32   #7706  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by theincognito View Post
Thats cool What about SDR?
It's been some years since I tested SDR HD anime, but IIRC, tu-inter/intra 4 and tskip made a huge improvement.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 12th July 2020, 23:28   #7707  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,484
Quote:
Originally Posted by Selur View Post
Can someone tell me how to properly use the 2pass options 'analysis-save' and 'analysis-load' ?
You have complete test here
https://forum.doom9.org/showthread.php?t=181362

reuse encoding is not here to obtain better quality refinement in multipass mode but for:
- make reencoding at other bitrate from 2pass with other initial bitrate with really higher speed
- make reencoding at other resolution from 2 pass with other initial resolution with really higher speed

be carefull: you have incompatible option like --rd-refine or --hme with reuse encoding

You have really powefull --abr-ladder option too: you can make simulanous encoding (different bitrate and resolution) with same source at really higher speed than successive encoding.

for make reuse you must make:
- first pass (at high speed if you want)
- second pass with --analysis-save analysis.dat --analysis-save-reuse-level 10
- third pass with --analysis-load analysis.dat --analysis-load-reuse-level 10

you can use --refine-intra 4 --refine-inter 3 --refine-mv 3 for best analyse level in third pass, but really lower speed.

you can try that if you want obtain "crf quality like" with the bitrate that you want for exemple:

Quote:
x265.exe --input hp.yuv --output HP-450-SSIM.265 --input-res 720x304 --output-depth 10 --fps 25 --crf 19.5 --qcomp 0.60 --pass 1 --stats hp.log --preset veryslow --slow-firstpass --no-limit-modes --rd-refine --tune ssim --hevc-aq --qp-adaptation-range 1.0 --bframes 3 --b-adapt 2 --min-keyint 1 --psnr --ssim --analysis-save HP_450_R10-abr1_analysis.dat --analysis-save-reuse-level 10 --refine-ctu-distortion 1
second pass at really high speed but little lower quality
Quote:
x265.exe --input hp.yuv --output HP-450-SSIM.265 --input-res 720x304 --output-depth 10 --fps 25 --bitrate 450 --qcomp 0.60 --pass 3 --stats hp.log --preset veryslow --no-limit-modes --tune ssim --hevc-aq --qp-adaptation-range 1.0 --bframes 3 --b-adapt 2 --min-keyint 1 --psnr --ssim --analysis-load HP_450_R10-abr1_analysis.dat --analysis-load-reuse-level 10 --refine-ctu-distortion 1
second pass at less high speed but with real "crf like" quality
Quote:
x265.exe --input hp.yuv --output HP-450-SSIM.265 --input-res 720x304 --output-depth 10 --fps 25 --bitrate 450 --qcomp 0.60 --pass 3 --stats hp.log --preset veryslow --no-limit-modes --tune ssim --hevc-aq --qp-adaptation-range 1.0 --bframes 3 --b-adapt 2 --min-keyint 1 --psnr --ssim --analysis-load HP_450_R10-abr1_analysis.dat --analysis-load-reuse-level 10 --refine-ctu-distortion 1 --refine-intra 4 --refine-inter 3 --refine-mv 3
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9

Last edited by Sagittaire; 12th July 2020 at 23:47.
Sagittaire is offline   Reply With Quote
Old 13th July 2020, 15:58   #7708  |  Link
theincognito
Registered User
 
Join Date: May 2014
Posts: 15
Quote:
Originally Posted by Asmodian View Post
I did some testing with a few SDR 1080p anime series (the common lower budget contemporary Japanese style) and --tskip allowed a large reduction in bitrate while also improving quality. It is not just 4K HDR, tskip is huge for 1080p anime too! I was using --rskip 1 and didn't test --rskip 2. I was also using --tu-inter 4, --tu-intra 4, and --limit-tu 4, which might help for 1080p more than 4K?

tskip really is amazing and it makes x265 much better for anime than x264. I care more about transparency than size but tskip seems to improve both. A must enable option for anime, IMO.

Thank you!

Edit:
An example command line I used targeting playback on a tablet. It runs really well on my new 3950X.
vspipe.exe --y4m "Flying Witch - 05.vpy" - | x265.exe --input - --y4m -o "Flying Witch - 05.265" --crf 22 -p 8 -F 1 --pools 32 --lookahead-slices 0 --gop-lookahead 23 --min-keyint 24 --hme --hist-scenecut --fades --selective-sao 2 --tskip --tu-inter 4 --tu-intra 4 --limit-tu 4 --pmode --rect --amp --rc-lookahead 60 --range limited --colorprim 1 --transfer 1 --colormatrix 1
Edit2: --limit-tu 4 is a speedup with a minor quality hit. Testing with it off does not slow down my 3950X very much with minor quality improvements. I think I will leave it off.
Quote:
Originally Posted by benwaggoner View Post
It's been some years since I tested SDR HD anime, but IIRC, tu-inter/intra 4 and tskip made a huge improvement.

Thank you
theincognito is offline   Reply With Quote
Old 13th July 2020, 16:16   #7709  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
@Sagittaire: Thanks for the link and the explanation.
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 13th July 2020, 17:18   #7710  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by Asmodian View Post
I did some testing with a few SDR 1080p anime series (the common lower budget contemporary Japanese style) and --tskip allowed a large reduction in bitrate while also improving quality. It is not just 4K HDR, tskip is huge for 1080p anime too! I was using --rskip 1 and didn't test --rskip 2. I was also using --tu-inter 4, --tu-intra 4, and --limit-tu 4, which might help for 1080p more than 4K?
I'm running a test encode right now to compare a tu of 3 with tskip and tu of 4 without tskip. My intuition is that anime can benefit from tskip at any resolution, because the sharpness of the lines is pretty consistant, and sharp lines on smooth backgrounds is exactly why we want to get very fine blocks (min 4x4 with amp and rect) so we can isolate the sharp lines from the rest of the image and selectively apply tskip. Tskip on a 32x32 block that isn't something weird and regular like a checkerboard is probably going to be worse than a normal block. But a 4x4 with amp gets down to pixel-level alignment.

Quote:
tskip really is amazing and it makes x265 much better for anime than x264. I care more about transparency than size but tskip seems to improve both. A must enable option for anime, IMO.
Yep. It's big help with motion graphics too. Probably for anything the screen coding extensions would help.

I'd guess videogame footage would probably be better with it too, especially with HUD and games with aliasing.

Quote:
Edit2: --limit-tu 4 is a speedup with a minor quality hit. Testing with it off does not slow down my 3950X very much with minor quality improvements. I think I will leave it off.
Yeah, I only had about a 1% perf change using --limit-tu 4 with the anime test clip. That might be because there just aren't many potential tskip blocks that trigger the early exit in this kind of content. I think the speed gains with --tskip-fast were a lot higher with regular content.

I thought there was somewhere in --log-level 2 that showed how many tskip and cu-lossless blocks were used, but I can't find it now. Am I misremembering?
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 15th July 2020, 08:48   #7711  |  Link
jlpsvk
Registered User
 
Join Date: Dec 2014
Posts: 240
when encoding 1917 with HDR10+ metadata, encode gets only 171168 frames instead of 171176. when encoding without HDR10+ metadata json, everything is OK. using latest LigH's build (3.4+9).

EDIT: used the same JSON with NVENC and all good... so probably x265 problem???
__________________
AMD Ryzen 9 5950X, 32GB DDR4-3200 CL16, RTX 3060, 2TB NVMe PCIE4.0, NAS with 8x16TB HDD

Last edited by jlpsvk; 19th July 2020 at 20:33.
jlpsvk is offline   Reply With Quote
Old 16th July 2020, 01:29   #7712  |  Link
fauxreaper
Registered User
 
Join Date: Oct 2014
Posts: 23
I was reading x265 source code and I found an interesting comment about merge mode in x265_git/source/encoder/analysis.cpp...

/* Todo: Fix the satd cost estimates. Why is merge being chosen in high motion areas: estimated distortion is too low? */

I think that this can explain why restricting merge usage (lowering max-merge and/or using rskip 2) generates a video with less artifacts.
fauxreaper is offline   Reply With Quote
Old 16th July 2020, 02:04   #7713  |  Link
Greenhorn
Registered User
 
Join Date: Apr 2018
Posts: 61
Quote:
Originally Posted by fauxreaper View Post
I was reading x265 source code and I found an interesting comment about merge mode in x265_git/source/encoder/analysis.cpp...

/* Todo: Fix the satd cost estimates. Why is merge being chosen in high motion areas: estimated distortion is too low? */

I think that this can explain why restricting merge usage (lowering max-merge and/or using rskip 2) generates a video with less artifacts.
Do you have a link to where you were browsing this? I'm looking at (https://bitbucket.org/multicoreware/...r/analysis.cpp) right now and not seeing this comment. (Wrong file?)
Greenhorn is offline   Reply With Quote
Old 16th July 2020, 02:28   #7714  |  Link
fauxreaper
Registered User
 
Join Date: Oct 2014
Posts: 23
Quote:
Originally Posted by Greenhorn View Post
Do you have a link to where you were browsing this? I'm looking at (https://bitbucket.org/multicoreware/...r/analysis.cpp) right now and not seeing this comment. (Wrong file?)
I think that I saw an old file.

https://bitbucket.org/multicoreware/...r/analysis.cpp
fauxreaper is offline   Reply With Quote
Old 16th July 2020, 17:43   #7715  |  Link
LeXXuz
21 years and counting...
 
LeXXuz's Avatar
 
Join Date: Oct 2002
Location: Germany
Posts: 716
Quote:
Originally Posted by benwaggoner View Post
I note from the x265 dev list that an --auto-aq mode is in development. Maybe we'll be able to stop sweating aq-mode someday.

> This patch does the following:
> 1. Automatically decides the AQ Mode for each frame, using its scene
> statistics, such as luma intensity and edge density.
> 2. Add option "--auto-aq" to enable auto detection of AQ Mode per frame.
Any news on this? I use mode 3 for very clean sources and mode 2 for everything else.

Btw are there any new switches that should be added and are not part of the standard presets?

Right now I use preset slower with output-depth 10, aq-mode 3 and a qpfile for more acurate and faster chapter jumps. I usually use CRF16/18 for 1080p SDR.

The very slow preset is way to slow for my needs, even on a Ryzen 9 system.
LeXXuz is offline   Reply With Quote
Old 16th July 2020, 18:24   #7716  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
I'd also set --rskip 2 and possibly lower the corresponding threshold a bit. --no-sao as you don't do low bitrate encodes.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 17th July 2020, 07:38   #7717  |  Link
106062316
アズールレーン
 
106062316's Avatar
 
Join Date: May 2020
Location: Taiwan
Posts: 3
patched x265 with lsmash and lavf

Code:
x265 [info]: HEVC encoder version 3.4+12-geff904199
x265 [info]: build info [Windows][GCC 11.0.0][64 bit] 8bit+10bit+12bit
x265 [info]: (lsmash 2.16.1)
x265 [info]: (libavformat 58.48.100)
x265 [info]: (libavcodec  58.96.100)
x265 [info]: (libavutil   56.55.100)
https://1drv.ms/u/s!Arxq3mixC4xAg0xX...NYJhm?e=gJJxt8
106062316 is offline   Reply With Quote
Old 23rd July 2020, 16:27   #7718  |  Link
jlpsvk
Registered User
 
Join Date: Dec 2014
Posts: 240
really nobody will look into this????

when encoding 1917 with HDR10+ metadata, encode gets only 171168 frames instead of 171176. when encoding without HDR10+ metadata json, everything is OK. using latest LigH's build (3.4+9). tested the same encode with nvencc64 and frames number is correct. this prevents me to use hdr10+ encode for muxing with Dolby Vision EL+RPU...
__________________
AMD Ryzen 9 5950X, 32GB DDR4-3200 CL16, RTX 3060, 2TB NVMe PCIE4.0, NAS with 8x16TB HDD
jlpsvk is offline   Reply With Quote
Old 24th July 2020, 09:16   #7719  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,753
Possibly best to report this in the x265 developer mailing list. I'll do so.

Please try to provide a small sample with only few frames (I'd suggest ~500 to have more than one GOP) if you can reproduce the issue in this case.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid

Last edited by LigH; 24th July 2020 at 09:21.
LigH is offline   Reply With Quote
Old 24th July 2020, 21:11   #7720  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
I just noticed that --limit-tu 0 seems to be a potentially dangerous setting coupled with --rskip 2. I've encoded episodes of 'Chernobyl' and saw that the average bitrate is quite low for 1080p material. I found one scene where the encode looked really ugly and with some testing, I checked that changing from --limit-tu 4 to --limit-tu 0 made it happen. --ctu 32 also fixed the issue.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 16:34.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.