Benwaggoner HEVC encoding challenge

[Joekiwi]

How well does NVENC do .?


Deleted as used wrong source - was quite surprised how high my y ssim score was - I might try again some other time

these were my settings anyway

NVEncC64.exe --vbrhq 1000 --codec h265 --preset quality --profile main10 --tier high --level 4.1 --output-depth 10 --max-bitrate 4000 --vbv-bufsize 12000 --aq-strength 1 --aq --aq-temporal --bref-mode each --bframes 4 --ref 5 --gop-len 120 --lookahead 32 --slices 1 --multiref-l0 4 --multiref-l1 2 --nonrefp --weightp --ssim --mv-precision q-pel --cabac -i

[Joekiwi]

Ok maybe disregard my last post - I'm using the wrong source - so probably it's probably a much easier to encode .
I just watched Bens' placebo encode and there was much more detail .
I just watched my source for the encode and it also lacked the detail.

I did download the 14 gb file - But when I unzipped it I got nothing - wasn't sure why - so downloaded another file mentioned later on that I think was only 579mbs ( from memory-I'm on a different PC ) .

So NVENC will be worst than my scores - still I can untick copy audio and get another 90 odd kbps to use . Ben did I need all those 4 files in your link to get the Source?

[Sagittaire]

Originally Posted by Joekiwi

I did download the 14 gb file - But when I unzipped it I got nothing - wasn't sure why - so downloaded another file mentioned later on that I think was only 579mbs ( from memory-I'm on a different PC ) .

Work for me. Perhpabs downloading problem?

Be carefull, Placebo Benwagonner encoding is really old encoding. And I suspect Rate Control problem (really bad quality in start scene). Ma's encoding produce really better overall quality. Bearm 4.5 encoding is really good too (be carefull it's zone encoding like real compressionnist encoding).

I'm using the wrong source - so probably it's probably a much easier to encode

Yes, Benwagonner use Dithering for make YV12 lossless source

[Sagittaire]

Well after many and many tests to find best x265 setting for this ToS source, here my result for only 1000 kbps at this moment. I will use he-aac 5.1 audio at 128 kbps for 1000 kbps, he-aac 5.1 audio at 160 kbps for 1500 kbps and lc-aac 5.1 audio at 192 kbps for 2000 kbps encoding:

WARNING: google drive make AVC transcoding for streaming. if you want to see real HEVC encoding, download the file.


1) crf 1 passe mode:
highest possible metric score in all mode
Preset "veryslow" for 1000 kbps and "optimized slower" for 1500 and 2000 kbps

ToS-1000-crf.mkv
ToS-1500-crf.mkv
ToS-2000-crf.mkv

D:\Mes Logiciels\Codec\x265>x265.exe --input C:\ToS_1920x800_xdither.y4m --output ToS-1000-cfr.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --crf 23.1 --pass 1 --slow-firstpass --stats ToS-1000-crf.log --qcomp 0.50 --preset veryslow --tune ssim --no-limit-modes --rd-refine --hevc-aq --qp-adaptation-range 1.0 --bframes 5 --b-adapt 2 --ref 5 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --tskip --qg-size 64 --merange 64 --analysis-save ToS_1000_R10-crf1_analysis.dat --analysis-save-reuse-level 10 --deblock -1,-1 --limit-sao
x265 [info]: frame I: 221, Avg QP:27.27 kb/s: 11216.30 PSNR Mean: Y:43.857 U:46.103 V:46.112 SSIM Mean: 0.973372 (15.747dB)
x265 [info]: frame P: 3910, Avg QP:29.09 kb/s: 2844.64 PSNR Mean: Y:43.033 U:46.037 V:45.997 SSIM Mean: 0.971443 (15.443dB)
x265 [info]: frame B: 13489, Avg QP:36.49 kb/s: 296.98 PSNR Mean: Y:41.376 U:45.256 V:45.240 SSIM Mean: 0.967579 (14.892dB)
x265 [info]: Weighted P-Frames: Y:6.0% UV:4.8%
x265 [info]: Weighted B-Frames: Y:4.1% UV:2.8%
x265 [info]: consecutive B-frames: 10.8% 7.6% 5.9% 24.9% 21.6% 29.2%

encoded 17620 frames in 30340.70s (0.58 fps), 999.28 kb/s, Avg QP:34.73, Global PSNR: 42.688, SSIM Mean Y: 0.9685090 (15.018 dB)

D:\Mes Logiciels\Codec\x265>x265.exe --input C:\ToS_1920x800_xdither.y4m --output ToS-1500-crf.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --crf 20.0 --preset slower --qcomp 0.50 --bframes 5 --b-adapt 2 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --tune ssim --ref 5 --limit-refs 3 --rd-refine --hevc-aq --qp-adaptation-range 1.0 --qg-size 64 --hme --hme-search 0,1,3 --hme-range 24,48,64 --deblock -1,-1 --limit-sao --csv ToS-1500-crf.csv --csv-log-level 1

encoded 17620 frames in 16079.49s (1.10 fps), 1504.59 kb/s, Avg QP:31.57, Global PSNR: 43.905, SSIM Mean Y: 0.9743358 (15.907 dB)

encoded 17620 frames in 18895.70s (0.93 fps), 2002.67 kb/s, Avg QP:29.41, Global PSNR: 44.772, SSIM Mean Y: 0.9775448 (16.487 dB)

2) ABR with 3 passes:
highest possible metric in NPass mode
Preset "veryslow" for 1000 kbps and "optimized slower" for 1500 and 2000 kbps

ToS-1000-abr.mkv
ToS-1500-abr.mkv
ToS-2000-abr.mkv

D:\Mes Logiciels\Codec\x265>x265.exe --input C:\ToS_1920x800_xdither.y4m --output ToS-1000-abr.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --bitrate 1000 --pass 2 --stats ToS-1000.log --qcomp 0.50 --preset veryslow --tune ssim --no-limit-modes --rd-refine --hevc-aq --qp-adaptation-range 1.0 --bframes 5 --b-adapt 2 --ref 5 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --tskip --qg-size 64 --merange 64 --analysis-save ToS_1000_R10-abr3_analysis.dat --analysis-save-reuse-level 10 --deblock -1,-1 --limit-sao
x265 [info]: frame I: 221, Avg QP:26.58 kb/s: 10729.30 PSNR Mean: Y:43.864 U:46.118 V:46.127 SSIM Mean: 0.972412 (15.593dB)
x265 [info]: frame P: 3910, Avg QP:29.26 kb/s: 2266.71 PSNR Mean: Y:42.187 U:45.456 V:45.336 SSIM Mean: 0.966785 (14.787dB)
x265 [info]: frame B: 13489, Avg QP:31.23 kb/s: 469.21 PSNR Mean: Y:41.386 U:44.917 V:44.851 SSIM Mean: 0.965572 (14.631dB)
x265 [info]: Weighted P-Frames: Y:6.0% UV:4.7%
x265 [info]: Weighted B-Frames: Y:4.1% UV:2.8%
x265 [info]: consecutive B-frames: 10.8% 7.6% 5.9% 24.9% 21.6% 29.2%

encoded 17620 frames in 34506.66s (0.51 fps), 996.78 kb/s, Avg QP:30.73, Global PSNR: 42.449, SSIM Mean Y: 0.9659273 (14.676 dB)

D:\Mes Logiciels\Codec\x265>x265.exe --input C:\ToS_1920x800_xdither.y4m --output ToS-1500-abr.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --bitrate 1500 --pass 3 --stats ToS-1500-abr.log --preset slower --qcomp 0.50 --bframes 5 --b-adapt 2 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --tune ssim --ref 5 --limit-refs 3 --rd-refine --hevc-aq --qp-adaptation-range 1.0 --qg-size 64 --hme --hme-search 0,1,3 --hme-range 24,48,64 --deblock -1,-1 --limit-sao --csv ToS-1500-abr.csv --csv-log-level 1

encoded 17620 frames in 16556.89s (1.06 fps), 1495.10 kb/s, Avg QP:27.88, Global PSNR: 43.626, SSIM Mean Y: 0.9724194 (15.594 dB)

encoded 17620 frames in 19555.37s (0.90 fps), 2000.35 kb/s, Avg QP:26.00, Global PSNR: 44.483, SSIM Mean Y: 0.9761716 (16.229 dB)

3) ABR with 3 passes with default setting:
default setting for rate control, adaptative quantisation and psy-rdo setting at same level research for me, cu, tu ... than crf (1) and abr (2) encoding.
Preset "veryslow" for 1000 kbps and "optimized slower" for 1500 and 2000 kbps.

ToS-1000-psy.mkv
ToS-1500-psy.mkv
ToS-2000-psy.mkv

D:\Mes Logiciels\Codec\x265>x265.exe --input C:\ToS_1920x800_xdither.y4m --output ToS-1000-psy.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --bitrate 1000 --pass 2 --stats ToS-1000-psy.log --preset veryslow --bframes 5 --b-adapt 2 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --no-limit-modes --ref 5 --rd-refine --tskip --qg-size 64 --merange 64 --analysis-save ToS_1000_R10-psy1_analysis.dat --analysis-save-reuse-level 10 --csv ToS-1000-psy.csv --csv-log-level 1
x265 [info]: frame I: 221, Avg QP:28.02 kb/s: 10100.18 PSNR Mean: Y:43.571 U:45.588 V:45.560 SSIM Mean: 0.974445 (15.925dB)
x265 [info]: frame P: 3910, Avg QP:31.83 kb/s: 2542.81 PSNR Mean: Y:41.707 U:44.944 V:44.790 SSIM Mean: 0.968909 (15.074dB)
x265 [info]: frame B: 13489, Avg QP:37.81 kb/s: 403.79 PSNR Mean: Y:40.508 U:44.297 V:44.197 SSIM Mean: 0.965387 (14.608dB)
x265 [info]: Weighted P-Frames: Y:5.8% UV:4.7%
x265 [info]: Weighted B-Frames: Y:3.8% UV:2.6%
x265 [info]: consecutive B-frames: 10.8% 7.6% 5.9% 24.9% 21.6% 29.2%

encoded 17620 frames in 32709.81s (0.54 fps), 1000.07 kb/s, Avg QP:36.36, Global PSNR: 41.710, SSIM Mean Y: 0.9662820 (14.721 dB)

D:\Mes Logiciels\Codec\x265>x265.exe --input C:\ToS_1920x800_xdither.y4m --output ToS-1500-psy.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --bitrate 1500 --pass 3 --stats ToS-1500-psy.log --preset slower --bframes 5 --b-adapt 2 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --ref 5 --limit-refs 3 --rd-refine --qg-size 64 --hme --hme-search 0,1,3 --hme-range 24,48,64 --csv ToS-1500-psy.csv --csv-log-level 1

encoded 17620 frames in 17323.16s (1.02 fps), 1497.51 kb/s, Avg QP:33.00, Global PSNR: 43.081, SSIM Mean Y: 0.9728444 (15.661 dB)

encoded 17620 frames in 20365.97s (0.87 fps), 1997.96 kb/s, Avg QP:30.48, Global PSNR: 44.045, SSIM Mean Y: 0.9763138 (16.255 dB)

4) Reuse ABR with 3 passes :
transcoding from ABR 3 passes encoding veryslow at 1000 kbps (2) with analysis reuse statistique at really high speed (by factor ~100x)

ToS-1000-hs.mkv
ToS-1500-hs.mkv
ToS-2000-hs.mkv

D:\Mes Logiciels\Codec\x265>x265.exe --input C:\ToS_1920x800_xdither.y4m --output ToS-1000-hs.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --bitrate 1000 --pass 3 --stats ToS-1000.log --qcomp 0.50 --preset veryslow --tune ssim --no-limit-modes --rd-refine --hevc-aq --qp-adaptation-range 1.0 --bframes 5 --b-adapt 2 --ref 5 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --tskip --qg-size 64 --merange 64 --analysis-load ToS_1000_R10-abr3_analysis.dat --analysis-load-reuse-level 10 --deblock -1,-1 --limit-sao

x265 [info]: frame I: 221, Avg QP:26.75 kb/s: 10671.81 PSNR Mean: Y:43.556 U:46.032 V:46.032 SSIM Mean: 0.971743 (15.489dB)
x265 [info]: frame P: 3910, Avg QP:29.48 kb/s: 2284.96 PSNR Mean: Y:41.984 U:45.347 V:45.232 SSIM Mean: 0.965561 (14.629dB)
x265 [info]: frame B: 13489, Avg QP:31.44 kb/s: 469.18 PSNR Mean: Y:41.190 U:44.813 V:44.752 SSIM Mean: 0.964374 (14.482dB)
x265 [info]: Weighted P-Frames: Y:6.0% UV:4.7%
x265 [info]: Weighted B-Frames: Y:4.1% UV:2.8%
x265 [info]: consecutive B-frames: 10.8% 7.6% 5.9% 24.9% 21.6% 29.2%

encoded 17620 frames in 431.14s (40.87 fps), 1000.08 kb/s, Avg QP:30.95, Global PSNR: 42.275, SSIM Mean Y: 0.9647300 (14.526 dB)

D:\Mes Logiciels\Codec\x265>x265.exe --input C:\ToS_1920x800_xdither.y4m --output ToS-1500-hs.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --bitrate 1500 --pass 3 --stats ToS-1000.log --qcomp 0.50 --preset veryslow --tune ssim --no-limit-modes --rd-refine --hevc-aq --qp-adaptation-range 1.0 --bframes 5 --b-adapt 2 --ref 5 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --tskip --qg-size 64 --merange 64 --analysis-load ToS_1000_R10-abr3_analysis.dat --analysis-load-reuse-level 10 --deblock -1,-1 --limit-sao

encoded 17620 frames in 477.85s (36.87 fps), 1499.96 kb/s, Avg QP:27.91, Global PSNR: 43.405, SSIM Mean Y: 0.9709562 (15.369 dB)

encoded 17620 frames in 485.03s (36.33 fps), 2001.12 kb/s, Avg QP:26.12, Global PSNR: 44.120, SSIM Mean Y: 0.9742588 (15.894 dB)

5) Reuse ABR with 3 passes, refine and psy optimisation :
transcoding from ABR 3 passes encoding veryslow at 1000 kbps (2) with analysis reuse statistique, highest possible refine level (higher speed by factor ~3x) and my best psy optimisation. Best result for my eyes.

ToS-1000-jfl.mkv
ToS-1500-jfl.mkv
ToS-2000-jfl.mkv

D:\Mes Logiciels\Codec\x265>x265.exe --input C:\ToS_1920x800_xdither.y4m --output ToS-1000-jfl.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --bitrate 1000 --pass 3 --stats ToS-1000.log --qcomp 0.60 --preset veryslow --no-limit-modes --hevc-aq --qp-adaptation-range 1.0 --bframes 5 --b-adapt 2 --ref 5 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --tskip --qg-size 64 --merange 64 --analysis-load ToS_1000_R10-abr3_analysis.dat --analysis-load-reuse-level 10 --deblock -1,-1 --limit-sao --rdoq-level 2 --psy-rdoq 1.0 --psy-rd 0.5 --refine-intra 4 --refine-inter 3

x265 [info]: frame I: 221, Avg QP:27.55 kb/s: 10061.19 PSNR Mean: Y:43.427 U:45.710 V:45.694 SSIM Mean: 0.972341 (15.582dB)
x265 [info]: frame P: 3910, Avg QP:30.28 kb/s: 2294.97 PSNR Mean: Y:41.977 U:45.093 V:44.928 SSIM Mean: 0.966663 (14.771dB)
x265 [info]: frame B: 13489, Avg QP:32.31 kb/s: 476.09 PSNR Mean: Y:41.138 U:44.526 V:44.419 SSIM Mean: 0.965165 (14.580dB)
x265 [info]: Weighted P-Frames: Y:6.0% UV:4.7%
x265 [info]: Weighted B-Frames: Y:4.1% UV:2.8%
x265 [info]: consecutive B-frames: 10.8% 7.6% 5.9% 24.9% 21.6% 29.2%

encoded 17620 frames in 12127.64s (1.45 fps), 999.93 kb/s, Avg QP:31.80, Global PSNR: 42.167, SSIM Mean Y: 0.9655874 (14.633 dB)

D:\Mes Logiciels\Codec\x265>x265.exe --input C:\ToS_1920x800_xdither.y4m --output ToS-1500-jfl.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --bitrate 1500 --pass 3 --stats ToS-1000.log --qcomp 0.60 --preset veryslow --no-limit-modes --hevc-aq --qp-adaptation-range 1.0 --bframes 5 --b-adapt 2 --ref 5 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --tskip --qg-size 64 --merange 64 --analysis-load ToS_1000_R10-abr3_analysis.dat --analysis-load-reuse-level 10 --deblock -1,-1 --limit-sao --rdoq-level 2 --psy-rdoq 1.0 --psy-rd 0.5 --refine-intra 4 --refine-inter 3

encoded 17620 frames in 16147.07s (1.09 fps), 1500.61 kb/s, Avg QP:28.97, Global PSNR: 43.469, SSIM Mean Y: 0.9724217 (15.594 dB)

encoded 17620 frames in 14034.95s (1.26 fps), 1998.10 kb/s, Avg QP:27.03, Global PSNR: 44.305, SSIM Mean Y: 0.9759682 (16.192 dB)

Conclusion at 1000 kbps:

- For my eyes, my best encoding (5) produce better quality than default psy x265 setting (4) or previous ultraplacebo Benwagonner encoding, and by far. It's really good quality for only 1000 kbps encoding at 1080p. For me it's a really reasonable quality for streaming on a small screen ... or for reduced internet bandwidth due to COVID19 crisis. For comparison it's by far better quality level than Netflix "premium" quality for same bitrate.
- crf encoding (1) produce best metric and by far but not best visual quality for my eyes. crf encoding has really agressive quantizer for bframe and that mean lower relative quality for low complexity part. Anyway it's higher relative quality for high complexity part too. But this compromise is not good for my eyes at 1000 kbps. At this really low bitrate level (1000 kbps for 1080p source), I prefer high compression for rate control curve (more "cbr like" and less "vbr like").
- Encoding with default x265 setting (3) don't produce good quality for my eyes. There are many problems, particulary on the edge. Perhaps that these defaults psy setting are not good for really low bitrate.
- The most interessing encoding is certainely reuse ABR transcoding (4). I make ABR at "veryslow quality preset" but at stratospheric speed (40.87 fps for 1080p source with little i5 3550 CPU), in 3 passes to have best possible constant quality rate control curve. And the result is impressive. Really better result for my eyes than crf (1) or default setting (4).


Conclusion at 2000 kbps:

- All encoding produce really good quality. Quality difference between profil encoding are not really high like for 1000 kbps encoding. I make these encoding with optimized "slower" x265 profil for (1), (2) and (3), and I obtain equivalent quality than old previous benwagonner encoding with "ultra placebo" x265 profil.
- Default setting for x265 at 2000 kbps (3) produce really better relative subjective quality than for 1000 kbps encoding. At this quality level, default setting are really good.
- You can produce really good quality at high speed with reuse encoding without refine (4).
- My prefered setting for my eyes are always reuse encoding with refine and my psy optimisation (5). For my eyes this profil have better temporal stability, noise retention, and overall quality but this time, by a really small margin.

[Opmox]

deleted

[Boulder]

Regarding comparisons of CRF vs. multipass: what's the verdict on doing the first pass with CRF and then running a second pass using the average bitrate that resulted from the CRF run? It won't work with this fixed bitrate level challenge, I was thinking more on the lines of squeezing some extra quality out of x265 if the 2-pass rate control seems to be better. Maybe there are also some options that could be left out of the CRF pass and activated only in the second pass to speed things up a bit (or use those reuse options)?

[Joekiwi]

Originally Posted by Sagittaire

Work for me. Perhpabs downloading problem?

Be carefull, Placebo Benwagonner encoding is really old encoding. And I suspect Rate Control problem (really bad quality in start scene). Ma's encoding produce really better overall quality. Bearm 4.5 encoding is really good too (be carefull it's zone encoding like real compressionnist encoding).




Yes, Benwagonner use Dithering for make YV12 lossless source

Thanks Sagittaire,
Will try again maybe later in week . I know NVENC will get absolutely slaughtered by yours and other encodes . - However I am curious .
I use to use paid software that promised superfast encoding speed ( ie via GPU ) . There are probably a lot of people out there like me - that used or still uses such software . - Since installing staxrip as a front end I feel I'm get much better results and that I'm in control.
I should really get to learn the x265 on CPU as well - easy on handbrake as only a few options - easy just to set CRF & preset - but will try to learn it on staxrip with it's myriad of options - or maybe just easier to grab someones command line and use the encoder directly-my Ryzen 3700x can handle it & still do NVENC coding at same time for no cost .

Without Bens' constraints GOP , max bit rate, vbv -I imagine your metrics would be quite better.

Maybe someone else can see good they can achieve by setting minimum 3fps encoding which is a good metric for encoding a movie while you sleep

[Sagittaire]

Originally Posted by Opmox

Here's another encode, same as before, no-sao but I changed rd 4 to 6 and me from umh to star, psnr went from 41.612 to 41.658
ToS_x265_1M_41.658.mkv

0.05 dB is ridiculous delta for "metric quality". and you can have small Rate Control difference to explain this delta. Choose the best way for your eyes ... ;-)

So you like everything blurry

mosquito noise are not detail and ringing are not texture retention ...

You have major problem in your encoding:
- Mosquito noise (it's not detail or noise retention but only encoding artefact)
- Ringing arround all the edge
- temporal blocking instability on flat part like the sky (certainely sao off)
- detail retention in low motion scene

I can make captures that will be terrible for your encoding.
There is a simple test to do: see if the flies(?) flying above the actress' head between frame 877 and 907 (37s to 38s) are visible. In your encoding it's not visible (or only on Pframe) simply because you have terrible ringing around the actress' head. Certainely because you have really high difference quality between inter and intra block (or between Pframe and bframe).
You have major temporal blocking and bluring on the rocket (18s to 25s) ... in fact it's the same problem for all the edge everywhere.




I think that if you use really higher bitrate (with your setting), all your "detail" or "texture" will no longer be there ...

[Sagittaire]

Originally Posted by Boulder

Regarding comparisons of CRF vs. multipass: what's the verdict on doing the first pass with CRF and then running a second pass using the average bitrate that resulted from the CRF run? It won't work with this fixed bitrate level challenge, I was thinking more on the lines of squeezing some extra quality out of x265 if the 2-pass rate control seems to be better. Maybe there are also some options that could be left out of the CRF pass and activated only in the second pass to speed things up a bit (or use those reuse options)?

1) I just looked for the right crf value to reach the right bitrate (step 0.1 by step 0.1 for crf value). I try to use stat files from crf mode for make classic multipass mode but the result in not good (multipass encoding seem conserve the crf mode decision for I-P-B quantiser with the same ratio).

2) the right way would be to control the quantizer ratio between differents frames types but these command (--ip-ratio and -ib-ratio) seem doesen't work in the crf mode. (work in multipass mode)

[Boulder]

Originally Posted by Sagittaire

1) I just looked for the right crf value to reach the right bitrate (step 0.1 by step 0.1 for crf value).

2) the right way would be to control the quantizer ratio between differents frames types but these command (--ip-ratio and -ib-ratio) seem doesen't work in the crf mode. (work in multipass mode)

My thought was to run a CRF 18 encode with my usual script and settings and then run a second pass to utilize the rate control if it indeed is better. That way I would get the filesizes I'm used to without having to do any compressibility tests and get the best quality out of it. I was just thinking out loud if I can loosen any settings or reuse the analysis without affecting the whole idea too much.

The ratios should definitely work, they are for example a part of --tune grain so in case they don't, it should be reported as an issue in the tracker.

[Sagittaire]

Originally Posted by Boulder

My thought was to run a CRF 18 encode with my usual script and settings and then run a second pass to utilize the rate control if it indeed is better. That way I would get the filesizes I'm used to without having to do any compressibility tests and get the best quality out of it. I was just thinking out loud if I can loosen any settings or reuse the analysis without affecting the whole idea too much.

I make tests like that and with reuse encoding too. But like Rate Control work good with NPass, I don't investigate more. But it work. Work good, I don't know, but it's possible.

The ratios should definitely work, they are for example a part of --tune grain so in case they don't, it should be reported as an issue in the tracker.

In fact --ip-ratio and --ib-ratio doesn't work with --crf and --cu-tree on (RDO for AQ decision if my memory is good, but perhaps confusion with x264 here?). If you desactive cu-tree, ip-ratio and ib-ratio work with crf. And --tune grain desactive --cu-tree. In my memory --cu-tree off desactive all Adaptative Quantisation (psy way too).

Like I say, I make many and many test ... ;-)

[benwaggoner]

Originally Posted by Opmox

Here's another encode, same as before, no-sao but I changed rd 4 to 6 and me from umh to star, psnr went from 41.612 to 41.658

The higher RD modes allow for more psychovisual optimizations, and those often reduce PSNR while they improve psychovisual quality. <0.5 PSNR shift isn't something that can be assumed to be better/worse without visual inspection.

[Opmox]

Originally Posted by benwaggoner

The higher RD modes allow for more psychovisual optimizations, and those often reduce PSNR while they improve psychovisual quality.

That make sense thanks!

Originally Posted by Sagittaire

mosquito noise are not detail and ringing are not texture retention ...
...
I think that if you use really higher bitrate (with your setting), all your "magical" detail will no longer be there ...

Wait, you can't see the difference here, here or here? I'm out

[Sagittaire]

Originally Posted by Opmox

That make sense thanks!



Wait, you can't see the difference here, here or here? I'm out

Will be better if you make capture for your encoding too ... ;-).

Anyway I confirm that I use highest possible curve rate compression (qcomp at 0.50) to have better quality in low motion and less in high motion. For this reason I prefer my ABR encoding to equivalent crf encoding.

Eyes are generaly more sensitive at detail and texture preservation in low motion part and less in high motion. It's for this reason that I prefer the Bearm encoding (HEVC encoder). But your eyes are perhaps more sensible at high motion scene (really short scene in this source). In this case, you can try encoding with higher qcomp default value (0.6 by default, 1.0 will be real VBR).

However I maintain that the quality for your encoding in the low motion scene is terrible. Temporal stability is catastrophic too in low motion. The rendering of moving objects in static scenes (like head movements for exemple) has really visible and have annoying temporal artifacts too. I prefer, and by far, my compromise for overall quality.

Anyway HVS is like taste and colors. Some prefer chocolate and others vanilla. Some users prefer x265 with sao and other not. But for my eyes your encoding is really bad, and particulary in scene introduction.

If Boulder or Benwagoner have an opinion ... ;-)

[Boulder]

Originally Posted by Sagittaire

In fact --ip-ratio and --ib-ratio doesn't work with --crf and --cu-tree on (RDO for AQ decision if my memory is good, but perhaps confusion with x264 here?). If you desactive cu-tree, ip-ratio and ib-ratio work with crf. And --tune grain desactive --cu-tree. In my memory --cu-tree off desactive all Adaptative Quantisation (psy way too).

They do work in CRF mode with cu-tree and AQ enabled.

P/B ratio 1.3

in:0 out:0 type:I q:21.94 q-aq:19.57 q-noVbv:21.94 q-Rceq:0.99 tex:582913 mv:9220 misc:2973 icu:3600.00 pcu:0.00 scu:0.00 ;
in:4 out:1 type:P q:21.94 q-aq:19.24 q-noVbv:21.94 q-Rceq:0.99 tex:614212 mv:13405 misc:3414 icu:3297.50 pcu:301.25 scu:1.25 ;
in:2 out:2 type:B q:23.08 q-aq:21.04 q-noVbv:23.08 q-Rceq:0.99 tex:386297 mv:23848 misc:6130 icu:959.50 pcu:2405.25 scu:235.25 ;
in:1 out:3 type:b q:24.21 q-aq:22.84 q-noVbv:24.21 q-Rceq:0.99 tex:221768 mv:22069 misc:7085 icu:288.50 pcu:2764.00 scu:547.50 ;
in:3 out:4 type:b q:24.21 q-aq:22.85 q-noVbv:24.21 q-Rceq:0.99 tex:234317 mv:23280 misc:6718 icu:393.75 pcu:2778.00 scu:428.25 ;
in:9 out:5 type:P q:21.94 q-aq:19.38 q-noVbv:21.94 q-Rceq:0.99 tex:589089 mv:11898 misc:3306 icu:3406.25 pcu:191.75 scu:2.00 ;
in:7 out:6 type:B q:23.08 q-aq:21.66 q-noVbv:23.08 q-Rceq:0.99 tex:346593 mv:21229 misc:5560 icu:1007.25 pcu:2402.25 scu:190.50 ;
in:5 out:7 type:b q:24.21 q-aq:22.80 q-noVbv:24.21 q-Rceq:0.99 tex:227102 mv:26712 misc:7243 icu:451.00 pcu:2715.25 scu:433.75 ;
in:6 out:8 type:b q:24.21 q-aq:22.80 q-noVbv:24.21 q-Rceq:0.99 tex:258433 mv:22466 misc:6381 icu:337.75 pcu:2954.25 scu:308.00 ;
in:8 out:9 type:b q:24.21 q-aq:22.78 q-noVbv:24.21 q-Rceq:0.99 tex:212971 mv:24199 misc:6965 icu:322.75 pcu:2758.50 scu:518.75 ;

P/B ratio 1.2

in:0 out:0 type:I q:21.94 q-aq:19.57 q-noVbv:21.94 q-Rceq:0.99 tex:582913 mv:9220 misc:2973 icu:3600.00 pcu:0.00 scu:0.00 ;
in:4 out:1 type:P q:21.94 q-aq:19.24 q-noVbv:21.94 q-Rceq:0.99 tex:614212 mv:13405 misc:3414 icu:3297.50 pcu:301.25 scu:1.25 ;
in:2 out:2 type:B q:22.73 q-aq:20.71 q-noVbv:22.73 q-Rceq:0.99 tex:414031 mv:24937 misc:6025 icu:972.75 pcu:2444.75 scu:182.50 ;
in:1 out:3 type:b q:23.52 q-aq:22.14 q-noVbv:23.52 q-Rceq:0.99 tex:271359 mv:22093 misc:6977 icu:381.75 pcu:2752.25 scu:466.00 ;
in:3 out:4 type:b q:23.52 q-aq:22.18 q-noVbv:23.52 q-Rceq:0.99 tex:284787 mv:22011 misc:6353 icu:424.25 pcu:2814.50 scu:361.25 ;
in:9 out:5 type:P q:21.94 q-aq:19.37 q-noVbv:21.94 q-Rceq:0.99 tex:589521 mv:12838 misc:3399 icu:3394.25 pcu:204.00 scu:1.75 ;
in:7 out:6 type:B q:22.73 q-aq:21.31 q-noVbv:22.73 q-Rceq:0.99 tex:376395 mv:20908 misc:5391 icu:1138.75 pcu:2305.75 scu:155.50 ;
in:5 out:7 type:b q:23.52 q-aq:22.15 q-noVbv:23.52 q-Rceq:0.99 tex:267303 mv:26748 misc:7184 icu:527.00 pcu:2682.50 scu:390.50 ;
in:6 out:8 type:b q:23.52 q-aq:22.11 q-noVbv:23.52 q-Rceq:0.99 tex:305806 mv:22149 misc:6090 icu:502.00 pcu:2792.75 scu:305.25 ;
in:8 out:9 type:b q:23.52 q-aq:22.21 q-noVbv:23.52 q-Rceq:0.99 tex:244590 mv:26586 misc:7529 icu:324.00 pcu:2809.75 scu:466.25 ;

[benwaggoner]

Originally Posted by Sagittaire

Well I try that too with ToS ... ;-)

--qg-size 64 save 2% size (from my memory)

But it also impairs the flexibility of AQ modes. Adaptive QP can only be done at a 64x64 block instead of 32x32. So if content changes within a 64x64 block, the encoder still has to use a single QP. Smaller qg-size allows for finer tuning for smaller details, but increases signalling overhead. Since metrics (even VMAF) kinda suck at measuring AQ gains, this is something that really need to be subjectively evaluated to see if the 2% gain is a worthwhile tradeoff.

This slower profil seem really good comprise between speed and quality. I wil test that for 1500 and 2000 encoding.

Yeah, slower is the preset where most of the advanced features of x265 and HEVC get enabled, with a lot of early exits and such so the perf hit isn't too bad. I use that as my starting point for pretty much anything.

[Sagittaire]

Originally Posted by Boulder

They do work in CRF mode with cu-tree and AQ enabled.

P/B ratio 1.3

in:0 out:0 type:I q:21.94 q-aq:19.57 q-noVbv:21.94 q-Rceq:0.99 tex:582913 mv:9220 misc:2973 icu:3600.00 pcu:0.00 scu:0.00 ;
in:4 out:1 type:P q:21.94 q-aq:19.24 q-noVbv:21.94 q-Rceq:0.99 tex:614212 mv:13405 misc:3414 icu:3297.50 pcu:301.25 scu:1.25 ;
in:2 out:2 type:B q:23.08 q-aq:21.04 q-noVbv:23.08 q-Rceq:0.99 tex:386297 mv:23848 misc:6130 icu:959.50 pcu:2405.25 scu:235.25 ;
in:1 out:3 type:b q:24.21 q-aq:22.84 q-noVbv:24.21 q-Rceq:0.99 tex:221768 mv:22069 misc:7085 icu:288.50 pcu:2764.00 scu:547.50 ;
in:3 out:4 type:b q:24.21 q-aq:22.85 q-noVbv:24.21 q-Rceq:0.99 tex:234317 mv:23280 misc:6718 icu:393.75 pcu:2778.00 scu:428.25 ;
in:9 out:5 type:P q:21.94 q-aq:19.38 q-noVbv:21.94 q-Rceq:0.99 tex:589089 mv:11898 misc:3306 icu:3406.25 pcu:191.75 scu:2.00 ;
in:7 out:6 type:B q:23.08 q-aq:21.66 q-noVbv:23.08 q-Rceq:0.99 tex:346593 mv:21229 misc:5560 icu:1007.25 pcu:2402.25 scu:190.50 ;
in:5 out:7 type:b q:24.21 q-aq:22.80 q-noVbv:24.21 q-Rceq:0.99 tex:227102 mv:26712 misc:7243 icu:451.00 pcu:2715.25 scu:433.75 ;
in:6 out:8 type:b q:24.21 q-aq:22.80 q-noVbv:24.21 q-Rceq:0.99 tex:258433 mv:22466 misc:6381 icu:337.75 pcu:2954.25 scu:308.00 ;
in:8 out:9 type:b q:24.21 q-aq:22.78 q-noVbv:24.21 q-Rceq:0.99 tex:212971 mv:24199 misc:6965 icu:322.75 pcu:2758.50 scu:518.75 ;

P/B ratio 1.2

in:0 out:0 type:I q:21.94 q-aq:19.57 q-noVbv:21.94 q-Rceq:0.99 tex:582913 mv:9220 misc:2973 icu:3600.00 pcu:0.00 scu:0.00 ;
in:4 out:1 type:P q:21.94 q-aq:19.24 q-noVbv:21.94 q-Rceq:0.99 tex:614212 mv:13405 misc:3414 icu:3297.50 pcu:301.25 scu:1.25 ;
in:2 out:2 type:B q:22.73 q-aq:20.71 q-noVbv:22.73 q-Rceq:0.99 tex:414031 mv:24937 misc:6025 icu:972.75 pcu:2444.75 scu:182.50 ;
in:1 out:3 type:b q:23.52 q-aq:22.14 q-noVbv:23.52 q-Rceq:0.99 tex:271359 mv:22093 misc:6977 icu:381.75 pcu:2752.25 scu:466.00 ;
in:3 out:4 type:b q:23.52 q-aq:22.18 q-noVbv:23.52 q-Rceq:0.99 tex:284787 mv:22011 misc:6353 icu:424.25 pcu:2814.50 scu:361.25 ;
in:9 out:5 type:P q:21.94 q-aq:19.37 q-noVbv:21.94 q-Rceq:0.99 tex:589521 mv:12838 misc:3399 icu:3394.25 pcu:204.00 scu:1.75 ;
in:7 out:6 type:B q:22.73 q-aq:21.31 q-noVbv:22.73 q-Rceq:0.99 tex:376395 mv:20908 misc:5391 icu:1138.75 pcu:2305.75 scu:155.50 ;
in:5 out:7 type:b q:23.52 q-aq:22.15 q-noVbv:23.52 q-Rceq:0.99 tex:267303 mv:26748 misc:7184 icu:527.00 pcu:2682.50 scu:390.50 ;
in:6 out:8 type:b q:23.52 q-aq:22.11 q-noVbv:23.52 q-Rceq:0.99 tex:305806 mv:22149 misc:6090 icu:502.00 pcu:2792.75 scu:305.25 ;
in:8 out:9 type:b q:23.52 q-aq:22.21 q-noVbv:23.52 q-Rceq:0.99 tex:244590 mv:26586 misc:7529 icu:324.00 pcu:2809.75 scu:466.25 ;

yes, my bad. I recheck my test and you are right. However I understood my error. For me, --pbratio 1.00 mean the same average quantizer for pframes and bframes. And it's the case for Npass mode but not for crf mode:


- In crf mode

D:\Mes Logiciels\Codec\x265>x265.exe --input C:\ToS_1920x800_xdither.y4m --output ToS-1.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --crf 25.0 --preset fast --qcomp 0.60 --bframes 5 --b-adapt 2 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --tune ssim --ref 5 --limit-refs 3 --rd-refine --hevc-aq --qp-adaptation-range 1.0 --qg-size 64 --merange 64 --deblock -1,-1 --limit-sao --frames 1332 --seek 214 --ipratio 1.40 --pbratio 1.00
x265 [info]: frame I: 16, Avg QP:24.65 kb/s: 17300.27 PSNR Mean: Y:43.462 U:45.390 V:45.166 SSIM Mean: 0.978414 (16.658dB)
x265 [info]: frame P: 275, Avg QP:28.94 kb/s: 2716.00 PSNR Mean: Y:40.881 U:43.748 V:43.285 SSIM Mean: 0.967960 (14.943dB)
x265 [info]: frame B: 1041, Avg QP:33.59 kb/s: 542.67 PSNR Mean: Y:40.540 U:44.024 V:43.546 SSIM Mean: 0.966582 (14.760dB)
x265 [info]: Weighted P-Frames: Y:2.2% UV:1.8%
x265 [info]: consecutive B-frames: 9.3% 1.4% 2.4% 25.4% 32.3% 29.2%

encoded 1332 frames in 128.92s (10.33 fps), 1192.66 kb/s, Avg QP:32.52, Global PSNR: 41.421, SSIM Mean Y: 0.9670086 (14.816 dB)

- In NPass mode

D:\Mes Logiciels\Codec\x265>x265.exe --input C:\ToS_1920x800_xdither.y4m --output ToS-1.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --bitrate 1192 --pass 3 --preset fast --qcomp 0.60 --bframes 5 --b-adapt 2 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --tune ssim --ref 5 --limit-refs 3 --rd-refine --hevc-aq --qp-adaptation-range 1.0 --qg-size 64 --merange 64 --deblock -1,-1 --limit-sao --frames 1332 --seek 214 --ipratio 1.40 --pbratio 1.00
x265 [info]: frame I: 16, Avg QP:28.06 kb/s: 10449.46 PSNR Mean: Y:41.454 U:43.768 V:43.453 SSIM Mean: 0.966755 (14.783dB)
x265 [info]: frame P: 275, Avg QP:30.44 kb/s: 2208.40 PSNR Mean: Y:39.873 U:42.736 V:42.292 SSIM Mean: 0.960179 (13.999dB)
x265 [info]: frame B: 1041, Avg QP:30.23 kb/s: 776.34 PSNR Mean: Y:40.138 U:43.218 V:42.801 SSIM Mean: 0.961323 (14.125dB)
x265 [info]: Weighted P-Frames: Y:2.5% UV:2.2%
x265 [info]: consecutive B-frames: 9.3% 1.4% 2.4% 25.4% 32.3% 29.2%

encoded 1332 frames in 145.19s (9.17 fps), 1188.19 kb/s, Avg QP:30.24, Global PSNR: 40.803, SSIM Mean Y: 0.9611523 (14.106 dB)

[mandarinka]

Originally Posted by Blue_MiSfit

WOW.

That is truly impressive. This reinforces my belief that film grain modeling is enormously important and will really make 1 Mbps 1080p totally viable. This is a game changing feature for AV1!

Do any of the encoders have / plan to integrate this in-loop during encoding? Working with YUV intermediates is pretty painful.

If you leave the denoising part to the encoder, it will probably use something bad that will remove legitimate non-noise detail from the base picture under it. Most standalone denoisers actually do, so using some of the rare exceptions tuned for better quality should be done.

I guess the huge problem here is now that even the good denoisers can't be made to not remove any details if you use the strengths that are required to remove all the noise content.

[Sagittaire]

Originally Posted by mandarinka

I guess the huge problem here is now that even the good denoisers can't be made to not remove any details if you use the strengths that are required to remove all the noise content.

Well retain grain/noise is bitrate problem. All the codec can retain noise if you use suffisant bitrate for that. Retain grain is problem at low bitrate and at low bitrate, the codec itself remove detail/grain/noise. And soft denoising is not problem in this case simply because codec will make strong denoising by itself. For me FGM is really usefull in "low bitrate" situation exactly like for this "1000 kbps / 1080p" challenge encoding.

[benwaggoner]

Originally Posted by Sagittaire

Well retain grain/noise is bitrate problem. All the codec can retain noise if you use suffisant bitrate for that. Retain grain is problem at low bitrate and at low bitrate, the codec itself remove detail/grain/noise. And soft denoising is not problem in this case simply because codec will make strong denoising by itself. For me FGM is really usefull in "low bitrate" situation exactly like for this "1000 kbps / 1080p" challenge encoding.

Yeah. FGM alone can probably save 20% off this particular challenge.