Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 1st May 2020, 13:28   #1  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,484
Test with analysis-reuse in x265?

Someone have made test with these x265 functionaly?

Code:
   --analysis-save <filename>    Dump analysis info into the specified file. Default Disabled
   --analysis-load <filename>    Load analysis buffers from the file specified. Default Disabled
   --analysis-reuse-file <filename>    Specify file name used for either dumping or reading analysis data. Deault x265_analysis.dat
   --analysis-reuse-level <1..10>      Level of analysis reuse indicates amount of info stored/reused in save/load mode, 1:least..10:most. Now deprecated. Default 0
   --analysis-save-reuse-level <1..10> Indicates the amount of analysis info stored in save mode, 1:least..10:most. Default 0
   --analysis-load-reuse-level <1..10> Indicates the amount of analysis info reused in load mode, 1:least..10:most. Default 0
   --refine-analysis-type <string>     Reuse anlaysis information received through API call. Supported options are avc and hevc. Default disabled - 0
   --scale-factor <int>          Specify factor by which input video is scaled down for analysis save mode. Default 0
   --refine-intra <0..4>         Enable intra refinement for encode that uses analysis-load.
                                    - 0 : Forces both mode and depth from the save encode.
                                    - 1 : Functionality of (0) + evaluate all intra modes at min-cu-size's depth when current depth is one smaller than min-cu-size's depth.
                                    - 2 : Functionality of (1) + irrespective of size evaluate all angular modes when the save encode decides the best mode as angular.
                                    - 3 : Functionality of (1) + irrespective of size evaluate all intra modes.
                                    - 4 : Re-evaluate all intra blocks, does not reuse data from save encode.
                                Default:0
   --refine-inter <0..3>         Enable inter refinement for encode that uses analysis-load.
                                    - 0 : Forces both mode and depth from the save encode.
                                    - 1 : Functionality of (0) + evaluate all inter modes at min-cu-size's depth when current depth is one smaller than
                                          min-cu-size's depth. When save encode decides the current block as skip(for all sizes) evaluate skip/merge.
                                    - 2 : Functionality of (1) + irrespective of size restrict the modes evaluated when specific modes are decided as the best mode by the save encode.
                                    - 3 : Functionality of (1) + irrespective of size evaluate all inter modes.
                                Default:0
   --[no-]dynamic-refine         Dynamically changes refine-inter level for each CU. Default disabled
   --refine-mv <1..3>            Enable mv refinement for load mode. Default 1
   --refine-ctu-distortion       Store/normalize ctu distortion in analysis-save/load.
                                    - 0 : Disabled.
                                    - 1 : Store/Load ctu distortion to/from the file specified in analysis-save/load.
                                Default 0 - Disabled
I make some test and I can make for exemple a really high quality encoding (placebo for exemple) at 1000 kbps and remake encoding at different bitrate but with the same analysis level at extremely high speed (really higher speed than fastest preset).

Anyway I don't understand how use refine like --refine-intra, --refine-inter or --refine-mv?

It's possible to change psy, deblock, sao seeting for new high speed "placebo level" encoding?
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9

Last edited by Sagittaire; 1st May 2020 at 14:03.
Sagittaire is offline   Reply With Quote
Old 1st May 2020, 21:37   #2  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,988
I have not personally messed with this, but my understanding is that this is extremely useful for ABR encoding. You save the analysis from a lower res encode and re-use it (with refinement) on a high res encode.

So for example if you have a 4k source and want to make 1080p and 4k variants, you do the 1080p encode, save the analysis data, then re-use it when doing the 4k encode. This saves a ton of largely redundant analysis on the 4k layer. There is a quality impact, but it's not much, and the compute you save can be sent on higher quality analysis

There was a great talk at Demuxed 2018 about this https://www.youtube.com/watch?v=-yPOE2F9fDI from Alex Giladi from Comcast
Blue_MiSfit is offline   Reply With Quote
Old 2nd May 2020, 00:17   #3  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,484
Quote:
Originally Posted by Blue_MiSfit View Post
I have not personally messed with this, but my understanding is that this is extremely useful for ABR encoding. You save the analysis from a lower res encode and re-use it (with refinement) on a high res encode.

So for example if you have a 4k source and want to make 1080p and 4k variants, you do the 1080p encode, save the analysis data, then re-use it when doing the 4k encode. This saves a ton of largely redundant analysis on the 4k layer. There is a quality impact, but it's not much, and the compute you save can be sent on higher quality analysis

There was a great talk at Demuxed 2018 about this https://www.youtube.com/watch?v=-yPOE2F9fDI from Alex Giladi from Comcast
Yes.

I think it's pro profil too. For streaming like Netflix or Amazon, you make only one high quality encoding and you can remake all reencoding at the bitrate you want with really high speed and high quality. It's certainely like that Netflix can adapt quality for all user.
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9
Sagittaire is offline   Reply With Quote
Old 2nd May 2020, 00:47   #4  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,988
Well, maybe. I think it's more about just saving or reallocating compute when encoding a given ABR ladder.
Blue_MiSfit is offline   Reply With Quote
Old 2nd May 2020, 01:11   #5  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,484
Quote:
Originally Posted by Blue_MiSfit View Post
Well, maybe. I think it's more about just saving or reallocating compute when encoding a given ABR ladder.
https://forum.doom9.org/showthread.p...=analysis-save

https://damienschroeder.wordpress.co...ttp-streaming/

https://x265.com/uhdkit/
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9

Last edited by Sagittaire; 2nd May 2020 at 01:15.
Sagittaire is offline   Reply With Quote
Old 3rd May 2020, 12:39   #6  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,484
Here little demonstration:

Quote:
D:\Mes Logiciels\Codec\x265>x265.exe --input ToS_1920x800_xdither.y4m --output ToS-2.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --bitrate 1000 --qcomp 0.50 --pass 3 --stats ToS-1000.log --preset placebo --tune ssim --no-limit-modes --rd-refine --hevc-aq --qp-adaptation-range 1.0 --bframes 3 --b-adapt 2 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --csv ToS-2.csv --analysis-save x265_analysis.dat --analysis-save-reuse-level 10
x265 [info]: frame I: 228, Avg QP:26.88 kb/s: 9556.60 PSNR Mean: Y:43.543 U:45.807 V:45.829 SSIM Mean: 0.971004 (15.377dB)
x265 [info]: frame P: 4945, Avg QP:29.57 kb/s: 2020.87 PSNR Mean: Y:41.571 U:44.831 V:44.699 SSIM Mean: 0.965905 (14.673dB)
x265 [info]: frame B: 12447, Avg QP:31.41 kb/s: 437.84 PSNR Mean: Y:41.589 U:44.997 V:44.938 SSIM Mean: 0.964320 (14.476dB)
x265 [info]: Weighted P-Frames: Y:4.7% UV:3.5%
x265 [info]: Weighted B-Frames: Y:3.6% UV:2.5%
x265 [info]: consecutive B-frames: 9.5% 11.0% 8.9% 70.6%

encoded 17620 frames in 71089.51s (0.25 fps), 1000.11 kb/s, Avg QP:30.83, Global PSNR: 42.438, SSIM Mean Y: 0.9648512 (14.541 dB)

D:\Mes Logiciels\Codec\x265>x265.exe --input ToS_1920x800_xdither.y4m --output ToS-3.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --bitrate 1000 --qcomp 0.50 --pass 2 --stats ToS-1000.log --preset placebo --tune ssim --no-limit-modes --rd-refine --hevc-aq --qp-adaptation-range 1.0 --bframes 3 --b-adapt 2 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --csv ToS-3.csv --analysis-load x265_analysis.dat --analysis-load-reuse-level 10

x265 [info]: frame I: 228, Avg QP:27.31 kb/s: 9479.60 PSNR Mean: Y:43.199 U:45.623 V:45.615 SSIM Mean: 0.969150 (15.108dB)
x265 [info]: frame P: 4945, Avg QP:29.71 kb/s: 2036.00 PSNR Mean: Y:41.281 U:44.737 V:44.589 SSIM Mean: 0.964907 (14.548dB)
x265 [info]: frame B: 12447, Avg QP:31.40 kb/s: 428.81 PSNR Mean: Y:41.372 U:44.946 V:44.872 SSIM Mean: 0.963784 (14.411dB)
x265 [info]: Weighted P-Frames: Y:4.7% UV:3.5%
x265 [info]: Weighted B-Frames: Y:3.6% UV:2.5%
x265 [info]: consecutive B-frames: 9.5% 11.0% 8.9% 70.6%

encoded 17620 frames in 948.83s (18.57 fps), 996.98 kb/s, Avg QP:30.87, Global PSNR: 42.240, SSIM Mean Y: 0.9641687 (14.457 dB)

There is little quality loss between the complete pass and analysis-load-reuse pass, but the speed is increase by x100 for placebo encoding level. In fact here my command is limited by HDD, but if I choose SSD directory for lossless y4m source, I have more than 30 fps. My little i5-3550 with placebo profil have higher speed than AMD 3990X. And you can choose other bitrate at the same speed for second pass. I make test to see if I can change psy setting, blocking setting in high speed second pass.

It's really powerfull fonctionality ... ;-)
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9

Last edited by Sagittaire; 3rd May 2020 at 14:50.
Sagittaire is offline   Reply With Quote
Old 9th May 2020, 22:18   #7  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,484
For those who are interested in this functionality:

1) soon a demonstration in the benwagonner challenge

2) You must use --refine-intra --refine-inter with --analysis-load and --analysis-load-reuse-level only if you change bitrate for reuse encoding for example and for make intra and inter refinement at this new bitrate and obtain better quality. Anyway with these option, you have strong speed penality (and high speed boost is the main interest of reuse reencoding). More refine level imply more quality but less speed. --refine-intra 4 --refine-inter 3 is the highest refinement level.

3) You can't use --refine-intra --refine-inter option with --rd-refine option (bug?)

4) --refine-mv is only usefull if you change resolution (not tested)
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9
Sagittaire is offline   Reply With Quote
Old 9th May 2020, 22:37   #8  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by Blue_MiSfit View Post
Well, maybe. I think it's more about just saving or reallocating compute when encoding a given ABR ladder.
There is nothing "just" about that! Cutting encode compute in half at the same quality allows for either turning up quality, speeding up publishing, or reducing costs.

At global scale with a global content library, even a 20% reduction in encode compute can save millions a year. As can a 5% reduction in average bitrate gain from more efficient use of compute. Reducing file size saves on storage and bandwidth to the viewer, and improves edge cache effeciency.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 9th May 2020, 22:39   #9  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by Sagittaire View Post
For those who are interested in this functionality:

1) soon a demonstration in the benwagonner challenge

2) You must use --refine-intra --refine-inter with --analysis-load and --analysis-load-reuse-level only if you change bitrate for reuse encoding for example and for make intra and inter refinement at this new bitrate and obtain better quality. Anyway with these option, you have strong speed penality (and high speed boost is the main interest of reuse reencoding). More refine level imply more quality but less speed. --refine-intra 4 --refine-inter 3 is the highest refinement level.

3) You can't use --refine-intra --refine-inter option with --rd-refine option (bug?)

4) --refine-mv is only usefull if you change resolution (not tested)
Have you tested the quality/speed tradeoffs of the different refine levels? It's always interesting to see the optimal parameters for the best quality @ perf, and it can be quite unintuitive as to what that'll be in practice. And can vary quite a bit depending on number of cores and the SIMD implementation.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 9th May 2020, 22:41   #10  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,484
Quote:
Originally Posted by benwaggoner View Post
Have you tested the quality/speed tradeoffs of the different refine levels? It's always interesting to see the optimal parameters for the best quality @ perf, and it can be quite unintuitive as to what that'll be in practice. And can vary quite a bit depending on number of cores and the SIMD implementation.
yes I make that actualy. I can post little exemple.
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9
Sagittaire is offline   Reply With Quote
Old 9th May 2020, 23:52   #11  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,484
Little test:

1) first classic way encoding, at slower profil, in 3 passes. You can see that third pass produce better quality than second pass (exactly the same setting for second and third pass)
first pass produce really high metric because first pass is CBR encoding, you have really bad quality in high motion and high quality for low motion (yes it's the best RC ... for metric ... in most case)


Quote:
D:\Mes Logiciels\Codec\x265>REM >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

D:\Mes Logiciels\Codec\x265>REM >> Initial encoding at 450 kbps

D:\Mes Logiciels\Codec\x265>REM >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

D:\Mes Logiciels\Codec\x265>x265.exe --input hp.yuv --output HP-450.265 --input-res 720x304 --output-depth 10 --fps 25 --bitrate 450 --qcomp 0.60 --pass 1 --slow-firstpass --stats hp.log --preset slower --ref 5 --limit-refs 3 --rd-refine --tune ssim --hevc-aq --qp-adaptation-range 1.0 --qg-size 64 --bframes 3 --b-adapt 2 --min-keyint 1 --psnr --ssim --analysis-save HP_450_R10-abr1_analysis.dat --analysis-save-reuse-level 10
e tmvp b-intra strong-intra-smoothing
x265 [info]: tools: deblock sao stats-write
x265 [info]: frame I: 84, Avg QP:23.31 kb/s: 2219.89 PSNR Mean: Y:45.263 U:47.668 V:48.594 SSIM Mean: 0.980535 (17.108dB)
x265 [info]: frame P: 873, Avg QP:24.43 kb/s: 902.29 PSNR Mean: Y:44.410 U:46.938 V:47.950 SSIM Mean: 0.976246 (16.243dB)
x265 [info]: frame B: 2119, Avg QP:29.14 kb/s: 188.15 PSNR Mean: Y:43.025 U:46.585 V:47.701 SSIM Mean: 0.971308 (15.422dB)
x265 [info]: Weighted P-Frames: Y:17.3% UV:12.6%
x265 [info]: Weighted B-Frames: Y:17.4% UV:13.3%
x265 [info]: consecutive B-frames: 14.3% 7.3% 21.0% 57.4%

encoded 3076 frames in 686.40s (4.48 fps), 446.31 kb/s, Avg QP:27.64, Global PSNR: 44.423, SSIM Mean Y: 0.9729615 (15.680 dB)

D:\Mes Logiciels\Codec\x265>x265.exe --input hp.yuv --output HP-450.265 --input-res 720x304 --output-depth 10 --fps 25 --bitrate 450 --qcomp 0.60 --pass 3 --stats hp.log --preset slower --ref 5 --limit-refs 3 --rd-refine --tune ssim --hevc-aq --qp-adaptation-range 1.0 --qg-size 64 --bframes 3 --b-adapt 2 --min-keyint 1 --psnr --ssim --analysis-save HP_450_R10-abr2_analysis.dat --analysis-save-reuse-level 10
x265 [info]: frame I: 84, Avg QP:22.50 kb/s: 2211.80 PSNR Mean: Y:45.599 U:47.925 V:48.853 SSIM Mean: 0.982087 (17.468dB)
x265 [info]: frame P: 873, Avg QP:24.87 kb/s: 752.71 PSNR Mean: Y:43.717 U:46.401 V:47.398 SSIM Mean: 0.974214 (15.886dB)
x265 [info]: frame B: 2119, Avg QP:26.90 kb/s: 252.56 PSNR Mean: Y:42.978 U:46.239 V:47.311 SSIM Mean: 0.971203 (15.407dB)
x265 [info]: Weighted P-Frames: Y:17.3% UV:12.6%
x265 [info]: Weighted B-Frames: Y:17.4% UV:13.3%
x265 [info]: consecutive B-frames: 14.3% 7.3% 21.0% 57.4%

encoded 3076 frames in 727.54s (4.23 fps), 448.01 kb/s, Avg QP:26.21, Global PSNR: 44.158, SSIM Mean Y: 0.9723548 (15.584 dB)

D:\Mes Logiciels\Codec\x265>x265.exe --input hp.yuv --output HP-450.265 --input-res 720x304 --output-depth 10 --fps 25 --bitrate 450 --qcomp 0.60 --pass 2 --stats hp.log --preset slower --ref 5 --limit-refs 3 --rd-refine --tune ssim --hevc-aq --qp-adaptation-range 1.0 --qg-size 64 --bframes 3 --b-adapt 2 --min-keyint 1 --psnr --ssim --analysis-save HP_450_R10-abr3_analysis.dat --analysis-save-reuse-level 10
x265 [info]: frame I: 84, Avg QP:22.83 kb/s: 2137.29 PSNR Mean: Y:45.409 U:47.790 V:48.707 SSIM Mean: 0.981198 (17.258dB)
x265 [info]: frame P: 873, Avg QP:24.62 kb/s: 766.48 PSNR Mean: Y:43.925 U:46.561 V:47.562 SSIM Mean: 0.974980 (16.017dB)
x265 [info]: frame B: 2119, Avg QP:26.58 kb/s: 249.12 PSNR Mean: Y:43.163 U:46.397 V:47.479 SSIM Mean: 0.971926 (15.517dB)
x265 [info]: Weighted P-Frames: Y:17.3% UV:12.6%
x265 [info]: Weighted B-Frames: Y:17.4% UV:13.3%
x265 [info]: consecutive B-frames: 14.3% 7.3% 21.0% 57.4%

encoded 3076 frames in 737.86s (4.17 fps), 447.51 kb/s, Avg QP:25.92, Global PSNR: 44.333, SSIM Mean Y: 0.9730459 (15.694 dB)


2) Second reuse encoding at 900 kbps with previous analyse reuse from 450 kbps encoding

- And 75.45 fps at 900 kbps versus 4.17 fps at 450 kbps with the same search level (slower) with --refine-intra 0 --refine-inter 0: 45.96 dB
- And 24.41 fps at 900 kbps versus 4.17 fps at 450 kbps with the same search level (slower) with --refine-intra 1 --refine-inter 1: 46.17 dB
- And 12.08 fps at 900 kbps versus 4.17 fps at 450 kbps with the same search level (slower) with --refine-intra 2 --refine-inter 2: 46.34 dB
- And 08.22 fps at 900 kbps versus 4.17 fps at 450 kbps with the same search level (slower) with --refine-intra 3 --refine-inter 3: 46.39 dB
- And 08.01 fps at 900 kbps versus 4.17 fps at 450 kbps with the same search level (slower) with --refine-intra 4 --refine-inter 3: 46.40 dB
- And 12.12 fps at 900 kbps versus 4.17 fps at 450 kbps with the same search level (slower) with --refine-intra 4 --dynamic-refine: 46.25 dB



Quote:
D:\Mes Logiciels\Codec\x265>REM >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

D:\Mes Logiciels\Codec\x265>REM >> Reuse encoding at 900 kbps

D:\Mes Logiciels\Codec\x265>REM >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

D:\Mes Logiciels\Codec\x265>x265.exe --input hp.yuv --output HP-450.265 --input-res 720x304 --output-depth 10 --fps 25 --bitrate 900 --qcomp 0.60 --pass 2 --stats hp.log --preset slower --ref 5 --limit-refs 3 --tune ssim --hevc-aq --qp-adaptation-range 1.0 --qg-size 64 --bframes 3 --b-adapt 2 --min-keyint 1 --psnr --ssim --analysis-load HP_450_R10-abr3_analysis.dat --analysis-load-reuse-level 10
yuv [info]: 720x304 fps 25000/1000 i420p8 frames 0 - 3075 of 3076
x265 [info]: frame I: 84, Avg QP:18.71 kb/s: 3388.21 PSNR Mean: Y:47.561 U:49.414 V:50.373 SSIM Mean: 0.988040 (19.223dB)
x265 [info]: frame P: 873, Avg QP:20.08 kb/s: 1544.82 PSNR Mean: Y:45.897 U:48.238 V:49.303 SSIM Mean: 0.983313 (17.776dB)
x265 [info]: frame B: 2119, Avg QP:22.10 kb/s: 531.47 PSNR Mean: Y:44.683 U:47.811 V:48.992 SSIM Mean: 0.979605 (16.905dB)
x265 [info]: Weighted P-Frames: Y:17.3% UV:12.6%
x265 [info]: Weighted B-Frames: Y:17.4% UV:13.3%
x265 [info]: consecutive B-frames: 14.3% 7.3% 21.0% 57.4%

encoded 3076 frames in 40.77s (75.45 fps), 897.08 kb/s, Avg QP:21.43, Global PSNR: 45.966, SSIM Mean Y: 0.9808877 (17.187 dB)

Quote:
D:\Mes Logiciels\Codec\x265>x265.exe --input hp.yuv --output HP-450.265 --input-res 720x304 --output-depth 10 --fps 25 --bitrate 900 --qcomp 0.60 --pass 2 --stats hp.log --preset slower --ref 5 --limit-refs 3 --tune ssim --hevc-aq --qp-adaptation-range 1.0 --qg-size 64 --bframes 3 --b-adapt 2 --min-keyint 1 --psnr --ssim --analysis-load HP_450_R10-abr3_analysis.dat --analysis-load-reuse-level 10 --refine-intra 1 --refine-inter 1
x265 [info]: frame I: 84, Avg QP:19.60 kb/s: 3050.04 PSNR Mean: Y:47.028 U:48.999 V:49.965 SSIM Mean: 0.986703 (18.762dB)
x265 [info]: frame P: 873, Avg QP:20.95 kb/s: 1512.18 PSNR Mean: Y:46.075 U:48.026 V:49.074 SSIM Mean: 0.983574 (17.845dB)
x265 [info]: frame B: 2119, Avg QP:22.92 kb/s: 558.59 PSNR Mean: Y:45.039 U:47.649 V:48.804 SSIM Mean: 0.980624 (17.127dB)
x265 [info]: Weighted P-Frames: Y:17.3% UV:12.6%
x265 [info]: Weighted B-Frames: Y:17.4% UV:13.3%
x265 [info]: consecutive B-frames: 14.3% 7.3% 21.0% 57.4%

encoded 3076 frames in 126.00s (24.41 fps), 897.26 kb/s, Avg QP:22.27, Global PSNR: 46.128, SSIM Mean Y: 0.9816273 (17.358 dB)

Quote:
D:\Mes Logiciels\Codec\x265>x265.exe --input hp.yuv --output HP-450.265 --input-res 720x304 --output-depth 10 --fps 25 --bitrate 900 --qcomp 0.60 --pass 2 --stats hp.log --preset slower --ref 5 --limit-refs 3 --tune ssim --hevc-aq --qp-adaptation-range 1.0 --qg-size 64 --bframes 3 --b-adapt 2 --min-keyint 1 --psnr --ssim --analysis-load HP_450_R10-abr3_analysis.dat --analysis-load-reuse-level 10 --refine-intra 2 --refine-inter 2
x265 [info]: frame I: 84, Avg QP:19.33 kb/s: 3138.92 PSNR Mean: Y:47.298 U:49.248 V:50.213 SSIM Mean: 0.987410 (19.000dB)
x265 [info]: frame P: 873, Avg QP:20.72 kb/s: 1502.33 PSNR Mean: Y:46.314 U:48.223 V:49.293 SSIM Mean: 0.984357 (18.057dB)
x265 [info]: frame B: 2119, Avg QP:22.65 kb/s: 559.64 PSNR Mean: Y:45.255 U:47.828 V:49.001 SSIM Mean: 0.981458 (17.318dB)
x265 [info]: Weighted P-Frames: Y:17.3% UV:12.6%
x265 [info]: Weighted B-Frames: Y:17.4% UV:13.3%
x265 [info]: consecutive B-frames: 14.3% 7.3% 21.0% 57.4%

encoded 3076 frames in 254.58s (12.08 fps), 897.62 kb/s, Avg QP:22.01, Global PSNR: 46.345, SSIM Mean Y: 0.9824434 (17.556 dB)

Quote:
D:\Mes Logiciels\Codec\x265>x265.exe --input hp.yuv --output HP-450.265 --input-res 720x304 --output-depth 10 --fps 25 --bitrate 900 --qcomp 0.60 --pass 2 --stats hp.log --preset slower --ref 5 --limit-refs 3 --tune ssim --hevc-aq --qp-adaptation-range 1.0 --qg-size 64 --bframes 3 --b-adapt 2 --min-keyint 1 --psnr --ssim --analysis-load HP_450_R10-abr3_analysis.dat --analysis-load-reuse-level 10 --refine-intra 3 --refine-inter 3
yuv [info]: 720x304 fps 25000/1000 i420p8 frames 0 - 3075 of 3076
x265 [info]: frame I: 84, Avg QP:19.27 kb/s: 3151.98 PSNR Mean: Y:47.359 U:49.338 V:50.312 SSIM Mean: 0.987559 (19.051dB)
x265 [info]: frame P: 873, Avg QP:20.70 kb/s: 1498.98 PSNR Mean: Y:46.344 U:48.251 V:49.326 SSIM Mean: 0.984447 (18.082dB)
x265 [info]: frame B: 2119, Avg QP:22.64 kb/s: 560.64 PSNR Mean: Y:45.312 U:47.861 V:49.041 SSIM Mean: 0.981677 (17.370dB)
x265 [info]: Weighted P-Frames: Y:17.3% UV:12.6%
x265 [info]: Weighted B-Frames: Y:17.4% UV:13.3%
x265 [info]: consecutive B-frames: 14.3% 7.3% 21.0% 57.4%

encoded 3076 frames in 374.06s (8.22 fps), 897.71 kb/s, Avg QP:22.00, Global PSNR: 46.391, SSIM Mean Y: 0.9826236 (17.600 dB)

Quote:
D:\Mes Logiciels\Codec\x265>x265.exe --input hp.yuv --output HP-450.265 --input-res 720x304 --output-depth 10 --fps 25 --bitrate 900 --qcomp 0.60 --pass 2 --stats hp.log --preset slower --ref 5 --limit-refs 3 --tune ssim --hevc-aq --qp-adaptation-range 1.0 --qg-size 64 --bframes 3 --b-adapt 2 --min-keyint 1 --psnr --ssim --analysis-load HP_450_R10-abr3_analysis.dat --analysis-load-reuse-level 10 --refine-intra 4 --refine-inter 3
x265 [info]: frame I: 84, Avg QP:19.27 kb/s: 3128.68 PSNR Mean: Y:47.433 U:49.345 V:50.331 SSIM Mean: 0.987750 (19.119dB)
x265 [info]: frame P: 873, Avg QP:20.70 kb/s: 1500.04 PSNR Mean: Y:46.349 U:48.255 V:49.329 SSIM Mean: 0.984472 (18.089dB)
x265 [info]: frame B: 2119, Avg QP:22.63 kb/s: 561.08 PSNR Mean: Y:45.316 U:47.861 V:49.042 SSIM Mean: 0.981702 (17.376dB)
x265 [info]: Weighted P-Frames: Y:17.3% UV:12.6%
x265 [info]: Weighted B-Frames: Y:17.4% UV:13.3%
x265 [info]: consecutive B-frames: 14.3% 7.3% 21.0% 57.4%

encoded 3076 frames in 384.22s (8.01 fps), 897.68 kb/s, Avg QP:21.99, Global PSNR: 46.397, SSIM Mean Y: 0.9826534 (17.608 dB)

3) Comparison control with classic 3 passes encoding at 900 kbps with exactly same profil than classic 3 passes encoding at 450 kbps

Result is really impressive for reuse encoding at 900 kbps (from 450 encoding): really equivalent metric performance at really higher speed.

Quote:
D:\Mes Logiciels\Codec\x265>x265.exe --input hp.yuv --output HP-450.265 --input-res 720x304 --output-depth 10 --fps 25 --bitrate 900 --qcomp 0.60 --pass 2 --stats hp2.log --preset slower --ref 5 --limit-refs 3 --rd-refine --tune ssim --hevc-aq --qp-adaptation-range 1.0 --qg-size 64 --bframes 3 --b-adapt 2 --min-keyint 1 --psnr --ssim
x265 [info]: frame I: 84, Avg QP:18.54 kb/s: 3379.67 PSNR Mean: Y:47.858 U:49.679 V:50.661 SSIM Mean: 0.988751 (19.489dB)
x265 [info]: frame P: 873, Avg QP:20.17 kb/s: 1510.20 PSNR Mean: Y:46.463 U:48.423 V:49.504 SSIM Mean: 0.984823 (18.188dB)
x265 [info]: frame B: 2119, Avg QP:22.14 kb/s: 541.89 PSNR Mean: Y:45.334 U:47.994 V:49.186 SSIM Mean: 0.981829 (17.406dB)
x265 [info]: Weighted P-Frames: Y:17.3% UV:12.6%
x265 [info]: Weighted B-Frames: Y:17.4% UV:13.3%
x265 [info]: consecutive B-frames: 14.3% 7.3% 21.0% 57.4%

encoded 3076 frames in 1054.08s (2.92 fps), 894.20 kb/s, Avg QP:21.48, Global PSNR: 46.477, SSIM Mean Y: 0.9828678 (17.662 dB)
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9

Last edited by Sagittaire; 10th May 2020 at 03:11.
Sagittaire is offline   Reply With Quote
Old 10th May 2020, 00:03   #12  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Wow, the average frame QP really drops in the second round.

With 57% of consecutive b-frames being 5, I would think that more would increase efficiency.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 10th May 2020, 00:15   #13  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,484
Quote:
Originally Posted by benwaggoner View Post
Wow, the average frame QP really drops in the second round.

With 57% of consecutive b-frames being 5, I would think that more would increase efficiency.
PbBbP is really efficient sequence. I have 1% save size between 3 bframes and 5 bframes in your ToS challenge. Higher bframe number is placebo for metric. But I have little subjective problem (temporal stability) in your ToS sample with 7 bframes (and higher).
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9

Last edited by Sagittaire; 10th May 2020 at 00:27.
Sagittaire is offline   Reply With Quote
Old 10th May 2020, 00:25   #14  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,484
Quote:
Originally Posted by benwaggoner View Post
Wow, the average frame QP really drops in the second round.
yes first encoding at 450 kbps for produce analysis.dat at 4 fps with classic encoding and after all the other encoding at 900 kbps or xxx kbps (analysis.dat reuse encoding) with 80 fps for refine 0 level.

In fact you can make first ultraplacebo encoding (0.1 fps) and make all the other encoding (with this analyse level) at ultra fast speed at all the bitrate you want. You can change all Rate Control level like you want too and I will test for AQ, deblock level, PSY (seem to be OK but perhaps that's refine is really necessary here).

I think this option will be really usefull if you want make test on particular option with really high speed encoding even for a complete movie.

and last but not least, with source in 1080p, the lossless source saturate my HDD with a little i5 3550 and I must choose SSD directory for lossless source. I can imagine that Ryzen 3700X can saturate SSD sata interface with 1080p lossless source with Reuse encoding.
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9

Last edited by Sagittaire; 10th May 2020 at 01:12.
Sagittaire is offline   Reply With Quote
Old 10th May 2020, 03:12   #15  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,484
Update: encoding control

3) Comparison control with classic 3 passes encoding at 900 kbps with exactly same profil than classic 3 passes encoding at 450 kbps

Result is really impressive for reuse encoding at 900 kbps (from 450 encoding): really equivalent metric performance at really higher speed. I will perhaps simply reuse 1000 kbps for make 1500 and 2000 kbps encoding for benwagonner challenge... ;-)

Quote:
D:\Mes Logiciels\Codec\x265>x265.exe --input hp.yuv --output HP-450.265 --input-res 720x304 --output-depth 10 --fps 25 --bitrate 900 --qcomp 0.60 --pass 2 --stats hp2.log --preset slower --ref 5 --limit-refs 3 --rd-refine --tune ssim --hevc-aq --qp-adaptation-range 1.0 --qg-size 64 --bframes 3 --b-adapt 2 --min-keyint 1 --psnr --ssim
x265 [info]: frame I: 84, Avg QP:18.54 kb/s: 3379.67 PSNR Mean: Y:47.858 U:49.679 V:50.661 SSIM Mean: 0.988751 (19.489dB)
x265 [info]: frame P: 873, Avg QP:20.17 kb/s: 1510.20 PSNR Mean: Y:46.463 U:48.423 V:49.504 SSIM Mean: 0.984823 (18.188dB)
x265 [info]: frame B: 2119, Avg QP:22.14 kb/s: 541.89 PSNR Mean: Y:45.334 U:47.994 V:49.186 SSIM Mean: 0.981829 (17.406dB)
x265 [info]: Weighted P-Frames: Y:17.3% UV:12.6%
x265 [info]: Weighted B-Frames: Y:17.4% UV:13.3%
x265 [info]: consecutive B-frames: 14.3% 7.3% 21.0% 57.4%

encoded 3076 frames in 1054.08s (2.92 fps), 894.20 kb/s, Avg QP:21.48, Global PSNR: 46.477, SSIM Mean Y: 0.9828678 (17.662 dB)
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9

Last edited by Sagittaire; 10th May 2020 at 03:35.
Sagittaire is offline   Reply With Quote
Old 24th May 2020, 22:53   #16  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: Poland
Posts: 2,843
Quote:
Originally Posted by Sagittaire View Post
yes first encoding at 450 kbps for produce analysis.dat at 4 fps with classic encoding and after all the other encoding at 900 kbps or xxx kbps (analysis.dat reuse encoding) with 80 fps for refine 0 level.

In fact you can make first ultraplacebo encoding (0.1 fps) and make all the other encoding (with this analyse level) at ultra fast speed at all the bitrate you want. You can change all Rate Control level like you want too and I will test for AQ, deblock level, PSY (seem to be OK but perhaps that's refine is really necessary here).

I think this option will be really usefull if you want make test on particular option with really high speed encoding even for a complete movie.

and last but not least, with source in 1080p, the lossless source saturate my HDD with a little i5 3550 and I must choose SSD directory for lossless source. I can imagine that Ryzen 3700X can saturate SSD sata interface with 1080p lossless source with Reuse encoding.
For me gain in speed was from 10fps for UHD to 30fps for fast preset (just different bitrate).
Re-use at level 10. Rest default. Not sure how come you can have such a high gain (probably slow CPU +placebo preset).

Last edited by kolak; 24th May 2020 at 23:00.
kolak is offline   Reply With Quote
Old 24th May 2020, 23:03   #17  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,484
Quote:
Originally Posted by kolak View Post
For me gain in speed was from 10fps for UHD to 30fps for fast preset (just different bitrate).
Re-use at level 10. Rest default. Not sure how come you can have such a high gain.
- work very well if you use high final quality (preset slow and higher) because the speed difference is higher.

- Be carefull to the source directory. Better to use high speed directory like SSD because for me this mode saturate HDD speed only with 720*304*25 losseless source.

- and be carefull to reuse analysis file size, because for 4K UHD, the file must be enormous.
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9
Sagittaire is offline   Reply With Quote
Old 24th May 2020, 23:29   #18  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: Poland
Posts: 2,843
So far tested on shot samples. You are right- 3min = 600MB, so will be about 12GB for 1h
kolak is offline   Reply With Quote
Old 2nd October 2022, 19:50   #19  |  Link
BuccoBruce
Registered User
 
Join Date: Apr 2022
Posts: 28
Sorry to bump this old thread, but I can't find any real documentation or resources on this feature.

Say I want to do a hypothetical ABR 2-pass "ladder", with only two output bitrates, 6000k and 3000k, at the same resolution.

Do I run "--analysis-save-reuse-level 10" on my 6000k 1st pass and then --analysis-load that .dat file as well as the original .stats for my 6000k pass 2 and my 3000k pass 2?
Should I be doing --analysis-save to another file on the 6000k pass 2 as well? Or even only doing it for my higher bitrate pass 2 and then loading that for 3000k pass 2?
Do I need to run pass 1 at 3000k as well to generate a new stats file while still using --analysis-load?

EDIT: It sounds like I would be best served using --stats normally for 6000k pass 1, then run "--analysis-save-reuse-level 10" on a 6000k pass 2, and then "--analysis-load-reuse-level 10" on 3000k pass 2?

Last edited by BuccoBruce; 2nd October 2022 at 20:54.
BuccoBruce is offline   Reply With Quote
Old 3rd October 2022, 01:04   #20  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
You could also try --abr-ladder. I've not tested it myself, but it seems like the "right" way to do this.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 16:57.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.