Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
1st May 2020, 13:28 | #1 | Link |
Testeur de codecs
Join Date: May 2003
Location: France
Posts: 2,484
|
Test with analysis-reuse in x265?
Someone have made test with these x265 functionaly?
Code:
--analysis-save <filename> Dump analysis info into the specified file. Default Disabled --analysis-load <filename> Load analysis buffers from the file specified. Default Disabled --analysis-reuse-file <filename> Specify file name used for either dumping or reading analysis data. Deault x265_analysis.dat --analysis-reuse-level <1..10> Level of analysis reuse indicates amount of info stored/reused in save/load mode, 1:least..10:most. Now deprecated. Default 0 --analysis-save-reuse-level <1..10> Indicates the amount of analysis info stored in save mode, 1:least..10:most. Default 0 --analysis-load-reuse-level <1..10> Indicates the amount of analysis info reused in load mode, 1:least..10:most. Default 0 --refine-analysis-type <string> Reuse anlaysis information received through API call. Supported options are avc and hevc. Default disabled - 0 --scale-factor <int> Specify factor by which input video is scaled down for analysis save mode. Default 0 --refine-intra <0..4> Enable intra refinement for encode that uses analysis-load. - 0 : Forces both mode and depth from the save encode. - 1 : Functionality of (0) + evaluate all intra modes at min-cu-size's depth when current depth is one smaller than min-cu-size's depth. - 2 : Functionality of (1) + irrespective of size evaluate all angular modes when the save encode decides the best mode as angular. - 3 : Functionality of (1) + irrespective of size evaluate all intra modes. - 4 : Re-evaluate all intra blocks, does not reuse data from save encode. Default:0 --refine-inter <0..3> Enable inter refinement for encode that uses analysis-load. - 0 : Forces both mode and depth from the save encode. - 1 : Functionality of (0) + evaluate all inter modes at min-cu-size's depth when current depth is one smaller than min-cu-size's depth. When save encode decides the current block as skip(for all sizes) evaluate skip/merge. - 2 : Functionality of (1) + irrespective of size restrict the modes evaluated when specific modes are decided as the best mode by the save encode. - 3 : Functionality of (1) + irrespective of size evaluate all inter modes. Default:0 --[no-]dynamic-refine Dynamically changes refine-inter level for each CU. Default disabled --refine-mv <1..3> Enable mv refinement for load mode. Default 1 --refine-ctu-distortion Store/normalize ctu distortion in analysis-save/load. - 0 : Disabled. - 1 : Store/Load ctu distortion to/from the file specified in analysis-save/load. Default 0 - Disabled Anyway I don't understand how use refine like --refine-intra, --refine-inter or --refine-mv? It's possible to change psy, deblock, sao seeting for new high speed "placebo level" encoding?
__________________
Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9 Last edited by Sagittaire; 1st May 2020 at 14:03. |
1st May 2020, 21:37 | #2 | Link |
Derek Prestegard IRL
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,989
|
I have not personally messed with this, but my understanding is that this is extremely useful for ABR encoding. You save the analysis from a lower res encode and re-use it (with refinement) on a high res encode.
So for example if you have a 4k source and want to make 1080p and 4k variants, you do the 1080p encode, save the analysis data, then re-use it when doing the 4k encode. This saves a ton of largely redundant analysis on the 4k layer. There is a quality impact, but it's not much, and the compute you save can be sent on higher quality analysis There was a great talk at Demuxed 2018 about this https://www.youtube.com/watch?v=-yPOE2F9fDI from Alex Giladi from Comcast |
2nd May 2020, 00:17 | #3 | Link | |
Testeur de codecs
Join Date: May 2003
Location: France
Posts: 2,484
|
Quote:
I think it's pro profil too. For streaming like Netflix or Amazon, you make only one high quality encoding and you can remake all reencoding at the bitrate you want with really high speed and high quality. It's certainely like that Netflix can adapt quality for all user.
__________________
Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9 |
|
2nd May 2020, 01:11 | #5 | Link | |
Testeur de codecs
Join Date: May 2003
Location: France
Posts: 2,484
|
Quote:
https://damienschroeder.wordpress.co...ttp-streaming/ https://x265.com/uhdkit/
__________________
Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9 Last edited by Sagittaire; 2nd May 2020 at 01:15. |
|
3rd May 2020, 12:39 | #6 | Link | |
Testeur de codecs
Join Date: May 2003
Location: France
Posts: 2,484
|
Here little demonstration:
Quote:
There is little quality loss between the complete pass and analysis-load-reuse pass, but the speed is increase by x100 for placebo encoding level. In fact here my command is limited by HDD, but if I choose SSD directory for lossless y4m source, I have more than 30 fps. My little i5-3550 with placebo profil have higher speed than AMD 3990X. And you can choose other bitrate at the same speed for second pass. I make test to see if I can change psy setting, blocking setting in high speed second pass. It's really powerfull fonctionality ... ;-)
__________________
Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9 Last edited by Sagittaire; 3rd May 2020 at 14:50. |
|
9th May 2020, 22:18 | #7 | Link |
Testeur de codecs
Join Date: May 2003
Location: France
Posts: 2,484
|
For those who are interested in this functionality:
1) soon a demonstration in the benwagonner challenge 2) You must use --refine-intra --refine-inter with --analysis-load and --analysis-load-reuse-level only if you change bitrate for reuse encoding for example and for make intra and inter refinement at this new bitrate and obtain better quality. Anyway with these option, you have strong speed penality (and high speed boost is the main interest of reuse reencoding). More refine level imply more quality but less speed. --refine-intra 4 --refine-inter 3 is the highest refinement level. 3) You can't use --refine-intra --refine-inter option with --rd-refine option (bug?) 4) --refine-mv is only usefull if you change resolution (not tested)
__________________
Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9 |
9th May 2020, 22:37 | #8 | Link | |
Moderator
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,770
|
Quote:
At global scale with a global content library, even a 20% reduction in encode compute can save millions a year. As can a 5% reduction in average bitrate gain from more efficient use of compute. Reducing file size saves on storage and bandwidth to the viewer, and improves edge cache effeciency. |
|
9th May 2020, 22:39 | #9 | Link | |
Moderator
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,770
|
Quote:
|
|
9th May 2020, 22:41 | #10 | Link | |
Testeur de codecs
Join Date: May 2003
Location: France
Posts: 2,484
|
Quote:
__________________
Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9 |
|
9th May 2020, 23:52 | #11 | Link | |||||||
Testeur de codecs
Join Date: May 2003
Location: France
Posts: 2,484
|
Little test:
1) first classic way encoding, at slower profil, in 3 passes. You can see that third pass produce better quality than second pass (exactly the same setting for second and third pass) first pass produce really high metric because first pass is CBR encoding, you have really bad quality in high motion and high quality for low motion (yes it's the best RC ... for metric ... in most case) Quote:
2) Second reuse encoding at 900 kbps with previous analyse reuse from 450 kbps encoding - And 75.45 fps at 900 kbps versus 4.17 fps at 450 kbps with the same search level (slower) with --refine-intra 0 --refine-inter 0: 45.96 dB - And 24.41 fps at 900 kbps versus 4.17 fps at 450 kbps with the same search level (slower) with --refine-intra 1 --refine-inter 1: 46.17 dB - And 12.08 fps at 900 kbps versus 4.17 fps at 450 kbps with the same search level (slower) with --refine-intra 2 --refine-inter 2: 46.34 dB - And 08.22 fps at 900 kbps versus 4.17 fps at 450 kbps with the same search level (slower) with --refine-intra 3 --refine-inter 3: 46.39 dB - And 08.01 fps at 900 kbps versus 4.17 fps at 450 kbps with the same search level (slower) with --refine-intra 4 --refine-inter 3: 46.40 dB - And 12.12 fps at 900 kbps versus 4.17 fps at 450 kbps with the same search level (slower) with --refine-intra 4 --dynamic-refine: 46.25 dB Quote:
Quote:
Quote:
Quote:
Quote:
3) Comparison control with classic 3 passes encoding at 900 kbps with exactly same profil than classic 3 passes encoding at 450 kbps Result is really impressive for reuse encoding at 900 kbps (from 450 encoding): really equivalent metric performance at really higher speed. Quote:
__________________
Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9 Last edited by Sagittaire; 10th May 2020 at 03:11. |
|||||||
10th May 2020, 00:15 | #13 | Link |
Testeur de codecs
Join Date: May 2003
Location: France
Posts: 2,484
|
PbBbP is really efficient sequence. I have 1% save size between 3 bframes and 5 bframes in your ToS challenge. Higher bframe number is placebo for metric. But I have little subjective problem (temporal stability) in your ToS sample with 7 bframes (and higher).
__________________
Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9 Last edited by Sagittaire; 10th May 2020 at 00:27. |
10th May 2020, 00:25 | #14 | Link |
Testeur de codecs
Join Date: May 2003
Location: France
Posts: 2,484
|
yes first encoding at 450 kbps for produce analysis.dat at 4 fps with classic encoding and after all the other encoding at 900 kbps or xxx kbps (analysis.dat reuse encoding) with 80 fps for refine 0 level.
In fact you can make first ultraplacebo encoding (0.1 fps) and make all the other encoding (with this analyse level) at ultra fast speed at all the bitrate you want. You can change all Rate Control level like you want too and I will test for AQ, deblock level, PSY (seem to be OK but perhaps that's refine is really necessary here). I think this option will be really usefull if you want make test on particular option with really high speed encoding even for a complete movie. and last but not least, with source in 1080p, the lossless source saturate my HDD with a little i5 3550 and I must choose SSD directory for lossless source. I can imagine that Ryzen 3700X can saturate SSD sata interface with 1080p lossless source with Reuse encoding.
__________________
Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9 Last edited by Sagittaire; 10th May 2020 at 01:12. |
10th May 2020, 03:12 | #15 | Link | |
Testeur de codecs
Join Date: May 2003
Location: France
Posts: 2,484
|
Update: encoding control
3) Comparison control with classic 3 passes encoding at 900 kbps with exactly same profil than classic 3 passes encoding at 450 kbps Result is really impressive for reuse encoding at 900 kbps (from 450 encoding): really equivalent metric performance at really higher speed. I will perhaps simply reuse 1000 kbps for make 1500 and 2000 kbps encoding for benwagonner challenge... ;-) Quote:
__________________
Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9 Last edited by Sagittaire; 10th May 2020 at 03:35. |
|
24th May 2020, 22:53 | #16 | Link | |
Registered User
Join Date: Nov 2004
Location: Poland
Posts: 2,843
|
Quote:
Re-use at level 10. Rest default. Not sure how come you can have such a high gain (probably slow CPU +placebo preset). Last edited by kolak; 24th May 2020 at 23:00. |
|
24th May 2020, 23:03 | #17 | Link | |
Testeur de codecs
Join Date: May 2003
Location: France
Posts: 2,484
|
Quote:
- Be carefull to the source directory. Better to use high speed directory like SSD because for me this mode saturate HDD speed only with 720*304*25 losseless source. - and be carefull to reuse analysis file size, because for 4K UHD, the file must be enormous.
__________________
Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9 |
|
2nd October 2022, 19:50 | #19 | Link |
Registered User
Join Date: Apr 2022
Posts: 28
|
Sorry to bump this old thread, but I can't find any real documentation or resources on this feature.
Say I want to do a hypothetical ABR 2-pass "ladder", with only two output bitrates, 6000k and 3000k, at the same resolution. Do I run "--analysis-save-reuse-level 10" on my 6000k 1st pass and then --analysis-load that .dat file as well as the original .stats for my 6000k pass 2 and my 3000k pass 2? Should I be doing --analysis-save to another file on the 6000k pass 2 as well? Or even only doing it for my higher bitrate pass 2 and then loading that for 3000k pass 2? Do I need to run pass 1 at 3000k as well to generate a new stats file while still using --analysis-load? EDIT: It sounds like I would be best served using --stats normally for 6000k pass 1, then run "--analysis-save-reuse-level 10" on a 6000k pass 2, and then "--analysis-load-reuse-level 10" on 3000k pass 2? Last edited by BuccoBruce; 2nd October 2022 at 20:54. |
3rd October 2022, 01:04 | #20 | Link |
Moderator
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,770
|
You could also try --abr-ladder. I've not tested it myself, but it seems like the "right" way to do this.
|
Thread Tools | Search this Thread |
Display Modes | |
|
|