Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
9th December 2018, 21:25 | #141 | Link |
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,733
|
Hey, while you are here.. could you consider porting the VMAF filter to Avisynth+? Currently it's Vapoursynth-only, and it could be really useful with the optimizer.
https://github.com/Netflix/vmaf
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... |
9th December 2018, 23:09 | #142 | Link | |
Registered User
Join Date: Mar 2018
Posts: 447
|
Quote:
Code:
optimizer -mode evaluate -vismode seriesheatmap -map searchRange searchRangeFinest -log "../scripts/some_script*.log" It's a bit difficult to see where the best results are so let's focus on the best 10% of results by adding -top 10 to the command: Ok, so it looks like most of the good results have searchRange=2 and searchRangeFinest is not that important. But wait, let's see another set of results which are using a different source video and also a different script (a total of 271729 results): Hmm... this one was using -top 100 but we need to focus on the best results again, using -top 20: Oh, this time the best results have searchRangeFinest 1 or 2 and searchRange is not that important. So I guess there's no fixed rule. Maybe it's possible to get good results in both ways, but the results here are a good indication the the most optimal settings sometimes need searchRange larger than searchRangeFinest and sometimes the opposite. Here's the latest version v0.9.16-beta which includes the heat map visualization. Sorry, haven't gotten around to do that yet. I did do some experiments several months ago. |
|
9th December 2018, 23:30 | #143 | Link | |
Registered User
Join Date: Mar 2018
Posts: 447
|
Quote:
Oh, and maybe you already noticed but the latest version has the heat map visualization you asked for. Since your script only has two parameters you don't need to specify the -map argument (the default is taking the fist two parameters from the script). It works with the autorefresh feature so if you want to watch the process you can call it like this: Code:
optimizer -mode evaluate -vismode heatmap -autorefresh true |
|
10th December 2018, 04:52 | #144 | Link | ||
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,733
|
Quote:
Quote:
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... |
||
10th December 2018, 09:32 | #145 | Link | |
Registered User
Join Date: Mar 2018
Posts: 447
|
Quote:
Code:
# script D:\optimizer\bin/script.avs # output out1="ssim: MAX(float)" out2="time: MIN(time) ms" file="perFrameResults.txt" 19.70072 1980 b=-7 c=-93 19.797161 1980 b=41 c=66 19.828617 1980 b=-86 c=-27 19.700043 1970 b=28 c=-56 19.80134 1970 b=-75 c=-67 ... |
|
10th December 2018, 16:44 | #146 | Link |
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,733
|
Running optimizer test2.avs -alg exhaustive -scripts best
# script C:\AvisynthOptimizer/test2.avs # output out1="ssim: MAX(float)" out2="time: MIN(time) ms" file="perFrameResults.txt" 0.987764 70 b=-80 c=10 0.987764 70 b=-79 c=10 0.98775 70 b=-78 c=10 0.987745 70 b=-77 c=10 At the end of the run, I get this information, but there is no script with that best data available. Test2.avs is like it is saved by me. Pareto front: 0.987969 70ms b=-68 c=52 0.987969 70ms b=-68 c=49 0.987969 70ms b=-72 c=45 0.98794 60ms b=-54 c=59 The heatmap is indeed very useful. I can easily see which values for b I can safely leave out and reduce the amount of combinations by a fair amount
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... |
10th December 2018, 19:03 | #147 | Link | ||
Registered User
Join Date: Mar 2018
Posts: 447
|
Quote:
Code:
optimizer -mode evaluate -scripts best Quote:
Code:
b = 33/100.0 # optimize b = _n_/100.0 | -100..100 ; filter:x 10 % 0 == | b |
||
10th December 2018, 19:14 | #148 | Link |
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,733
|
Thanks, will try that. What if I have multiple log files, like I usually do after the for loop procedure.
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... |
10th December 2018, 19:25 | #149 | Link | |
Registered User
Join Date: Mar 2018
Posts: 447
|
Quote:
Code:
optimizer -mode evaluate -log "./scripts/part_of_filename*.log" Last edited by zorr; 10th December 2018 at 19:33. Reason: Added another -log example usage. |
|
10th December 2018, 19:29 | #150 | Link |
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,733
|
Cheers, that will smooth things out a lot. It's a bit of a pain putting everything through Excel. I have to do that already too many times at work these days
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... |
12th December 2018, 18:03 | #151 | Link |
Registered User
Join Date: Sep 2010
Location: Russia
Posts: 85
|
How do I filter/limit the boolean or string values? Suppose I want to have a variable always be false when another var is below 1. Usually I'd write it as filter:Clevel 1 > NULL true ? x != so that when Clevel is above 1, x is compared to something that it is guaranteed not to be, thus resulting in an always true statement, allowing both true and false. But AvsOptim doesn't know what NULL is or whatever else I try to put instead of that.
|
12th December 2018, 20:46 | #152 | Link | |
Registered User
Join Date: Mar 2018
Posts: 447
|
Quote:
(Clevel > 1) or (x == false) and translated to reverse polish notation Clevel 1 > x false == or Note that in your original filter and this one false is the only allowed value when Clevel is below 2. If you want it false when Clevel is below 1 then use Clevel > 0 or Clevel >= 1. |
|
23rd December 2018, 15:27 | #153 | Link |
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,733
|
I came up with an idea to reduce the amount of needed iterations, works at least for the resizer test that I run in exhaustive mode.
My results seem to be a bell-shaped curve, so it would be safe to skip the rest of the iterations of b against constant c. Thus, when the SSIM result of a pair is lower than the previous result, start the next round of varying b against c.
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... |
25th December 2018, 22:34 | #154 | Link | |
Registered User
Join Date: Mar 2018
Posts: 447
|
Quote:
-alg mutation -pop 1 -runs 1 -mutcount 1 -mutamount 0.01 The search will start at some random location and gradually move towards the optimum. The heat map looks like this: The heat map will show a "cross" when the optimum has been reached (the optimum being located at the center of the cross). |
|
26th December 2018, 17:48 | #155 | Link | |
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,733
|
Quote:
Code:
Run 1 best: 4.952373 570 b=-54 c=52 Run 2 best: 4.934916 600 b=-56 c=58 Run 3 best: 4.9637823 590 b=-46 c=34 Run 4 best: 4.97074 600 b=-41 c=60 Run 5 best: 4.8731537 570 b=-67 c=57 Run 6 best: 4.952373 570 b=-54 c=52 Run 7 best: 4.934916 600 b=-56 c=58 Run 8 best: 4.9637823 600 b=-47 c=34 Run 9 best: 4.97074 580 b=-41 c=60 Run 10 best: 4.8731537 590 b=-67 c=57
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... Last edited by Boulder; 26th December 2018 at 17:50. |
|
28th December 2018, 22:35 | #156 | Link | |
Registered User
Join Date: Mar 2018
Posts: 447
|
Quote:
I think this is where the dynamic iteration count could be useful because it can stop the optimizing process when no better results are found. I ran some tests and found settings where the optimum was found in 10 out of 10 runs (using b and c range -100..100). Try these: -alg mutation -iters dyn -dyniters 12 -dynphases 2 -pop 1 -runs 1 -mutcount 1 -mutamount 0.1 0.01 The mutation amount is large (0.1) in the beginning and small (0.01) in the end, and looks like this: It takes on average about 100 iterations per run. That's 28 times faster than the exhaustive algorithm, not bad! |
|
29th December 2018, 14:52 | #157 | Link |
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,733
|
Thanks, I'll definitely test that with the next batch of encodes I have. A hundred iterations sounds tremendous, as I can then use more frames for analysis to make sure I get a good allround result.
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... |
31st December 2018, 23:35 | #158 | Link |
Registered User
Join Date: Nov 2009
Posts: 2,361
|
Is it possible to resume a test that finished too early and take logs into consideration to refine future mutations?
I'm using the last dynamic mode posted above but it finished too early. I might have understood it wrong but in my tests the best result is always the one with least change, naturally we want a change since we are denoising. The following always ends with a smdegrain(tr=2,thSAD=100,contrasharp=true,blksize=8,overlap=4,divide=0,refinemotion=true,lsb=true) Code:
sigma = 20*20 # optimize sigma = _n_*20 | 100..600 | sigma blockSize = 8 # optimize blockSize = _n_ | 4,8,16,32 ; min:divide 0 > 8 2 ? ; filter:overlap 2 * x <= | blockSize overlap = 4 # optimize overlap = _n_ | 4,6,8,10,12,14,16 ; max:blockSize 2 / ; filter:x divide 0 > 8 2 ? % 0 == | overlap tr = 3 # optimize tr = _n_ | 2..4 | tr divide = 0 # optimize divide = _n_ | 0..2 ; max:blockSize 8 >= 2 0 ? overlap 4 % 0 == 2 0 ? min | divide denoised=smdegrain(tr=tr,thSAD=sigma,contrasharp=true,prefilter=2,blksize=blockSize,divide=divide,overlap=overlap,refinemotion=true,lsb=true) Last edited by Dogway; 1st January 2019 at 00:26. |
1st January 2019, 15:26 | #159 | Link | |||
Registered User
Join Date: Mar 2018
Posts: 447
|
Quote:
Quote:
Quote:
Can you show what kind of values you are returning from the script? |
|||
1st January 2019, 17:28 | #160 | Link |
Registered User
Join Date: Dec 2005
Location: Germany
Posts: 1,795
|
I would be nice if we would have also halo and "other crap" simulators to easily test (optimize) other kinds of filters.
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth VapourSynth Portable FATPACK || VapourSynth Database |
|
|