Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 25th March 2021, 00:25   #141  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 447
Quote:
Originally Posted by WorBry View Post
How would one print a log file that compiles the per-frame test results in this case?
Looks like there is no way to read the similarity metric back into a script variable which is how the current SSIM metric is handled. Perhaps this could be implemented by passing a (global?) variable name and GMSD / MDSI functions would assign that variable?
zorr is offline   Reply With Quote
Old 25th March 2021, 13:15   #142  |  Link
WolframRhodium
Registered User
 
Join Date: Jan 2016
Posts: 162
Quote:
Originally Posted by zorr View Post
Looks like there is no way to read the similarity metric back into a script variable which is how the current SSIM metric is handled. Perhaps this could be implemented by passing a (global?) variable name and GMSD / MDSI functions would assign that variable?
It is possible to write the data to a file via FrameEval or get_frame.

For example,
Code:
def output_frameprop(
    clip: vs.VideoNode, filename: str, prop_name: str
) -> vs.VideoNode:

    records = []
    frames = clip.num_frames

    def output(n, f, clip=clip):
        records.append((n, f.props[prop_name]))

        nonlocal frames
        frames -= 1

        if frames == 0:
            with open(filename, "w") as f:
                f.write(f"frameIdx {prop_name}\n")
                f.writelines(
                    f"{idx:8} {value:{len(prop_name)}}\n" 
                    for idx, value in sorted(records))

        return clip

    return core.std.FrameEval(clip, output, clip)


clip = calculate_some_metric(clip)
clip = output_frameprop(clip, "stats.txt", metric_name)
And evaluate the script from beginning to end via vspipe / benchmark utility from vsedit. The result is then in file "stats.txt".
WolframRhodium is offline   Reply With Quote
Old 25th March 2021, 16:12   #143  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
@WolframRhodium I think he means the avisynth version. For VS we already have this scrpt here https://github.com/theChaosCoder/zop...er/zoptilib.py
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline   Reply With Quote
Old 25th March 2021, 16:33   #144  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,197
Yes, I mean the ported AVISynth version. It appears that Asd-g is not a forum member.
__________________
Nostalgia's not what it used to be
WorBry is offline   Reply With Quote
Old 25th March 2021, 16:41   #145  |  Link
WolframRhodium
Registered User
 
Join Date: Jan 2016
Posts: 162
Sorry about that, I did not read the thread carefully. I thint it is possible through WriteFile, but I am not very familiar with it.
WolframRhodium is offline   Reply With Quote
Old 25th March 2021, 21:09   #146  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
Quote:
Originally Posted by zorr View Post
Looks like there is no way to read the similarity metric back into a script variable which is how the current SSIM metric is handled. Perhaps this could be implemented by passing a (global?) variable name and GMSD / MDSI functions would assign that variable?
I think it should work like in Vapoursynth now.

https://github.com/Asd-g/AviSynthPlu...MDSI.avsi#L103

Quote:
"""propSet("_FrameMDSI",
Avisynth+ 3.X supports clip props just like vapoursynth.

They are read for show=true at the script bottom
Code:
ScriptClip("""Subtitle("FrameMDSI: " + String(propGetFloat("_FrameMDSI")))"""
So maybe zopti needs a "zoptilib.py" version for avisynth now?
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline   Reply With Quote
Old 28th March 2021, 01:17   #147  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 447
Thanks Chaos, I didn't notice that the metric can indeed be read from the frame properties. So here's an example script using GMSD:

Code:
TEST_FRAMES = 5			# how many frames are tested
MIDDLE_FRAME = 50		# middle frame number

RawSourcePlus("D:/optimizer/test/flower/flower_cif.yuv", width=352, height=288, pixel_type="I420")
source=ColorYUV(levels="PC->TV")
noisy=source.AddGrain(25, 0, 0, seed=1)
#return noisy

#denoised=noisy.FFT3DFilter(sigma=4, bt=4, bw=16, bh=16, ow=8, oh=8)		# best settings by Fizick

sigma = 400/100.0 	# optimize sigma = _n_/100.0 | 100..800 | sigma
bt = 4				# optimize bt = _n_ | -1..5 | blockTemporal
blockSize = 32		# optimize blockSize = _n_ | 2..64 ; min:overlap 2 * | blockSize
overlap = 16		# optimize overlap = _n_ | 0..32 ; max:blockSize 2 / | overlap
denoised=noisy.FFT3DFilter(sigma=sigma, bt=bt, bw=blockSize, bh=blockSize, ow=overlap, oh=overlap)

# cut out the part used in quality / speed evaluation
source = source.Trim(MIDDLE_FRAME - TEST_FRAMES/2 + (TEST_FRAMES%2==0?1:0), MIDDLE_FRAME + TEST_FRAMES/2)
denoised = denoised.Trim(MIDDLE_FRAME - TEST_FRAMES/2 + (TEST_FRAMES%2==0?1:0), MIDDLE_FRAME + TEST_FRAMES/2)
last = denoised

last = GMSD(source, denoised)

global total = 0.0
global gmsd_total = 0.0
FrameEvaluate(last, """
	global gmsd = propGetFloat("_PlaneGMSD")
	global gmsd = (gmsd == 0.0 ? 1.0 : gmsd)
	global gmsd_total = gmsd_total + gmsd	
""")	

# measure runtime, plugin writes the value to global avstimer variable
global avstimer = 0.0
AvsTimer(frames=1, type=0, total=false, name="Optimizer")

# per frame logging (gmsd, time)
delimiter = "; "
resultFile = "perFrameResults.txt"	# output out1="gmsd: MIN(float)" out2="time: MIN(time) ms" file="perFrameResults.txt"
WriteFile(resultFile, "current_frame", "delimiter", "gmsd", "delimiter", "avstimer")

# write "stop" at the last frame to tell the optimizer that the script has finished
frame_count = FrameCount()
WriteFileIf(resultFile, "current_frame == frame_count-1", """ "stop " """, "gmsd_total", append=true)

return last
This is just a slightly modified version of the script I used in the hands-on tutorial. Other than renaming the variables the only relevant changes are highlighted in red.

In this script GMSD gave indentical optimal values compared to SSIM except for the sigma which is 415 instead of 479. I didn't try MDSI yet.
zorr is offline   Reply With Quote
Old 28th March 2021, 12:47   #148  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
Is there a way to utilize Avisynth+'s Prefetch to speed up calculation? I tested it briefly but there were problems probably related to writing to the file asynchronously.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 28th March 2021, 17:11   #149  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 447
Quote:
Originally Posted by Boulder View Post
Is there a way to utilize Avisynth+'s Prefetch to speed up calculation? I tested it briefly but there were problems probably related to writing to the file asynchronously.
I haven't tried it myself but it's worth an investigation. I think there's going to be problems if multiple threads are writing to the same file.

Is it possible to force WriteFile to execute on one thread while the filters before it are multithreaded? The documentation states that you cannot set MT mode on script function calls, but would placing a MT_SERIALIZED plugin call just before WriteFile have that side effect? Or can you just say SetFilterMTMode("WriteFile", MT_SERIALIZED)?

On the other hand, perhaps the WriteFile is already MT_SERIALIZED but the threads are writing into it in arbitrary order and the last line is not always last as it should be. In that case the per frame results should be written into a temporary thread-safe variable which would be written to file in correct order when all the frames are finished. It works like that in Vapoursynth using a map, but Avisynth doesn't have such a data structure so I'm not sure how to implement that. Perhaps a simple array would suffice as long as it's thread safe.
zorr is offline   Reply With Quote
Old 28th March 2021, 18:04   #150  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
I'm not sure if WriteFile is internally marked as MT_SERIALIZED, but in general, it should already have a "correct" mode set. It is very much likely that the order of frames is very random since they finish at different times.

Setting both Prefetch(threads=1, frames=24) and Prefetch(threads=24, frames=1) give this error: java.lang.NumberFormatException: For input string: "1I don't know what 'delimiter' means.0.179144I don't know what 'delimiter' means.3150.284912" and the log file is full of these errors.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 29th March 2021, 02:55   #151  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 447
Quote:
Originally Posted by Boulder View Post
Setting both Prefetch(threads=1, frames=24) and Prefetch(threads=24, frames=1) give this error: java.lang.NumberFormatException: For input string: "1I don't know what 'delimiter' means.0.179144I don't know what 'delimiter' means.3150.284912" and the log file is full of these errors.
I got the same result, but I was able to make it work by defining both delimiter and frame_count as global. I'm still not sure if the result file will always be correct but it seems to work. However I didn't seem to gain any performance with the prefetch. Perhaps the serialized file writing escalates upstream and makes everything run serially.
zorr is offline   Reply With Quote
Old 29th March 2021, 05:07   #152  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
Quote:
Originally Posted by zorr View Post
I got the same result, but I was able to make it work by defining both delimiter and frame_count as global. I'm still not sure if the result file will always be correct but it seems to work. However I didn't seem to gain any performance with the prefetch. Perhaps the serialized file writing escalates upstream and makes everything run serially.
Yes, I think that's how it goes. Perhaps the only way to make it faster and threadsafe in Avisynth would be to use an array as you suggested. Or is it possible to make all the frames create their own log file and combine them when it's time to calculate the total score?

The VS version is also not perfect in multithreading sense but it is somewhat faster.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 16th April 2021, 00:02   #153  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 447
Zopti v1.1.0 released

A new major release is here! Zopti now supports executing multiple scripts simultaneously which can lead to very nice performance improvement. This can be very useful when the script itself is not able to utilize all of your cores.

Here's the full release info:
  • added support for multithreaded execution of scripts with argument -threads
    -default value of -threads is 1 meaning one script is executed at a time just like before
    -all optimization algorithms (nsga-ii, spea2, mutation, exhaustive) support the the -threads option
    -due to the nature of the heuristic algorithms (nsga-ii, spea2, mutation) there can be at most P threads running the scripts at the same time, where P is the population size
    -exhaustive algorithm has no thread limits
    -validate mode also supports -threads. there the maximum number of threads is the number of validated results (size of the pareto front)
    -note: using more than one thread will make time measurements less accurate but will not change the quality measurements
  • avsr version update (v0.2.2)
  • added jMetal.log.ini to disable multithreading related JMetal logging
  • updated Zoptilib to version 1.0.9m
  • bugfix: evaluate could not parse log files when Zopti was used

Download link is at the first post. I will shortly post some data on the multithreading performance.
zorr is offline   Reply With Quote
Old 16th April 2021, 00:49   #154  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 447
I tested the new -threads argument using the denoising tutorial script, this time using GMSD as the similarity metric and using 50 frames instead of 5. Here's the script:

Code:
import vapoursynth as vs
from zoptilib import Zopti
core = vs.core

TEST_FRAMES = 50		# how many frames are tested
MIDDLE_FRAME = 100		# middle frame number
	
# read input video
video = core.raws.Source(r'D:\optimizer\test\flower\flower_cif.yuv', 352, 288, src_fmt='I420')

zopti = Zopti(r'results.txt', metrics=['gmsd', 'time'])

# input color range is PC (full)
video = video.std.SetFrameProp(prop="_ColorRange", intval=0)

# add noise
noisy = video.grain.Add(var=25, seed=3)
#noisy.set_output()

# remove noise
sigma = 400/100 	# optimize sigma = _n_/100 | 400..600 ; filter:x 5 % 0 == | sigma
bt = 5			# optimize bt = _n_ | 3,5 | blockTemporal
blockSize = 14		# optimize blockSize = _n_ | 6,12 ; min:overlap 2 * | blockSize
overlap = 16		# optimize overlap = _n_ | 4..6 ; max:blockSize 2 / | overlap
denoised = noisy.fft3dfilter.FFT3DFilter(sigma=sigma, bt=bt, bw=blockSize, bh=blockSize, ow=overlap, oh=overlap)

# cut out the part used in quality / speed evaluation
video = video[MIDDLE_FRAME - TEST_FRAMES//2 + (1 if (TEST_FRAMES%2==0) else 0) : MIDDLE_FRAME + TEST_FRAMES//2 + 1]
denoised = denoised[MIDDLE_FRAME - TEST_FRAMES//2 + (1 if (TEST_FRAMES%2==0) else 0) : MIDDLE_FRAME + TEST_FRAMES//2 + 1]

zopti.run(video, denoised)
There's also an AviSynth version of the script which I posted to the Avisynth Zopti thread.

The script only has 246 valid parameter combinations to test so we can try all of them using
Code:
zopti denoise_ex.vpy -alg exhaustive -threads 1
It takes 140 seconds to test all of the combinations using one thread. Let's see the performance when using more threads:



The best result is about 20 seconds using all 24 threads (on a Ryzen 3900X which has 12 cores / 24 threads).

Another way to look at the scaling is to calculate how much faster we get the results when using -threads:



Using 24 threads is 7,1 times as fast as using only one thread. Of course this is just one data point and I don't mean to imply that you can always get such a performance improvement. I would have liked to include more tests using more real-world usage scenarios (HD source etc) but my stock cooled processor cannot handle those for more than a few seconds, it becomes so hot that my PC shuts down.

Comparing AviSynth and VapourSynth it looks like VS has the edge when using one or just a few threads, but the differences almost vanish when using 12 or more threads. The VS is also able to utilize more than 16 cores while with AVS the performance starts to degrade. The point where more threads are just slowing down is probably dependent on the specific script, I will have to run more tests on that.
zorr is offline   Reply With Quote
Old 17th April 2021, 08:37   #155  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
With the new multithreading support, is it beneficial to increase the population size even with this kind of approach to optimizing the resizer params? A word of warning: VS uses a lot of memory for caching by default so it may be wise to lower the max_cache_size if you launch multiple instances

zopti "test.vpy" -alg mutation -iters dyn -dyniters 18 -dynphases 2 -pop 1 -runs 1 -mutcount 1 -mutamount 0.1 0.01 -initial script
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 17th April 2021, 14:13   #156  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 447
Quote:
Originally Posted by Boulder View Post
With the new multithreading support, is it beneficial to increase the population size even with this kind of approach to optimizing the resizer params?

zopti "test.vpy" -alg mutation -iters dyn -dyniters 18 -dynphases 2 -pop 1 -runs 1 -mutcount 1 -mutamount 0.1 0.01 -initial script
In this case you have to increase the population size from 1, otherwise there's no use for more than one thread. It will for sure make the search faster but the quality (or the probability that you will find the optimal values) might go down a little bit. You can compensate for that with increasing -dyniters slightly. The net result should be as good (or better) results faster. You may have to do some testing to find how many threads give you the best bang for buck though.

Quote:
Originally Posted by Boulder View Post
A word of warning: VS uses a lot of memory for caching by default so it may be wise to lower the max_cache_size if you launch multiple instances
Yes that's one downside of using -threads. It's process level multithreading and every VSPipe process will use as much memory as a single one would. So the amount of free memory might limit how many threads you can actually use even though CPU is still not maxed out.
zorr is offline   Reply With Quote
Old 28th May 2021, 21:48   #157  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 447
Zopti v1.2.0 released

Time for a new update! This version has some major additions so I bumped the version to 1.2.0.
  • new option: -timeout (default is disabled). if the execution of Avisynth script takes longer than timeout seconds
    -the script execution process (and subprocesses) will be terminated
    -the result is set to worst possible as a penalty so it will not be considered as a valid candidate
    -useful with scripts where certain parameter combinations can result in execution times several hundred times slower than usual
    -NOTE: only implemented for AviSynth scripts (for now)
    -example: zopti script.avs -pop 24 -iters 1000 -timeout 60
  • new option: -continue
    -aborted optimization can be continued from the last complete generation (one generation is N iterations, where N is the population size)
    -give the log file name of the aborted optimization as the value of continue
    -old results up to last complete generation are copied to a new log file and optimization will continue from there
    -useful if optimization was unexpectedly aborted or the CPU resources are needed for something else for a while
    -example: zopti script.avs -pop 24 -iters 10000 -continue "mvtools spea2 run-01.log"
    -NOTE: it is possible to change the optimization parameters (population, mutation count etc.) from those used in the continued log
    -NOTE: random seed is not restored so the results will not be identical to what they would have been if execution had not been aborted
    -NOTE: only works with algorithm spea2 (for now)
  • new visualization mode: history
    -displays the best found value and all tried values of certain optimized parameter (given with -param)
    -invalid results (script execution has failed OR timed out) are highlighted in orange
    -also shows the phase of dynamic iteration (if applicable)
    -accepts parameter -range to limit the displayed history to certain iteration range
    -example: zopti -mode evaluate -vismode history -param lambda (displays all history of parameter "lambda")
    -example: zopti -mode evaluate -vismode history -param lambda -range 1000 5000 (displays history from iteration 1000 to iteration 5000)
    -example: zopti -mode evaluate -vismode history -param lambda -range -10000 (displays history of the last 10000 iterations)
  • argument -dynphases can now be zero (previously minimum was 1)
  • pareto front and total runtime is displayed also after mutation or exhaustive algorithm is finished
  • if population is based on a log file and the pareto front is smaller than the population size, the population will be filled with mutations of the pareto front instead of completely random combinations
  • if a log file for the optimization already exists it will be cleared in the beginning of the optimization
  • XChart updated to version 3.8.0 with some customization

And here's what -vismode history looks like:



The black thick line shows the best found value of the chosen parameter. The red diamonds indicate a point where a new best result has been found and this parameter's value was changed. The white diamonds indicate a new best result but this parameter's value remained the same.

In the example optimization was finished and then restarted with different settings at around 190 000 iterations.

The -timeout was needed when I started running MVTools2 tests with all the parameters and it turned out that some combinations took an enormous amount of time to finish (typical time was 5 seconds but some combinations took over 30 minutes...). Using a timeout of 60 seconds solved that problem.

-continue is useful if you want to keep optimizing a result that is finished, you can try different settings. Also very handy when the optimization has been aborted by hardware failure or something like Windows update... you can just restart it using the same settings. This also makes it possible to run huge iteration counts little by little even if you need/want to use the computer for other things once in a while.

Download link updated at first post.
zorr is offline   Reply With Quote
Old 29th May 2021, 07:57   #158  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
Thank you for the new version It's always nice to see development even though the old product is running well.

One request though: would it be possible to remove the need for keeping the script that was used when running the optimization? I sometimes do many seasons of some TV series sequentially, and reuse the scripts since the properties are usually almost the same between the episodes from various seasons. Zopti doesn't like it when it cannot find the original script as I've already renamed it. I think writing the best result on the first line (to be able to see it right after opening the files in Notepad++ ) of the log file would do just fine.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 29th May 2021, 20:39   #159  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 447
Quote:
Originally Posted by Boulder View Post
Thank you for the new version It's always nice to see development even though the old product is running well.
You're welcome.

Quote:
Originally Posted by Boulder View Post
One request though: would it be possible to remove the need for keeping the script that was used when running the optimization?
What it's looking for in the script are the parameters and their valid values. Those are needed (for example) in deciding how to map the values to x,y coordinates in the heat maps.

Quote:
Originally Posted by Boulder View Post
Zopti doesn't like it when it cannot find the original script as I've already renamed it. I think writing the best result on the first line (to be able to see it right after opening the files in Notepad++ ) of the log file would do just fine.
If the script still has the same parameters (and value ranges) it doesn't matter which script is loaded. You could change the first line of the log file (starts with # script) to point to some other existing script.

On the other hand if you just run -vismode none then the script should not be needed and the best results could be displayed. I'll have to get back to you on that.
zorr is offline   Reply With Quote
Old 1st June 2021, 22:49   #160  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
Info: The latest VMAF release added PSNR-HVS and CIEDE2000 metrics.
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 06:02.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.