Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 21st November 2020, 10:05   #141  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 5,061
Yes, looks similar:
Code:
# output out1="MDSI: MIN(float)" file="results.txt"
30.7859 b=-60 c=30 
31.428144 b=-47 c=12 
30.756887 b=-63 c=30  # phase=0,00
30.797626 b=-59 c=30
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 24th March 2021, 19:38   #142  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,197
Quote:
Originally Posted by ChaosKing View Post
Btw Asd-g ported some metrics like MDSI, GMSD or vsSSIM to avs https://github.com/Asd-g/AviSynthPlus-Scripts
Didn't see this before. Excellent. Love the color-enhanced maps. How would one print a log file that compiles the per-frame test results in this case?
__________________
Nostalgia's not what it used to be

Last edited by WorBry; 24th March 2021 at 20:25.
WorBry is offline   Reply With Quote
Old 25th March 2021, 00:25   #143  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 265
Quote:
Originally Posted by WorBry View Post
How would one print a log file that compiles the per-frame test results in this case?
Looks like there is no way to read the similarity metric back into a script variable which is how the current SSIM metric is handled. Perhaps this could be implemented by passing a (global?) variable name and GMSD / MDSI functions would assign that variable?
zorr is offline   Reply With Quote
Old 25th March 2021, 13:15   #144  |  Link
WolframRhodium
Registered User
 
Join Date: Jan 2016
Posts: 115
Quote:
Originally Posted by zorr View Post
Looks like there is no way to read the similarity metric back into a script variable which is how the current SSIM metric is handled. Perhaps this could be implemented by passing a (global?) variable name and GMSD / MDSI functions would assign that variable?
It is possible to write the data to a file via FrameEval or get_frame.

For example,
Code:
def output_frameprop(
    clip: vs.VideoNode, filename: str, prop_name: str
) -> vs.VideoNode:

    records = []
    frames = clip.num_frames

    def output(n, f, clip=clip):
        records.append((n, f.props[prop_name]))

        nonlocal frames
        frames -= 1

        if frames == 0:
            with open(filename, "w") as f:
                f.write(f"frameIdx {prop_name}\n")
                f.writelines(
                    f"{idx:8} {value:{len(prop_name)}}\n" 
                    for idx, value in sorted(records))

        return clip

    return core.std.FrameEval(clip, output, clip)


clip = calculate_some_metric(clip)
clip = output_frameprop(clip, "stats.txt", metric_name)
And evaluate the script from beginning to end via vspipe / benchmark utility from vsedit. The result is then in file "stats.txt".
WolframRhodium is offline   Reply With Quote
Old 25th March 2021, 16:12   #145  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,468
@WolframRhodium I think he means the avisynth version. For VS we already have this scrpt here https://github.com/theChaosCoder/zop...er/zoptilib.py
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database || https://github.com/avisynth-repository
ChaosKing is offline   Reply With Quote
Old 25th March 2021, 16:33   #146  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,197
Yes, I mean the ported AVISynth version. It appears that Asd-g is not a forum member.
__________________
Nostalgia's not what it used to be
WorBry is offline   Reply With Quote
Old 25th March 2021, 16:41   #147  |  Link
WolframRhodium
Registered User
 
Join Date: Jan 2016
Posts: 115
Sorry about that, I did not read the thread carefully. I thint it is possible through WriteFile, but I am not very familiar with it.
WolframRhodium is offline   Reply With Quote
Old 25th March 2021, 21:09   #148  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,468
Quote:
Originally Posted by zorr View Post
Looks like there is no way to read the similarity metric back into a script variable which is how the current SSIM metric is handled. Perhaps this could be implemented by passing a (global?) variable name and GMSD / MDSI functions would assign that variable?
I think it should work like in Vapoursynth now.

https://github.com/Asd-g/AviSynthPlu...MDSI.avsi#L103

Quote:
"""propSet("_FrameMDSI",
Avisynth+ 3.X supports clip props just like vapoursynth.

They are read for show=true at the script bottom
Code:
ScriptClip("""Subtitle("FrameMDSI: " + String(propGetFloat("_FrameMDSI")))"""
So maybe zopti needs a "zoptilib.py" version for avisynth now?
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database || https://github.com/avisynth-repository
ChaosKing is offline   Reply With Quote
Old 28th March 2021, 01:17   #149  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 265
Thanks Chaos, I didn't notice that the metric can indeed be read from the frame properties. So here's an example script using GMSD:

Code:
TEST_FRAMES = 5			# how many frames are tested
MIDDLE_FRAME = 50		# middle frame number

RawSourcePlus("D:/optimizer/test/flower/flower_cif.yuv", width=352, height=288, pixel_type="I420")
source=ColorYUV(levels="PC->TV")
noisy=source.AddGrain(25, 0, 0, seed=1)
#return noisy

#denoised=noisy.FFT3DFilter(sigma=4, bt=4, bw=16, bh=16, ow=8, oh=8)		# best settings by Fizick

sigma = 400/100.0 	# optimize sigma = _n_/100.0 | 100..800 | sigma
bt = 4				# optimize bt = _n_ | -1..5 | blockTemporal
blockSize = 32		# optimize blockSize = _n_ | 2..64 ; min:overlap 2 * | blockSize
overlap = 16		# optimize overlap = _n_ | 0..32 ; max:blockSize 2 / | overlap
denoised=noisy.FFT3DFilter(sigma=sigma, bt=bt, bw=blockSize, bh=blockSize, ow=overlap, oh=overlap)

# cut out the part used in quality / speed evaluation
source = source.Trim(MIDDLE_FRAME - TEST_FRAMES/2 + (TEST_FRAMES%2==0?1:0), MIDDLE_FRAME + TEST_FRAMES/2)
denoised = denoised.Trim(MIDDLE_FRAME - TEST_FRAMES/2 + (TEST_FRAMES%2==0?1:0), MIDDLE_FRAME + TEST_FRAMES/2)
last = denoised

last = GMSD(source, denoised)

global total = 0.0
global gmsd_total = 0.0
FrameEvaluate(last, """
	global gmsd = propGetFloat("_PlaneGMSD")
	global gmsd = (gmsd == 0.0 ? 1.0 : gmsd)
	global gmsd_total = gmsd_total + gmsd	
""")	

# measure runtime, plugin writes the value to global avstimer variable
global avstimer = 0.0
AvsTimer(frames=1, type=0, total=false, name="Optimizer")

# per frame logging (gmsd, time)
delimiter = "; "
resultFile = "perFrameResults.txt"	# output out1="gmsd: MIN(float)" out2="time: MIN(time) ms" file="perFrameResults.txt"
WriteFile(resultFile, "current_frame", "delimiter", "gmsd", "delimiter", "avstimer")

# write "stop" at the last frame to tell the optimizer that the script has finished
frame_count = FrameCount()
WriteFileIf(resultFile, "current_frame == frame_count-1", """ "stop " """, "gmsd_total", append=true)

return last
This is just a slightly modified version of the script I used in the hands-on tutorial. Other than renaming the variables the only relevant changes are highlighted in red.

In this script GMSD gave indentical optimal values compared to SSIM except for the sigma which is 415 instead of 479. I didn't try MDSI yet.
zorr is offline   Reply With Quote
Old 28th March 2021, 12:47   #150  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 5,061
Is there a way to utilize Avisynth+'s Prefetch to speed up calculation? I tested it briefly but there were problems probably related to writing to the file asynchronously.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 28th March 2021, 17:11   #151  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 265
Quote:
Originally Posted by Boulder View Post
Is there a way to utilize Avisynth+'s Prefetch to speed up calculation? I tested it briefly but there were problems probably related to writing to the file asynchronously.
I haven't tried it myself but it's worth an investigation. I think there's going to be problems if multiple threads are writing to the same file.

Is it possible to force WriteFile to execute on one thread while the filters before it are multithreaded? The documentation states that you cannot set MT mode on script function calls, but would placing a MT_SERIALIZED plugin call just before WriteFile have that side effect? Or can you just say SetFilterMTMode("WriteFile", MT_SERIALIZED)?

On the other hand, perhaps the WriteFile is already MT_SERIALIZED but the threads are writing into it in arbitrary order and the last line is not always last as it should be. In that case the per frame results should be written into a temporary thread-safe variable which would be written to file in correct order when all the frames are finished. It works like that in Vapoursynth using a map, but Avisynth doesn't have such a data structure so I'm not sure how to implement that. Perhaps a simple array would suffice as long as it's thread safe.
zorr is offline   Reply With Quote
Old 28th March 2021, 18:04   #152  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 5,061
I'm not sure if WriteFile is internally marked as MT_SERIALIZED, but in general, it should already have a "correct" mode set. It is very much likely that the order of frames is very random since they finish at different times.

Setting both Prefetch(threads=1, frames=24) and Prefetch(threads=24, frames=1) give this error: java.lang.NumberFormatException: For input string: "1I don't know what 'delimiter' means.0.179144I don't know what 'delimiter' means.3150.284912" and the log file is full of these errors.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 29th March 2021, 02:55   #153  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 265
Quote:
Originally Posted by Boulder View Post
Setting both Prefetch(threads=1, frames=24) and Prefetch(threads=24, frames=1) give this error: java.lang.NumberFormatException: For input string: "1I don't know what 'delimiter' means.0.179144I don't know what 'delimiter' means.3150.284912" and the log file is full of these errors.
I got the same result, but I was able to make it work by defining both delimiter and frame_count as global. I'm still not sure if the result file will always be correct but it seems to work. However I didn't seem to gain any performance with the prefetch. Perhaps the serialized file writing escalates upstream and makes everything run serially.
zorr is offline   Reply With Quote
Old 29th March 2021, 05:07   #154  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 5,061
Quote:
Originally Posted by zorr View Post
I got the same result, but I was able to make it work by defining both delimiter and frame_count as global. I'm still not sure if the result file will always be correct but it seems to work. However I didn't seem to gain any performance with the prefetch. Perhaps the serialized file writing escalates upstream and makes everything run serially.
Yes, I think that's how it goes. Perhaps the only way to make it faster and threadsafe in Avisynth would be to use an array as you suggested. Or is it possible to make all the frames create their own log file and combine them when it's time to calculate the total score?

The VS version is also not perfect in multithreading sense but it is somewhat faster.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 16th April 2021, 00:02   #155  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 265
Zopti v1.1.0 released

A new major release is here! Zopti now supports executing multiple scripts simultaneously which can lead to very nice performance improvement. This can be very useful when the script itself is not able to utilize all of your cores.

Here's the full release info:
  • added support for multithreaded execution of scripts with argument -threads
    -default value of -threads is 1 meaning one script is executed at a time just like before
    -all optimization algorithms (nsga-ii, spea2, mutation, exhaustive) support the the -threads option
    -due to the nature of the heuristic algorithms (nsga-ii, spea2, mutation) there can be at most P threads running the scripts at the same time, where P is the population size
    -exhaustive algorithm has no thread limits
    -validate mode also supports -threads. there the maximum number of threads is the number of validated results (size of the pareto front)
    -note: using more than one thread will make time measurements less accurate but will not change the quality measurements
  • avsr version update (v0.2.2)
  • added jMetal.log.ini to disable multithreading related JMetal logging
  • updated Zoptilib to version 1.0.9m
  • bugfix: evaluate could not parse log files when Zopti was used

Download link is at the first post. I will shortly post some data on the multithreading performance.
zorr is offline   Reply With Quote
Old 16th April 2021, 00:49   #156  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 265
I tested the new -threads argument using the denoising tutorial script, this time using GMSD as the similarity metric and using 50 frames instead of 5. Here's the script:

Code:
import vapoursynth as vs
from zoptilib import Zopti
core = vs.core

TEST_FRAMES = 50		# how many frames are tested
MIDDLE_FRAME = 100		# middle frame number
	
# read input video
video = core.raws.Source(r'D:\optimizer\test\flower\flower_cif.yuv', 352, 288, src_fmt='I420')

zopti = Zopti(r'results.txt', metrics=['gmsd', 'time'])

# input color range is PC (full)
video = video.std.SetFrameProp(prop="_ColorRange", intval=0)

# add noise
noisy = video.grain.Add(var=25, seed=3)
#noisy.set_output()

# remove noise
sigma = 400/100 	# optimize sigma = _n_/100 | 400..600 ; filter:x 5 % 0 == | sigma
bt = 5			# optimize bt = _n_ | 3,5 | blockTemporal
blockSize = 14		# optimize blockSize = _n_ | 6,12 ; min:overlap 2 * | blockSize
overlap = 16		# optimize overlap = _n_ | 4..6 ; max:blockSize 2 / | overlap
denoised = noisy.fft3dfilter.FFT3DFilter(sigma=sigma, bt=bt, bw=blockSize, bh=blockSize, ow=overlap, oh=overlap)

# cut out the part used in quality / speed evaluation
video = video[MIDDLE_FRAME - TEST_FRAMES//2 + (1 if (TEST_FRAMES%2==0) else 0) : MIDDLE_FRAME + TEST_FRAMES//2 + 1]
denoised = denoised[MIDDLE_FRAME - TEST_FRAMES//2 + (1 if (TEST_FRAMES%2==0) else 0) : MIDDLE_FRAME + TEST_FRAMES//2 + 1]

zopti.run(video, denoised)
There's also an AviSynth version of the script which I posted to the Avisynth Zopti thread.

The script only has 246 valid parameter combinations to test so we can try all of them using
Code:
zopti denoise_ex.vpy -alg exhaustive -threads 1
It takes 140 seconds to test all of the combinations using one thread. Let's see the performance when using more threads:



The best result is about 20 seconds using all 24 threads (on a Ryzen 3900X which has 12 cores / 24 threads).

Another way to look at the scaling is to calculate how much faster we get the results when using -threads:



Using 24 threads is 7,1 times as fast as using only one thread. Of course this is just one data point and I don't mean to imply that you can always get such a performance improvement. I would have liked to include more tests using more real-world usage scenarios (HD source etc) but my stock cooled processor cannot handle those for more than a few seconds, it becomes so hot that my PC shuts down.

Comparing AviSynth and VapourSynth it looks like VS has the edge when using one or just a few threads, but the differences almost vanish when using 12 or more threads. The VS is also able to utilize more than 16 cores while with AVS the performance starts to degrade. The point where more threads are just slowing down is probably dependent on the specific script, I will have to run more tests on that.
zorr is offline   Reply With Quote
Old 17th April 2021, 08:37   #157  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 5,061
With the new multithreading support, is it beneficial to increase the population size even with this kind of approach to optimizing the resizer params? A word of warning: VS uses a lot of memory for caching by default so it may be wise to lower the max_cache_size if you launch multiple instances

zopti "test.vpy" -alg mutation -iters dyn -dyniters 18 -dynphases 2 -pop 1 -runs 1 -mutcount 1 -mutamount 0.1 0.01 -initial script
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 17th April 2021, 14:13   #158  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 265
Quote:
Originally Posted by Boulder View Post
With the new multithreading support, is it beneficial to increase the population size even with this kind of approach to optimizing the resizer params?

zopti "test.vpy" -alg mutation -iters dyn -dyniters 18 -dynphases 2 -pop 1 -runs 1 -mutcount 1 -mutamount 0.1 0.01 -initial script
In this case you have to increase the population size from 1, otherwise there's no use for more than one thread. It will for sure make the search faster but the quality (or the probability that you will find the optimal values) might go down a little bit. You can compensate for that with increasing -dyniters slightly. The net result should be as good (or better) results faster. You may have to do some testing to find how many threads give you the best bang for buck though.

Quote:
Originally Posted by Boulder View Post
A word of warning: VS uses a lot of memory for caching by default so it may be wise to lower the max_cache_size if you launch multiple instances
Yes that's one downside of using -threads. It's process level multithreading and every VSPipe process will use as much memory as a single one would. So the amount of free memory might limit how many threads you can actually use even though CPU is still not maxed out.
zorr is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 15:27.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.