Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 10th May 2019, 20:34   #3321  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,657
Quote:
Originally Posted by WolframRhodium View Post
One of the solutions is to stack frames together:
Code:
from functools import partial

rgb_clip_stacked = core.std.StackVertical([rgb_clip[i::10] for i in range(10)])

h = rgb_clip.height
sh = rgb_clip_stacked.height

def apply_lut(n, clip, file_list): 
    cube_file = file_list[n]
    cube_file_path = path + '//' + cube_file 
    return core.timecube.Cube(clip, cube=cube_file_path) 

accu_file_stacked = core.std.FrameEval(
    rgb_clip_stacked, 
    partial(apply_lut, clip=rgb_clip_stacked, file_list=file_list)
)
accu_file = core.std.Interleave(
    [core.std.Crop(accu_file_stacked, top=h*i, bottom=sh-h*(i+1)) for i in range(10)]
)
This will explode the memory in seconds. Stacking 10 frames of 1920x1080 here already took up 1GB of memory. In real life each segments would have around 200 frames.
lansing is offline   Reply With Quote
Old 11th May 2019, 07:45   #3322  |  Link
jackoneill
unsigned int
 
jackoneill's Avatar
 
Join Date: Oct 2012
Location: 🇪🇺
Posts: 760
Quote:
Originally Posted by lansing View Post
Can I improve the loading speed with some lazy loading?

Code:
path = r"my file path"
file_list = os.listdir(path) # saving the lut file names to an array

clip = core.lsmas.LWLibavSource(file)
clip = clip[0:990]
rgb_clip = core.resize.Bicubic(clip, matrix_in_s="709", format=vs.RGBS)

accu_file = core.std.BlankClip(rgb_clip, length=1) #dummy initial file for concat

for cube_file, frame in zip(file_list, range(0, 991, 10)):
	cube_file_path = path + '//' + cube_file
	accu_file += core.timecube.Cube(rgb_clip[frame: frame+10], cube=cube_file_path)

accu_file.set_output()
Instead of using vspipe, maybe you could use clip.output() in a loop. Then you might achieve lazy loading.
__________________
Buy me a "coffee" and/or hire me to write code!
jackoneill is offline   Reply With Quote
Old 11th May 2019, 14:18   #3323  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,657
Quote:
Originally Posted by jackoneill View Post
Instead of using vspipe, maybe you could use clip.output() in a loop. Then you might achieve lazy loading.
WolframRhodium's solution with FrameEval() is lazy loading, but what kills it is the startup time for the timecube, as each call took a second to load.

What I'm thinking now is maybe I can have a temp clip file on the top level and then inside the callback function use "global" to get that file into the function and update it on some logic? Something like this:

Code:
temp_clip = clip

def apply_lut(n, clip, file_list):
    global temp_clip

    if n > blah:
        temp_clip = core.timecube.Cube(clip, cube=cube_file_path)
        return core.timecube.Cube(clip, cube=cube_file_path)
    if n < blah and n > blah:        
        return temp_clip
    else:
        return clip

accu_file = core.std.FrameEval(rgb_clip, partial(apply_lut, clip=rgb_clip, file_list=file_list))
lansing is offline   Reply With Quote
Old 12th May 2019, 16:56   #3324  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,657
How can I print out log right in the script for debugging? I called print(some_string) and it's not outputting anything in the log of the editor.
lansing is offline   Reply With Quote
Old 12th May 2019, 17:54   #3325  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
After learning, the hard way, that VapourSynth (vspipe.exe) requires the environment variable %USERPROFILE% to be set correctly – otherwise loading of plugins that have been installed via VSRepo will fail – I was wondering why VapourSynth relies on %USERPROFILE% to deduce the location of the <AppData> directory. Wouldn't it be more obvious and more reliable to look at %APPDATA% instead? Sure, most of the time the path of <AppData> will be equal to "<UserProfile>\AppData\Roaming", but I think we can not really rely on that. If there exists a dedicated environment variable for <AppData>, why not make use of it? And, maybe, fall back "<UserProfile>\AppData\Roaming", if either %APPDATA% is not set or the specified path does not exist. Or even better: Don't rely on environment variables at all (you never know what some user has set up here!), but rather use the SHGetKnownFolderPath() system function?

__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 12th May 2019 at 18:03.
LoRd_MuldeR is offline   Reply With Quote
Old 12th May 2019, 19:44   #3326  |  Link
_Al_
Registered User
 
Join Date: May 2011
Posts: 321
Quote:
Originally Posted by lansing View Post
How can I print out log right in the script for debugging? I called print(some_string) and it's not outputting anything in the log of the editor.
naming script *.py, not *.vpy and running that script from any Python console, like IDLE etc.
_Al_ is offline   Reply With Quote
Old 12th May 2019, 20:45   #3327  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,548
Quote:
Originally Posted by LoRd_MuldeR View Post
...
I don't use environment variables. I only use this code to retrieve the path:
Code:
SHGetFolderPath(nullptr, CSIDL_APPDATA, nullptr, SHGFP_TYPE_CURRENT, appDataBuffer.data());
Of course it doesn't mean windows itself won't implement it by using %USERPROFILE% behind the scenes anyway. I guess.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 12th May 2019, 22:29   #3328  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
Quote:
Originally Posted by Myrsloik View Post
Of course it doesn't mean windows itself won't implement it by using %USERPROFILE% behind the scenes anyway. I guess.
You are right. My test shows that SHGetFolderPath() with parameters CSIDL_APPDATA and SHGFP_TYPE_CURRENT fails, if environment variable %USERPROFILE% is set to the wrong directory.

There is no such problem, if %USERPROFILE% is not set at all. Furthermore, if %USERPROFILE% is set to the wrong directory, SHGetFolderPath() with parameters CSIDL_APPDATA and SHGFP_TYPE_DEFAULT works.

Interestingly, SHGetFolderPath() with parameters CSIDL_PROFILE does not seem to care about %USERPROFILE% at all

My conclusion for now: If call SHGetFolderPath() with parameters CSIDL_APPDATA and SHGFP_TYPE_CURRENT has failed, then we should retry with SHGFP_TYPE_DEFAULT flag.
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 12th May 2019 at 22:35.
LoRd_MuldeR is offline   Reply With Quote
Old 13th May 2019, 00:16   #3329  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,657
Quote:
Originally Posted by _Al_ View Post
naming script *.py, not *.vpy and running that script from any Python console, like IDLE etc.
It doesn't work when I'm calling print inside a callback function that was used in FrameEval().
lansing is offline   Reply With Quote
Old 13th May 2019, 01:00   #3330  |  Link
WolframRhodium
Registered User
 
Join Date: Jan 2016
Posts: 162
Quote:
Originally Posted by lansing View Post
It doesn't work when I'm calling print inside a callback function that was used in FrameEval().
You can print values in vsedit by raising an exception.

Last edited by WolframRhodium; 13th May 2019 at 01:05.
WolframRhodium is offline   Reply With Quote
Old 13th May 2019, 01:43   #3331  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,657
Quote:
Originally Posted by WolframRhodium View Post
You can print values in vsedit by raising an exception.
How do I do that?
lansing is offline   Reply With Quote
Old 13th May 2019, 01:50   #3332  |  Link
_Al_
Registered User
 
Join Date: May 2011
Posts: 321
Quote:
Originally Posted by lansing View Post
It doesn't work when I'm calling print inside a callback function that was used in FrameEval().
you have to request actual frames, because if you do not preview it, nothing is happening, so at the end of your script you can add:
Code:
for frame in range(0, len(accu_file)):
    accu_file.get_frame(frame)
_Al_ is offline   Reply With Quote
Old 13th May 2019, 09:43   #3333  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,657
Quote:
Originally Posted by _Al_ View Post
you have to request actual frames, because if you do not preview it, nothing is happening, so at the end of your script you can add:
Code:
for frame in range(0, len(accu_file)):
    accu_file.get_frame(frame)
This is only for sequential reading, I'm trying to find out a problem caused by reading backward.
lansing is offline   Reply With Quote
Old 13th May 2019, 15:19   #3334  |  Link
_Al_
Registered User
 
Join Date: May 2011
Posts: 321
that loop could be reversed
Code:
for frame in reversed(range(0, len(accu_file))):
     accu_file.get_frame(frame)
or just clip could be reversed before that loop:
accu_file=accu_file[::-1]
_Al_ is offline   Reply With Quote
Old 17th May 2019, 00:08   #3335  |  Link
Natty
Noob
 
Join Date: Mar 2017
Posts: 221
is there a function in vs similar to avs's function setmemorymax()
i read http://www.vapoursynth.com/doc/ but couldn't find.
having issues of very high ram usage
Natty is offline   Reply With Quote
Old 17th May 2019, 00:39   #3336  |  Link
gonca
Registered User
 
Join Date: Jul 2012
Posts: 1,213
Quote:
Originally Posted by Natty View Post
is there a function in vs similar to avs's function setmemorymax()
i read http://www.vapoursynth.com/doc/ but couldn't find.
having issues of very high ram usage
Try
core.max_cache_size =xxxx
gonca is offline   Reply With Quote
Old 17th May 2019, 01:50   #3337  |  Link
aldix
Registered User
 
Join Date: Sep 2012
Posts: 156
dear all,

been away for long, but figured i'll finally take up python/vapoursynth.

however, been messing around and having problems with an avisynth encoding script i'm trying to port over whole day.

and now i just can't wrap my head around what i'm doing wrong. script itself is pieced together from what i've found here based on the plugins i need to use.

Quote:
import os
import sys
import vapoursynth as vs
scriptPath = "C:/vapoursynth editor r19 64bit/scripts/"
sys.path.append(os.path.abspath(scriptPath)
import finesharp as finesharp
import dehalo_alpha as dehalo_alpha
core = vs.get_core()
clip = core.ffms2.Source("c:/temp/video.mkv")
clip = core.fmtc.resample(clip,w="1280",h="720", kernel="blackmanminlobe", taps="4")
src_clip = core.mv.Super(clip,pel="1", sharp="2", rfilter="2")
shp = finesharp.sharpen(src_clip,mode=3)
bv1 = core.mv.analyse(src_clip,isb="True",delta="1",overlap="8",blksize="16",truemotion="False",search="5",chroma="True")
fv1 = core.mv.analyse(src_clip,isb="False",delta="1",overlap="8",blksize="16",truemotion="False",search="5",chroma="True")
video = core.mv.Degrain1(clip,src_clip,bv1,fv1,thsad="85")
video = dehalo_alpha(self,video,rx="1.1",ry="1.1",brightstr="0.8",ss="1.5")
video = core.f3kdb.Deband(video,sample_mode="2",dynamic_grain="False",keep_tv_range="False",dither_algo="3",input_depth="8",output_depth="8",y="48",cb="48",cr="48",grainY="48",grainC="48")
video.set_output()
Quote:
Failed to evaluate the script:
Python exception: invalid syntax (C:/vapoursynth editor r19 64bit/GoT_python_test.vpy, line 6)

Traceback (most recent call last):
File "src\cython\vapoursynth.pyx", line 1924, in vapoursynth.vpy_evaluateScript
File "C:/vapoursynth editor r19 64bit/test.vpy", line 6
import finesharp as finesharp
^
SyntaxError: invalid syntax
thank you so much in advance!

Last edited by aldix; 17th May 2019 at 02:47.
aldix is offline   Reply With Quote
Old 17th May 2019, 02:32   #3338  |  Link
gonca
Registered User
 
Join Date: Jul 2012
Posts: 1,213
Code:
scriptPath = "C:/vapoursynth editor r19 64bit/scripts/"
This might be an issue "/"
gonca is offline   Reply With Quote
Old 17th May 2019, 02:47   #3339  |  Link
aldix
Registered User
 
Join Date: Sep 2012
Posts: 156
right. i also started to think that maybe spaces shouldn't be there so i made a new folder.

however, even if i remove the "/" i still get an error, just a different one now. any idea?

Quote:
Failed to evaluate the script:
Python exception: No module named 'finesharp'

Traceback (most recent call last):
File "src\cython\vapoursynth.pyx", line 1927, in vapoursynth.vpy_evaluateScript
File "src\cython\vapoursynth.pyx", line 1928, in vapoursynth.vpy_evaluateScript
File "C:/vapoursynth editor r19 64bit/test.vpy", line 6, in
import finesharp as finesharp
ModuleNotFoundError: No module named 'finesharp'
aldix is offline   Reply With Quote
Old 17th May 2019, 02:54   #3340  |  Link
_Al_
Registered User
 
Join Date: May 2011
Posts: 321
sys.path.append(os.path.abspath(scriptPath)) # parentheses is missing
_Al_ is offline   Reply With Quote
Reply

Tags
speed, vaporware, vapoursynth

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 09:18.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.