Avisynth and virtualdub 32bit, going OOM

[tsk1979]

I am making a timelapse.

I have various sequences ranging from 450-700 frames.

So I just go with
a1=imagesource(...)
a2=imagesource(...)
.
.
Then I dissolve a1,a2.... and so on

I am seeing that when processing longer sequence (700 images of 1080P) size, it goes OOM.

I'm using 32bit because I need MSU deflicker plugin.
I am not sure if I am hitting the 2GB limit too. It seems odd that ImageSource will hit the memory limit.

Machine is Xeon i7 1231v3 and RAM is 16GB. I also have a Nvidia medium end graphics card(not that it matters).

Any idea I can figure out how much memory did my process request? Did it request more than 2GB?

Alternatively, I can assemble a video in 64bit and then run deflicker by reading that video in and then reading it out, but that is going to degrade quality(unless I go for uncompressed video(which will be massive running into many GBs)

Reading that video in 32bit is going to give me more problems I guess.

Those who do timelapse with 4000-5000 frames, how do you do it?

[vivan]

Dealing with a lot of clips is rather painful in avisynth.
You can rename all files and load them all a once, check examples here http://avisynth.nl/index.php/ImageSource

[raffriff42]

Right, if ImageSource is like AVISource, there's a limit of around 20-30 instances in 32bit no matter how much RAM you have.
http://forum.doom9.org/showthread.php?t=131687

Can you read all the images into one clip, then trim it into subclips and dissolve those...
m=imagesource(...)
a1=m.Trim(0, 500)
a2=m.Trim(501-999)
...

[tsk1979]

Thanks, so do you recommend I have one Imagesource command reading about 3700 frames, then trim them and then dissolve?

I was using around 9 imagesource commands. I guess that is the culprit.

Now I have named my files from 1..480 etc., and kept in different folders. I guess I need to rename them and then put them in one folder.

Lets see how that goes!

Alternatively I was thinking I will create 9 different uncompressed avi's and then load them 9 different avisource/directshowsource, but I guess its going to go boom on me.

I will try the single command route.

I was thinking having too many images in a single imagesource was the problem. But I guess having 9 imagesource commands was the problem

Thanks!

[wonkey_monkey]

You could try:

imagesource(...) # sequence of images
changefps(3*framerate) # pad it out a bit
convertfps(10*framerate) # 10 frame dissolve (fixed)
assumefps(30) # or whatever the target framerate is

Restrictions: "image on" time is can only be an integer multiple of image dissolve time.

[creaothceann]

Originally Posted by tsk1979

Now I have named my files from 1..480 etc., and kept in different folders. I guess I need to rename them and then put them in one folder.

Lets see how that goes!

Renamer has an option for that: add a rule, insert meta tag :File_FolderName:, add all files, rename, move all files into one directory.

[tsk1979]

Originally Posted by davidhorman

You could try:

imagesource(...) # sequence of images
changefps(3*framerate) # pad it out a bit
changefps(10*framerate) # 10 frame dissolve
assumefps(30) # or whatever the target framerate is

Restrictions: "image on" time is can only be an integer multiple of image dissolve time.

I could not understand this. You are saying I should create 9 different videos using this and then just add them(lossless adding)

[Gavino]

Originally Posted by davidhorman

changefps(10*framerate) # 10 frame dissolve

???

All this does is add 9 duplicates to every frame.
Perhaps you meant ConvertFPS(0.1*framerate) ?
But ConvertFPS() only blends two adjacent frames, and in any case the output frame rate must be at least two-thirds of the source rate.

[poisondeathray]

Originally Posted by tsk1979

Alternatively I was thinking I will create 9 different uncompressed avi's and then load them 9 different avisource/directshowsource, but I guess its going to go boom on me.

You don't have to use "uncompressed" ; you can use lossless compression such as lagarith, ut video codec, FFV1 , even x264 in lossless mode. Expect filesizes to be ~ 1/3 to 1/2 of uncompressed

Do you use the deflicker plugin after applying the dissolves? That can sometimes cause problems (not in terms of the memory issue, but in terms of the deflicker algorithm)

Those who do timelapse with 4000-5000 frames, how do you do it?

Timelapses work ok in NLE's. Maybe because of x64, maybe because of better memory handling. I think even the x86 versions handle them ok - maybe they "release" memory in a better way, or maybe they do a better job of caching to disk, not sure

[wonkey_monkey]

I probably misunderstood your task.

[tsk1979]

Originally Posted by poisondeathray

You don't have to use "uncompressed" ; you can use lossless compression such as lagarith, ut video codec, FFV1 , even x264 in lossless mode. Expect filesizes to be ~ 1/3 to 1/2 of uncompressed

Do you use the deflicker plugin after applying the dissolves? That can sometimes cause problems (not in terms of the memory issue, but in terms of the deflicker algorithm)



Timelapses work ok in NLE's. Maybe because of x64, maybe because of better memory handling. I think even the x86 versions handle them ok - maybe they "release" memory in a better way, or maybe they do a better job of caching to disk, not sure

Yes, I apply deflicker in virtualdub, so I presume it will be applied after dissolve since I load script through virtualdub.


I can do this.
Use 9 different imagesource, no filters, create a lossless compressed video x264

Then I just pass it to directshosource with all the filters I want?

[vivan]

To be honest I was thinking about hundreds of ImageSource calls, not just 9... It shouldn't be a problem. Do you do any other processing other then Dissolve() (and Deflicker in VDub)? MT?

As for memory consumption - you could try AVSMeter.

For lossless I prefer lossless codecs like new MagicYUV or UT Video - they are very fast (VirtualDub will be able to open resulting .avi files, in avisynth you should use AVISource). With x264 don't use directshowsource (it depends on system codecs and might be unreliable - like skipping/inserting frames) - use ffvideosource or LWLibavVideoSource instead.