Problem processing large video

[Cary Knoop]

I am trying to process a 4K video encoded ProRes 4:2:2 HQ.

Even when I try to simply open (commenting out the decimate) nothing seems to happen (no significant disk, CPU or memory activity):

from vapoursynth import core
import vapoursynth as vs

c = core.ffms2.Source('a:\video.mov')
#c = core.std.SelectEvery(c, cycle=3, offsets=[0,1])
  
c.set_output()

Running: Vapoursynth R45. Windows 10 on a Threadripper 1950X with 64GB of memory, source disk A:\ is an NVME drive.

[Cary Knoop]

Well, I just was proven to be impatient.
It's super slow though.

[Myrsloik]

Originally Posted by Cary Knoop

Well, I just was proven to be impatient.
It's super slow though.

Your drive is super slow

[Cary Knoop]

Originally Posted by Myrsloik

Your drive is super slow

My A:\ is mapped to an NVME PNY CS2030 480GB drive.

Not really my idea of super slow:



Source: https://www.techadvisor.co.uk/review...eview-3655600/

[VS_Fan]

You could try:

• accessing your disk directly without mapping it;
• download and run CrystalDiskMark on your disk

[asarian]

Originally Posted by Cary Knoop

Well, I just was proven to be impatient.
It's super slow though.

You realize FFMS2 needs to create an index of the file first, right?! On a large 4k film. that could takes minutes even before actual processing starts.

[poisondeathray]

Another option is LSmash , which does not need to index ISO base media formats , such as MOV, MP4

[Cary Knoop]

Originally Posted by poisondeathray

Another option is LSmash , which does not need to index ISO base media formats , such as MOV, MP4

I tried that as well, but it was very slow as well.

Never had problems opening files but in this case, for a very large and low compressed video, it is definitely slow. And this was only a 12 minutes master, what if it is a 2-hour master.

The suggestion that it must be my hard drive I frankly find not very helpful.

[Cary Knoop]

Originally Posted by asarian

You realize FFMS2 needs to create an index of the file first, right?! On a large 4k film. that could takes minutes even before actual processing starts.

I do, but it's very slow.

[poisondeathray]

Originally Posted by Cary Knoop

I tried that as well, but it was very slow as well.

Slow before opening ? As if it were indexing ?

or slow seeking performance "after" you were "patient ?"

Did you use core.lsmas.LibavSMASHSource , which is for ISO base media formats ? - This doesn't index and should open immediately

(core.lsmas.LWLibavSource is for other formats, and indexes)

[Cary Knoop]

Originally Posted by poisondeathray

Slow before opening ? As if it were indexing ?

or slow seeking performance "after" you were "patient ?"

Did you use core.lsmas.LibavSMASHSource , which is for ISO base media formats ? - This doesn't index and should open immediately

(core.lsmas.LWLibavSource is for other formats, and indexes)

Ah, I tried core.lsmas.LWLibavSource() instead of core.lsmas.LibavSMASHSource()!

And indeed, once it is working, consecutive access is almost instantly.

Thanks!

[Selur]

When using the ffms2 I usually create the index file with ffmsindex (add the created index to the call) and then open the Vapoursynth script.
This doesn't really speed things up, but it allows to know what time is spend on creating the index.

[asarian]

Originally Posted by Cary Knoop

Ah, I tried core.lsmas.LWLibavSource() instead of core.lsmas.LibavSMASHSource()!

And indeed, once it is working, consecutive access is almost instantly.

Thanks!

If you have a nVidia card, you could also give DGDecodeNV a try. Creating the .dgi file may take a while, but its frame-serving processing power is phenomenal.

[poisondeathray]

Originally Posted by asarian

If you have a nVidia card, you could also give DGDecodeNV a try. Creating the .dgi file may take a while, but its frame-serving processing power is phenomenal.

Won't help for prores

[asarian]

Originally Posted by poisondeathray

Won't help for prores

My bad. Fully missed he was talking about ProRes material.

[ChaosKing]

Is mov in ffms2 frame accurate? You should test it.

[poisondeathray]

Originally Posted by ChaosKing

I mov in ffms2 frame accurate? You should test it.

I-frame formats, like prores, are usually frame accurate

long gop in mov, such as some xdcam variants, h264 are usually frame accurate too with both lsmash/ffms2 in my experience