Question: Would be possible to use multiple threads to speed up indexing process? For example by using some kind of chunk indexing?
I'm asking because NVMe SSDs with read speed of 3GiB/s+ are getting much more affordable. I have Xeon E5-2690@3.2GHz (8C/16T) and I see that i'm experiencing a bottleneck due to single threaded code.
In this example I'm indexing 4k movie (~50Mbps) from RAM disk...
1,24GiB/s is a max I can get on single core. ffmsindex is even more cpu intensive (only ~0,5GiB/s).
Currently indexing of ~60 GiB 4k movie takes ~1 minute. With proper multi-threading we could easily reduce indexing time to ~15s!