VirtualDub2: Video compression error: An unknown error occurred (may be corrupt data)

[stormy1777]

This will continue the discussion from this thread:

https://forum.doom9.org/showthread.p...34#post1874534

basically, when trying to encode using virtualdub2's internal x264-8bit plugin, first attempt sometimes works, however, subsequent attempts fail as well as run video analysis with following error:

---------------------------
VirtualDub Error
---------------------------
Video compression error: An unknown error occurred (may be corrupt data). (error code -100)
---------------------------
OK   
---------------------------

Latest mmap zip is there:

https://anonfile.com/q5BfW2r4nf/VirtualDub2_-_crash_zip

showing memory consumption in various stages.. we can add more data to this thread as it is needed. Another important finding is while looking at vdub using process explorer found:

ffdshow.ax is "loaded" in memory map, not sure if that is expected?

also, noticed 'x264vfw.dll' loaded, but the compression chosen is the builtin x264-8bit, NOT the one down at the bottom: x264vfw!!

images at: https://imgur.com/a/t2D4tqU

This reproduces with latest as well as older vdub2, I suspect something at the OS (windows 10) is somehow causing this to mixup/messup/pollute the limited address space for vdub32...