Disabled "Drop frame when seeking", no change.
The script I'm using does quite a lot of processing including Autocrop, AutoContrast, MCDegrain, FFT3DFilter, Resize and more.
I just find it a little disconcerting to not know where in the clip I am, will have to get used to using keyboard I guess.
EDIT: Or perfect my Scroll - Stop - Scroll - Stop, technique