Here is the debug osd when the issue occurs (animated png)
The issue seems to occur when the scaling switches from "luma downscaling" to "image downscaling" (red arrow next to Picard's shoulder) which occurs when scaling crosses the 1.2x threshold (red arrow at bottom).
I have tried changing the scaling algorithm in MadVR settings without success.