[qyot27]

Originally Posted by hello_hello

So for video plugins that currently use the LSB hack to support 16 bit processing with AVS, is that basically a waste of time if your input is 8 bit and the output will be the same? I thought the idea was to process with greater precision, even if the video is dithered down to 8 bit in the end.
Not the I use the 16 bit hack much, but I thought that was the main point of it.

That...depends. Supersampling is generally understood more in the vein of 'resize up, filter at the higher resolution, then resize back down', but that's staying inside of the same bit depth. Running filters on high bit depth input might provide a smoother gradient scale, and you might be able to retain *some* of that when you dither back down to 8 bit and use a lossy encoder on it, but it may also be possible to get similar results without processing it in >8bit. That's why the LSB hack is typically intended to be input to programs that can stitch it back into a proper high bit depth stream, since at least then it can keep the same >8bit gradient scale.

There's also the question of how much CPU and memory all of that would eat and whether whatever benefit you do end up seeing is worth it, or whether keeping it all in the same bit depth is faster for the same general perceptual quality (and a lot of that is naturally subjective).

On the audio thing.... if a filter supports both 16 bit int and float, are you still better off converting to float first, precision-wise?

The only thing I've ever really heard about the benefit of processing in float vs. integer is the claim that float processing prevents clipping. How true that is in practice, I don't know. Also whether you'd even need to worry about clipping, depending on what filter(s) you're using.

If there's no obvious difference in quality, whichever is faster.
If neither is distinctly faster, use what the encoder you're giving it to supports.

Regarding the encoder part, if you're dealing with something like ffmpeg, ffmpeg as a command line program is distinct from the libavcodec encoders you'd be feeding the stream to. For instance, with MP3: LAME supports multiple formats including 16-bit integer, 32-bit integer, and float, whereas libshine only supports 16-bit integer. So optimally, even though ffmpeg has no problems opening float audio (and will perform the conversion prior to handing it to the encoder), if you wanted to encode with libshine, it would be more streamlined to output 16-bit int from AviSynth, so ffmpeg wouldn't have to convert it first. But if you were going to use LAME, you could use any of the three formats it allows.

Originally Posted by Wilbert

@qyot27, i changed the online documentation to reflect the actual behaviour.

Thanks.