Adding/using dither/ GradFun3 to VirtualDub2

[Groucho2004]

Originally Posted by stormy1777

BTW, added a call to: f3kdb() (using a *2* line script, open and f3kdb()), it (greatly) reduced filesize

What? f3kdb adds grain which increases complexity and therefore required bitrate for a given CRF.

[hello_hello]

Originally Posted by stormy1777

I'm trying to use the GUI as much as possible, if it was integrated, more likely i'd use it... not that i have an issue with scripts, it's just more time consuming as shown by the Xmax greeting set, scripting is a very POWERFUL tool, so no doubt will use it from time to time ...

It sounds like one needs to compile some code to generate a .vdf file, i was naively thinking it is some xml/txt that points to some binary/dll

It works the other way around though. You can load VD plugins into Avisynth with LoadVirtualDubPlugin()

Edit: Originally all VD plugins were RGB only, but I know that slowly changed, and Avisynth+ has even more options compared to Avisynth 2.6 and also mentions VD2 support, but I only had a quick read and at the moment I don't understand most of it.
http://avisynth.nl/index.php/LoadPlu...rtualDubPlugin
http://avisynth.nl/index.php/FAQ_usi...ualdub_plugins

Avisynth is a little integrated into VD2, although I used VD2's script editor for the first time about 30 seconds ago, however you can open VD2, open the script editor under the Tools menu, and type in a line to open the source video.
I typed LsmashVideoSource("D:\test.mp4") into the text editor, Gradfun3() on the second line, used the "File/Save and open as AVI" menu, and the Avisynth output opened in VD2.

One issue I found, was when saving a script for the first time, you must specify the file name with the avs file extension. The script editor appears not to add it automatically and it results in an "AVI Import Error" (which I recall you stumbling across in another thread). It also doesn't seem possible to resave a copy of a script with a new name after you've saved it the first time, which is a little annoying. The script editor has a few options I don't understand yet, but I had no problem editing the video with VD2 as if I'd opened the source directly.

You're obviously familiar with VD and prefer using it (nothing wrong with that), but I don't think it's hard to take advantage of both worlds.

One disadvantage of editing with VirtualDub is you have to encode the video or save it as a project and you can't open the unfinished project with a program other than VD (correct me if I'm wrong).

Sometimes it might be easier to do things via Avisynth. For example applying different filtering to different sections of a video. I do this sort of thing frequently:

A.Trim(0,2191).Crop(8,0,-8-0).Spline36Resize(640,480)\
++A.Trim(2192,2255).Spline36Resize(640,480).Tweak(Sat=0.9)\
++A.Trim(2256,3638)Spline36Resize(640,480).GreyScale()\
++A.Trim(3639,0).Crop(18,14,-8-0).Spline36Resize(640,480)

A totally fictional example, and you can probably do something similar with VD using it's own filters, but I've barely used VD for much more than basic cutting and remuxing myself, so I'm not sure how easy it is.

Anyway, whatever works for you....

[hello_hello]

Originally Posted by stormy1777

BTW, added a call to: f3kdb() (using a *2* line script, open and f3kdb()), it (greatly) reduced filesize...

That seems a little odd given by default f3kdb() appears to add a fair amount of noise/grain, but I did notice for at least one encode (I haven't tested it scientifically) the file size did increase when I didn't use GradFun3, and it adds noise too.

Given I've re-encoded the video I was using for testing to compare bitrates, but I haven't re-encoded it without any debanding as a comparison, I'll do so shortly just to see how the bitrate compares.

[hello_hello]

Here's the file/size bitrate results again, encoded without any filtering included. The same x264 settings each time. Similar to preset slower, with tune film and CRF18.

Re-encoded without any filtering, 547 MB, 1318 kb/s

With GradFun3(), 559 MB, 1347 kb/s

With f3kdb(), 656MB, 1582 kb/s

With f3kdb(grainY=32,grainC=32), 577MB, 1391 kb/s

I'm not claiming that'd be typical. It's just how it worked out for one source.

I haven't tried Groucho2004's most recent suggestions for f3kdb yet. That'll probably happen another day after my motivation has replenished itself.

[stormy1777]

Hello!Hello!!

This suggestion:

A.Trim(0,2191).Crop(8,0,-8-0).Spline36Resize(640,480)\
++A.Trim(2192,2255).Spline36Resize(640,480....

should be outlawed... it's crazy... I'm doing 100 other things, this Video thing is really a SIDE deal, so cannot afford to dig too deep constantly...

As for file size, started to not trust myself, so re-did this re-encode 6 more times, each is about 24minutes the PC that does this is not my working desktop, so no worries it can run 100%cpu/gpu non-stop and it doesn't matter

using the Script Editor had 2 lines:

LSmashVideoSource("M:\temp\test.MP4")
f3kdb()

crf=14, film, slower, Press F7 (on script editor and main window), change to MP4 and let it run audio Copy Stream, Video full processing.

The f3kdb ran 3 times (not back to back, but alternating), and all 3 resulted in identical filesize and checksum/content is 100% identical - that is expected i suppose

size: 203.2MB Frames: 5269, kbps 7897 1920x1080 25fps


Then, the 3 runs withOUT f3kdb (just # the second line, and press F7 in Script Editor - video appears to RELOAD), then F7 in main window, and sizes did NOT come the same!!! each run resulted in different size file????

sizes: 
    276.0MB frames 5269, 1920x1080, 25fps, 10795kbps
    276.2MB frames 5269, 1920x1080, 25fps, 10804kbps
    284.3MB frames 5269, 1920x1080, 25fps, 11125kbps

I have absolutely NO clue why:

1) files are not identical in the non-f3kdb case, that is very very strange, it should be a digital encoding, should be 100% same, unless there is some "randomness" to help compress??? but then file sizes have a pretty large delta (276-284)??

2) f3kdb vs. non-f3kdb results in SMALLER filesize, again, not sure, but I suspect it is HIGHLY dependent on source video, and so, maybe it ADDS "grain" or "pixels" which might add complexity, but it may "FLATTEN" certain areas (deband??), which would make them "compress better" so over-all, it adds, but after compression, it REDUCES, but that's just a theory, by all means it reduced size on my files thus far, which allowed to REDUCE CRF from 16 to 14 and retain roughly same filesize..

unless the f3kdb plugin is installed incorrectly, and somehow misses bits, that i would not know, video "seems ok", my eyes are just getting used to this level of attention

Thanks. Stormy.

[hello_hello]

I don't know what's going on there. I used VD2 to encode some of the video I'd been using for testing (I'd been encoding with MeGUI). I added Trim(0,5000) to the end of the script so as just to encode a section of it. The output format was MP4. No audio included.

Two encodes without f3kdb came out exactly the same. I used the File/SaveAs method and x264 was configured for Preset Slower, High Profile Level 4.1, Tune Film and CRF18. Even Properties in Explorer showed the same file size in number of bytes. (1426 kb/s & 35,712,281 bytes).

With f3kdb in the script, two encodes also came out exactly the same. I thought they might be a little different, but maybe the noise being added by f3kdb isn't so random (1823 kb/s & 45,639,536 bytes).

The script was this:

FFVideoSource("E:\0102.mkv")
f3kdb()
Trim(0,5000)

[stormy1777]

I don't want to waste everyone's time, this might be something specific with my "practices", which I'm not sure are even valid, my background is not really A/V related, I sort of learn as i go along

Was curious, did few more tests, it's pretty hard to say what's going on... this clip is ~30min long, trying to shorten it (time-lapse) to ~3min, so, bumped framerate to 165 (source is 25), then set "Convert to fps: 25", resulting file ends up 25fps.. all the tests were with this configuration.

the ones with the f3k always show up smaller than without it

in the GUI, did a selection of frames (0-6000) then F7 (which saves just that section), and sizes with f3k were constant and always lower without it (sizes without were also constant!)
then took the end of the clip, frames 6000-end, F7, again, sizes behaved same as above, so, no size differences....
went back to full clip, some came at 276MB, others at 284MB, for same settings, so on full clip there IS size difference

so it seems this size change happens with larger source?, not sure.. but at this point, I'm fine with what it is...
Saybe the "convert to 25" picks frames randomly each time, and thus the changes, not sure... the other "suspicion" was F5 vs F7 in the Script editor (causing it to load "incorrectly"?), and 'alternating' i.e. not doing same clip over and over, and finally, using the SAME UI interface all this time (not re-launching it anew for each encode) again, I have no conclusive proof, but there seems to be something there..

also, f3k is, as expected, highly dependent on input source, maybe if there IS banding in source, it reduces, if not so much, it might increase?

again, I'm fine with how it is, so don't worry about it too much, as more clips arrive in the coming days i may learn more....

Stormy.

[hello_hello]

I can't explain why the full clip should produce different results each time. Were you doing the frame rate stuff with VD?

I was only encoding 5001 frames from my original test clip, but I couldn't get VD to produce inconsistent results. File sizes were always the same with f3kdb() and always the same without it.

I tried duplicating what I think you're doing to the frame rate with Avisynth, and the result was identical, with the exception of there being an extra frame at the end, and the bitrate was higher with f3kdb than without it, but I suspect you're correct and it could be somewhat source dependant.

FFVideoSource("E:\0102.mkv")
f3kdb()
Trim(0,5000)
AssumeFPS(165)
ChangeFPS(25)

The file sizes being different for the full encode does seem a bit of a mystery. I checked my encodes with MPC-HC. The right arrow on the keyboard tells MPC-HC to jump from one keyframe to the next, and in each case they were always in the same place. I don't know why that'd change just because the encode was longer.

Unless LSmash isn't always outputting the requested frame correctly. Do the encodes appear to contain the same frames?
I'd be tempted to try again, just in case, using LWLibavVideoSource (it'll index the first time you open the source) or even FFVideoSource, but I'm a bit obsessive about discovering the "why" in situations like that.
Even though I didn't do it for the example above, for FFMS2 it's often better to decode in a single thread, or at least do so if you're experiencing problems.

FFVideoSource("E:\0102.mkv", threads=1)

[stormy1777]

Yes, I can sometimes spend days or weeks chasing such things, heck, if you're in, I'll stick around for a bit longer.. so far, server is holding up nicely despite the hours of almost continuous encoding

Attached is the screenshot of the FPS trickery (not sure if that is the right way, i think next time I'll get the source at higher FPS).

Also attached is File Info from VD once opened, see size is different, totaly number of frames is reported same as 5269, kbps is higher in one vs. the other, not sure why....

For completeness this is mediainfo of the two files (both are without f3k):

Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom (isom/iso2/avc1/mp41)
File size                                : 276 MiB
Duration                                 : 3 min 30 s
Overall bit rate                         : 11.0 Mb/s
Writing application                      : Lavf57.78.100

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L5
Format settings                          : CABAC / 8 Ref Frames
Format settings, CABAC                   : Yes
Format settings, RefFrames               : 8 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 3 min 30 s
Bit rate                                 : 10.8 Mb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 25.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.208
Stream size                              : 271 MiB (98%)
Writing library                          : x264 core 152 r2851M ba24899
Encoding settings                        : cabac=1 / ref=8 / deblock=1:-1:-1 / analyse=0x3:0x133 / me=umh / subme=9 / psy=1 / psy_rd=1.00:0.15 / 
mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-3 / threads=6 /
 lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 /
 b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40
 / intra_refresh=0 / rc_lookahead=60 / rc=crf / mbtree=1 / crf=14.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40
 / aq=1:1.00
Color range                              : Limited
Matrix coefficients                      : BT.470 System B/G
Codec configuration box                  : avcC

Audio
ID                                       : 2
Format                                   : MPEG Audio
Format version                           : Version 1
Format profile                           : Layer 3
Codec ID                                 : mp4a-6B
Duration                                 : 3 min 30 s
Duration_LastFrame                       : -20 ms
Bit rate mode                            : Constant
Bit rate                                 : 192 kb/s
Channel(s)                               : 2 channels
Sampling rate                            : 44.1 kHz
Compression mode                         : Lossy
Stream size                              : 4.81 MiB (2%)
Default                                  : Yes
Alternate group                          : 1

LARGER file:

Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom (isom/iso2/avc1/mp41)
File size                                : 284 MiB
Duration                                 : 3 min 30 s
Overall bit rate                         : 11.3 Mb/s
Writing application                      : Lavf57.78.100

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L5
Format settings                          : CABAC / 8 Ref Frames
Format settings, CABAC                   : Yes
Format settings, RefFrames               : 8 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 3 min 30 s
Bit rate                                 : 11.1 Mb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 25.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.215
Stream size                              : 279 MiB (98%)
Writing library                          : x264 core 152 r2851M ba24899
Encoding settings                        : cabac=1 / ref=8 / deblock=1:-1:-1 / analyse=0x3:0x133 / me=umh / subme=9 / psy=1 / psy_rd=1.00:0.15 /
 mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-3 / threads=6 /
 lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 /
 b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40
 / intra_refresh=0 / rc_lookahead=60 / rc=crf / mbtree=1 / crf=14.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40
 / aq=1:1.00
Color range                              : Limited
Matrix coefficients                      : BT.470 System B/G
Codec configuration box                  : avcC

Audio
ID                                       : 2
Format                                   : MPEG Audio
Format version                           : Version 1
Format profile                           : Layer 3
Codec ID                                 : mp4a-6B
Duration                                 : 3 min 30 s
Duration_LastFrame                       : -20 ms
Bit rate mode                            : Constant
Bit rate                                 : 192 kb/s
Channel(s)                               : 2 channels
Sampling rate                            : 44.1 kHz
Compression mode                         : Lossy
Stream size                              : 4.81 MiB (2%)
Default                                  : Yes
Alternate group                          : 1

Results with FFVideoSource as they become available...

[stormy1777]

Ok, so.. Tried FFVideoSource and it did NOT reproduce, tried saving 3 times alternate between with/out f3k function (2 lines in script), pressed F5 or F7 in Script Editor, then F7 in main UI, just changed file name to increment counter, and it did NOT reproduce. All sizes were identical.

For the record, on this source, with f3k@crf-14 198.2MB vs. withOUT f3k: 276.0MB... so it seems something related to the indexing done by LSmashVideoSource.

EDIT: Indeed FFVideoSource did take a bit of time to initially OPEN the file (creating index), but it was 100% reliable in final size.. reading on the lsmash open, the advantage is no-upfront index, so maybe that logic along with the FPS trickery is causing each run to fall on different frames, hence different final compressed size.. really NO clue, i would argue that is a bug, but do not have the resources or knowledge to prove it.. switched to FFVideoSource, it is a bit slower on the initial OPEN, but fine afterwards, and 100% predictable, at least on this specific source

Thanks for all the tips.. i think we can move on to other things

[hello_hello]

If you still have the encodes using LSmash, you could try the function below, although it does seem like LSmash isn't being frame accurate with your source files. You could do much the same thing with StackVertical() to look for completely different frames, because when they're completely different it should be obvious.
Anyway, if the frames from each video are exactly the same, all you should see is a solid grey output from Avisynth when using this function. Of course it'd probably pay to index the files you're comparing with ffms2.

You'd use it like this:

A = FFVideoSource("D:\Encode 1.mp4")
B = FFVideoSource("D:\Encode 2.mp4")
ShowDiff(A, B)

This is the function. Save it with an avsi extension and put it in the Avisynth plugins folder.

function ShowDiff(clip Clip1, clip Clip2, bool "Amp", bool "Show", bool "Comp")
{
    Amp = default(Amp, false)
    Show = default(Show, false)
    Comp = default(Comp, false)

    N1 = Clip1.levels(96, 1.0, 160, 96, 160).greyscale()
    N2 = Clip1.subtract(Clip2)
    N3 = (Amp) ? N2.levels(124, 1.0, 131, 0, 255) : N2
    N4 = (Show) ? N3.merge(N1) : N3
    N5 = (Comp) ? Clip1.compare(Clip2, show_graph=true) : N4

    return N5
}

[stormy1777]

Originally Posted by hello_hello

If you still have the encodes using LSmash, you could try the function below

Very nice!!! Thanks for that tip!! did that, and the results are as follows:

• FIRST TWO FRAMES show the trees as "embossed"

• after that, it is all GREY throughout the whole ~25minutes, with exception of some flying birds (attached screenshot), after the birds get out of the frame, it goes back to GREY..

• LAST FRAME IS GREY; TWO FRAMES before that show the trees as "embossed"

So, I'm guessing it misses first/second frame (at start/end) for some reason due to some wrong "initialization" then it goes "downhill" from there, with minor delta that gets added up, probably why only shows up on longer videos and on re-open... the source has little motion...

so, this "frame inaccuracy" might be an old bug or something, anyways, it's great to have gone thus far. Thanks to all who are so supportive

Stormy.

PS: in the birds photo, if u look below barely u can make out the tree slightly embossed, the first/last frames show it VERY clearly and entirely, the rest of the video shows it like in this screenshot, so it adds up to larger file size.

EDIT: BTW, I think extension for plugin should be avsi and not avis

[hello_hello]

Originally Posted by stormy1777

Very nice!!! Thanks for that tip!! did that, and the results are as follows:

I won't take the credit. I usually add a link to a function if I've borrowed it from someone else, but I must have forgotten this time. I'm pretty sure it was a function jagabo posted at VideoHelp at some stage.

Originally Posted by stormy1777

EDIT: BTW, I think extension for plugin should be avsi and not avis

Yep. I had a stupid.

[Blankmedia]

À, Who won between the two of them?


Gradfun3 or neo_f3kdb

[kedautinh12]

I don't know but i seen many people use f3kdb than gradfun3