Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
28th November 2018, 14:03 | #41 | Link |
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
|
What? f3kdb adds grain which increases complexity and therefore required bitrate for a given CRF.
__________________
Groucho's Avisynth Stuff |
28th November 2018, 14:54 | #42 | Link | |
Registered User
Join Date: Mar 2011
Posts: 4,829
|
Quote:
Edit: Originally all VD plugins were RGB only, but I know that slowly changed, and Avisynth+ has even more options compared to Avisynth 2.6 and also mentions VD2 support, but I only had a quick read and at the moment I don't understand most of it. http://avisynth.nl/index.php/LoadPlu...rtualDubPlugin http://avisynth.nl/index.php/FAQ_usi...ualdub_plugins Avisynth is a little integrated into VD2, although I used VD2's script editor for the first time about 30 seconds ago, however you can open VD2, open the script editor under the Tools menu, and type in a line to open the source video. I typed LsmashVideoSource("D:\test.mp4") into the text editor, Gradfun3() on the second line, used the "File/Save and open as AVI" menu, and the Avisynth output opened in VD2. One issue I found, was when saving a script for the first time, you must specify the file name with the avs file extension. The script editor appears not to add it automatically and it results in an "AVI Import Error" (which I recall you stumbling across in another thread). It also doesn't seem possible to resave a copy of a script with a new name after you've saved it the first time, which is a little annoying. The script editor has a few options I don't understand yet, but I had no problem editing the video with VD2 as if I'd opened the source directly. You're obviously familiar with VD and prefer using it (nothing wrong with that), but I don't think it's hard to take advantage of both worlds. One disadvantage of editing with VirtualDub is you have to encode the video or save it as a project and you can't open the unfinished project with a program other than VD (correct me if I'm wrong). Sometimes it might be easier to do things via Avisynth. For example applying different filtering to different sections of a video. I do this sort of thing frequently: A.Trim(0,2191).Crop(8,0,-8-0).Spline36Resize(640,480)\ ++A.Trim(2192,2255).Spline36Resize(640,480).Tweak(Sat=0.9)\ ++A.Trim(2256,3638)Spline36Resize(640,480).GreyScale()\ ++A.Trim(3639,0).Crop(18,14,-8-0).Spline36Resize(640,480) A totally fictional example, and you can probably do something similar with VD using it's own filters, but I've barely used VD for much more than basic cutting and remuxing myself, so I'm not sure how easy it is. Anyway, whatever works for you.... Last edited by hello_hello; 28th November 2018 at 16:25. |
|
28th November 2018, 15:03 | #43 | Link | |
Registered User
Join Date: Mar 2011
Posts: 4,829
|
Quote:
Given I've re-encoded the video I was using for testing to compare bitrates, but I haven't re-encoded it without any debanding as a comparison, I'll do so shortly just to see how the bitrate compares. |
|
28th November 2018, 16:16 | #44 | Link |
Registered User
Join Date: Mar 2011
Posts: 4,829
|
Here's the file/size bitrate results again, encoded without any filtering included. The same x264 settings each time. Similar to preset slower, with tune film and CRF18.
Re-encoded without any filtering, 547 MB, 1318 kb/s With GradFun3(), 559 MB, 1347 kb/s With f3kdb(), 656MB, 1582 kb/s With f3kdb(grainY=32,grainC=32), 577MB, 1391 kb/s I'm not claiming that'd be typical. It's just how it worked out for one source. I haven't tried Groucho2004's most recent suggestions for f3kdb yet. That'll probably happen another day after my motivation has replenished itself. Last edited by hello_hello; 28th November 2018 at 16:18. |
28th November 2018, 23:58 | #45 | Link | |
Registered User
Join Date: Jul 2002
Posts: 241
|
Hello!Hello!!
This suggestion: Quote:
As for file size, started to not trust myself, so re-did this re-encode 6 more times, each is about 24minutes the PC that does this is not my working desktop, so no worries it can run 100%cpu/gpu non-stop and it doesn't matter using the Script Editor had 2 lines: Code:
LSmashVideoSource("M:\temp\test.MP4") f3kdb() The f3kdb ran 3 times (not back to back, but alternating), and all 3 resulted in identical filesize and checksum/content is 100% identical - that is expected i suppose size: 203.2MB Frames: 5269, kbps 7897 1920x1080 25fps Then, the 3 runs withOUT f3kdb (just # the second line, and press F7 in Script Editor - video appears to RELOAD), then F7 in main window, and sizes did NOT come the same!!! each run resulted in different size file???? Code:
sizes: 276.0MB frames 5269, 1920x1080, 25fps, 10795kbps 276.2MB frames 5269, 1920x1080, 25fps, 10804kbps 284.3MB frames 5269, 1920x1080, 25fps, 11125kbps 1) files are not identical in the non-f3kdb case, that is very very strange, it should be a digital encoding, should be 100% same, unless there is some "randomness" to help compress??? but then file sizes have a pretty large delta (276-284)?? 2) f3kdb vs. non-f3kdb results in SMALLER filesize, again, not sure, but I suspect it is HIGHLY dependent on source video, and so, maybe it ADDS "grain" or "pixels" which might add complexity, but it may "FLATTEN" certain areas (deband??), which would make them "compress better" so over-all, it adds, but after compression, it REDUCES, but that's just a theory, by all means it reduced size on my files thus far, which allowed to REDUCE CRF from 16 to 14 and retain roughly same filesize.. unless the f3kdb plugin is installed incorrectly, and somehow misses bits, that i would not know, video "seems ok", my eyes are just getting used to this level of attention Thanks. Stormy. |
|
29th November 2018, 02:14 | #46 | Link |
Registered User
Join Date: Mar 2011
Posts: 4,829
|
I don't know what's going on there. I used VD2 to encode some of the video I'd been using for testing (I'd been encoding with MeGUI). I added Trim(0,5000) to the end of the script so as just to encode a section of it. The output format was MP4. No audio included.
Two encodes without f3kdb came out exactly the same. I used the File/SaveAs method and x264 was configured for Preset Slower, High Profile Level 4.1, Tune Film and CRF18. Even Properties in Explorer showed the same file size in number of bytes. (1426 kb/s & 35,712,281 bytes). With f3kdb in the script, two encodes also came out exactly the same. I thought they might be a little different, but maybe the noise being added by f3kdb isn't so random (1823 kb/s & 45,639,536 bytes). The script was this: FFVideoSource("E:\0102.mkv") f3kdb() Trim(0,5000) Last edited by hello_hello; 29th November 2018 at 02:25. |
29th November 2018, 18:38 | #47 | Link |
Registered User
Join Date: Jul 2002
Posts: 241
|
I don't want to waste everyone's time, this might be something specific with my "practices", which I'm not sure are even valid, my background is not really A/V related, I sort of learn as i go along
Was curious, did few more tests, it's pretty hard to say what's going on... this clip is ~30min long, trying to shorten it (time-lapse) to ~3min, so, bumped framerate to 165 (source is 25), then set "Convert to fps: 25", resulting file ends up 25fps.. all the tests were with this configuration. the ones with the f3k always show up smaller than without it in the GUI, did a selection of frames (0-6000) then F7 (which saves just that section), and sizes with f3k were constant and always lower without it (sizes without were also constant!) then took the end of the clip, frames 6000-end, F7, again, sizes behaved same as above, so, no size differences.... went back to full clip, some came at 276MB, others at 284MB, for same settings, so on full clip there IS size difference so it seems this size change happens with larger source?, not sure.. but at this point, I'm fine with what it is... Saybe the "convert to 25" picks frames randomly each time, and thus the changes, not sure... the other "suspicion" was F5 vs F7 in the Script editor (causing it to load "incorrectly"?), and 'alternating' i.e. not doing same clip over and over, and finally, using the SAME UI interface all this time (not re-launching it anew for each encode) again, I have no conclusive proof, but there seems to be something there.. also, f3k is, as expected, highly dependent on input source, maybe if there IS banding in source, it reduces, if not so much, it might increase? again, I'm fine with how it is, so don't worry about it too much, as more clips arrive in the coming days i may learn more.... Stormy. |
30th November 2018, 11:50 | #48 | Link |
Registered User
Join Date: Mar 2011
Posts: 4,829
|
I can't explain why the full clip should produce different results each time. Were you doing the frame rate stuff with VD?
I was only encoding 5001 frames from my original test clip, but I couldn't get VD to produce inconsistent results. File sizes were always the same with f3kdb() and always the same without it. I tried duplicating what I think you're doing to the frame rate with Avisynth, and the result was identical, with the exception of there being an extra frame at the end, and the bitrate was higher with f3kdb than without it, but I suspect you're correct and it could be somewhat source dependant. FFVideoSource("E:\0102.mkv") f3kdb() Trim(0,5000) AssumeFPS(165) ChangeFPS(25) The file sizes being different for the full encode does seem a bit of a mystery. I checked my encodes with MPC-HC. The right arrow on the keyboard tells MPC-HC to jump from one keyframe to the next, and in each case they were always in the same place. I don't know why that'd change just because the encode was longer. Unless LSmash isn't always outputting the requested frame correctly. Do the encodes appear to contain the same frames? I'd be tempted to try again, just in case, using LWLibavVideoSource (it'll index the first time you open the source) or even FFVideoSource, but I'm a bit obsessive about discovering the "why" in situations like that. Even though I didn't do it for the example above, for FFMS2 it's often better to decode in a single thread, or at least do so if you're experiencing problems. FFVideoSource("E:\0102.mkv", threads=1) Last edited by hello_hello; 30th November 2018 at 12:05. |
30th November 2018, 14:20 | #49 | Link |
Registered User
Join Date: Jul 2002
Posts: 241
|
Yes, I can sometimes spend days or weeks chasing such things, heck, if you're in, I'll stick around for a bit longer.. so far, server is holding up nicely despite the hours of almost continuous encoding
Attached is the screenshot of the FPS trickery (not sure if that is the right way, i think next time I'll get the source at higher FPS). Also attached is File Info from VD once opened, see size is different, totaly number of frames is reported same as 5269, kbps is higher in one vs. the other, not sure why.... For completeness this is mediainfo of the two files (both are without f3k): Code:
Format : MPEG-4 Format profile : Base Media Codec ID : isom (isom/iso2/avc1/mp41) File size : 276 MiB Duration : 3 min 30 s Overall bit rate : 11.0 Mb/s Writing application : Lavf57.78.100 Video ID : 1 Format : AVC Format/Info : Advanced Video Codec Format profile : High@L5 Format settings : CABAC / 8 Ref Frames Format settings, CABAC : Yes Format settings, RefFrames : 8 frames Codec ID : avc1 Codec ID/Info : Advanced Video Coding Duration : 3 min 30 s Bit rate : 10.8 Mb/s Width : 1 920 pixels Height : 1 080 pixels Display aspect ratio : 16:9 Frame rate mode : Constant Frame rate : 25.000 FPS Color space : YUV Chroma subsampling : 4:2:0 Bit depth : 8 bits Scan type : Progressive Bits/(Pixel*Frame) : 0.208 Stream size : 271 MiB (98%) Writing library : x264 core 152 r2851M ba24899 Encoding settings : cabac=1 / ref=8 / deblock=1:-1:-1 / analyse=0x3:0x133 / me=umh / subme=9 / psy=1 / psy_rd=1.00:0.15 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-3 / threads=6 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=60 / rc=crf / mbtree=1 / crf=14.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00 Color range : Limited Matrix coefficients : BT.470 System B/G Codec configuration box : avcC Audio ID : 2 Format : MPEG Audio Format version : Version 1 Format profile : Layer 3 Codec ID : mp4a-6B Duration : 3 min 30 s Duration_LastFrame : -20 ms Bit rate mode : Constant Bit rate : 192 kb/s Channel(s) : 2 channels Sampling rate : 44.1 kHz Compression mode : Lossy Stream size : 4.81 MiB (2%) Default : Yes Alternate group : 1 LARGER file: Code:
Format : MPEG-4 Format profile : Base Media Codec ID : isom (isom/iso2/avc1/mp41) File size : 284 MiB Duration : 3 min 30 s Overall bit rate : 11.3 Mb/s Writing application : Lavf57.78.100 Video ID : 1 Format : AVC Format/Info : Advanced Video Codec Format profile : High@L5 Format settings : CABAC / 8 Ref Frames Format settings, CABAC : Yes Format settings, RefFrames : 8 frames Codec ID : avc1 Codec ID/Info : Advanced Video Coding Duration : 3 min 30 s Bit rate : 11.1 Mb/s Width : 1 920 pixels Height : 1 080 pixels Display aspect ratio : 16:9 Frame rate mode : Constant Frame rate : 25.000 FPS Color space : YUV Chroma subsampling : 4:2:0 Bit depth : 8 bits Scan type : Progressive Bits/(Pixel*Frame) : 0.215 Stream size : 279 MiB (98%) Writing library : x264 core 152 r2851M ba24899 Encoding settings : cabac=1 / ref=8 / deblock=1:-1:-1 / analyse=0x3:0x133 / me=umh / subme=9 / psy=1 / psy_rd=1.00:0.15 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-3 / threads=6 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=60 / rc=crf / mbtree=1 / crf=14.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00 Color range : Limited Matrix coefficients : BT.470 System B/G Codec configuration box : avcC Audio ID : 2 Format : MPEG Audio Format version : Version 1 Format profile : Layer 3 Codec ID : mp4a-6B Duration : 3 min 30 s Duration_LastFrame : -20 ms Bit rate mode : Constant Bit rate : 192 kb/s Channel(s) : 2 channels Sampling rate : 44.1 kHz Compression mode : Lossy Stream size : 4.81 MiB (2%) Default : Yes Alternate group : 1 |
1st December 2018, 09:40 | #50 | Link |
Registered User
Join Date: Jul 2002
Posts: 241
|
Ok, so.. Tried FFVideoSource and it did NOT reproduce, tried saving 3 times alternate between with/out f3k function (2 lines in script), pressed F5 or F7 in Script Editor, then F7 in main UI, just changed file name to increment counter, and it did NOT reproduce. All sizes were identical.
For the record, on this source, with f3k@crf-14 198.2MB vs. withOUT f3k: 276.0MB... so it seems something related to the indexing done by LSmashVideoSource. EDIT: Indeed FFVideoSource did take a bit of time to initially OPEN the file (creating index), but it was 100% reliable in final size.. reading on the lsmash open, the advantage is no-upfront index, so maybe that logic along with the FPS trickery is causing each run to fall on different frames, hence different final compressed size.. really NO clue, i would argue that is a bug, but do not have the resources or knowledge to prove it.. switched to FFVideoSource, it is a bit slower on the initial OPEN, but fine afterwards, and 100% predictable, at least on this specific source Thanks for all the tips.. i think we can move on to other things Last edited by stormy1777; 1st December 2018 at 10:17. |
1st December 2018, 18:07 | #51 | Link |
Registered User
Join Date: Mar 2011
Posts: 4,829
|
If you still have the encodes using LSmash, you could try the function below, although it does seem like LSmash isn't being frame accurate with your source files. You could do much the same thing with StackVertical() to look for completely different frames, because when they're completely different it should be obvious.
Anyway, if the frames from each video are exactly the same, all you should see is a solid grey output from Avisynth when using this function. Of course it'd probably pay to index the files you're comparing with ffms2. You'd use it like this: A = FFVideoSource("D:\Encode 1.mp4") B = FFVideoSource("D:\Encode 2.mp4") ShowDiff(A, B) This is the function. Save it with an avsi extension and put it in the Avisynth plugins folder. Code:
function ShowDiff(clip Clip1, clip Clip2, bool "Amp", bool "Show", bool "Comp") { Amp = default(Amp, false) Show = default(Show, false) Comp = default(Comp, false) N1 = Clip1.levels(96, 1.0, 160, 96, 160).greyscale() N2 = Clip1.subtract(Clip2) N3 = (Amp) ? N2.levels(124, 1.0, 131, 0, 255) : N2 N4 = (Show) ? N3.merge(N1) : N3 N5 = (Comp) ? Clip1.compare(Clip2, show_graph=true) : N4 return N5 } Last edited by hello_hello; 3rd December 2018 at 16:30. |
3rd December 2018, 07:38 | #52 | Link | |
Registered User
Join Date: Jul 2002
Posts: 241
|
Quote:
So, I'm guessing it misses first/second frame (at start/end) for some reason due to some wrong "initialization" then it goes "downhill" from there, with minor delta that gets added up, probably why only shows up on longer videos and on re-open... the source has little motion... so, this "frame inaccuracy" might be an old bug or something, anyways, it's great to have gone thus far. Thanks to all who are so supportive Stormy. PS: in the birds photo, if u look below barely u can make out the tree slightly embossed, the first/last frames show it VERY clearly and entirely, the rest of the video shows it like in this screenshot, so it adds up to larger file size. EDIT: BTW, I think extension for plugin should be avsi and not avis Last edited by stormy1777; 3rd December 2018 at 08:02. |
|
3rd December 2018, 17:02 | #53 | Link | |
Registered User
Join Date: Mar 2011
Posts: 4,829
|
Quote:
Yep. I had a stupid. |
|
Thread Tools | Search this Thread |
Display Modes | |
|
|