Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
20th April 2021, 09:30 | #1 | Link |
Registered User
Join Date: Jul 2016
Location: Mansfield, Ohio (formerly San Jose, California)
Posts: 280
|
SVSmoothFps, params_string, scene question
Bounty offered in my 2nd post.
I don't read 'C' so even if I found the source code (which I think is closed source) I wouldn't be able to answer my questions. My question is about the mode setting in the scene_string in the SVSmoothFps function. My frame diagrams and questions are in the notes. I would run some tests using a cooked up source having frame numbers, but I'm a novice and as of now, the only access I have to SVSmoothFps is via InterFrame and InterFrame doesn't support changes to 'scene:'. In that regard, I'd be grateful for any suggestions. Regards, Mark Filipak. Code:
SVSmoothFps(source,super,vectors,params_string,[sar]:float,[mt]:integer) params_string = "{"+rate_string+","+algo_string+","+mask_string+","+scene_string+","+light_string+"}" : Extended "scene change" controls. scene_string = "scene:{mode:3,blend:false,limits:{m1:1600,m2:2800,scene:4000,zero:200,blocks:20},luma:1.5}" : 0 ...uniform interpolation for maximum smoothness. For example for 24->60 conversion output will be: "1mmmm1mmmm..." [note 1] 1 ..."1m" mode that gives "1mm1m1mm1m..." [note 2] 2 ..."2m" mode: "1m11m11m11..." [note 3] 3 ...adaptive mode that switches between modes 0,1,2 based on overall vector field quality. Is 'mode:0' the following? (which excludes frame [B] from the interpolation) Code:
<---------------------- 1/6 s (scale = 480 characters/s) ----------------------> [A_________________][B_________________][C_________________][D_________________] - input [AAAAA_][AAAAC_][AAACC_][AACCC_][ACCCC_][CCCCC_][CCCCA_][CCCAA_][CCAAA_][CAAAA_] - output (via interpolated motion vectors) ---><--------------- 1/12 s ---------------><--------------- 1/12 s ------------ - picture centers & picture judder: none Code:
<---------------------- 1/6 s (scale = 480 characters/s) ----------------------> [A_________________][B_________________][C_________________][D_________________] - input [AAAAA_][AAABB_][ABBBB_][BBBBC_][BBCCC_][CCCCC_][CCCDD_][CDDDD_][DDDDA_][DDAAA_] - output (via interpolated motion vectors) ---><----- 1/24 s -----><----- 1/24 s -----><----- 1/24 s -----><----- 1/24 s -- - picture centers & picture judder: none Is the following what "1mm1m1mm1m..." means? Code:
<---------------------- 1/6 s (scale = 480 characters/s) ----------------------> [A_________________][B_________________][C_________________][D_________________] - input [AAAAAA][AAAABB][AABBBB][BBBBBB][BBBCCC][CCCCCC][CCCCDD][CCDDDD][DDDDDD][DDDAAA] - output (via interpolated motion vectors) ------><--- 7/180 s ----><------ 2/45 s ------><--- 7/180 s ----><------ 2/45 s - picture centers & picture judder: 7:8 cadence @ 12 judders/s. Is the following what "1m11m11m11..." means? Code:
<---------------------- 1/6 s (scale = 480 characters/s) ----------------------> [A_________________][B_________________][C_________________][D_________________] - input [AA____][AB____][BB____][BB____][BC____][CC____][CC____][CD____][DD____][DD____] - output (via interpolated motion vectors) s ----><-- 13/360 s ---><------ 17/360 s ------><--- 161/1440 s ----><---- 8/225 - picture centers & picture judder: 516:1145 cadence @ 6 judders/s. Last edited by markfilipak; 21st April 2021 at 04:40. |
20th April 2021, 17:05 | #2 | Link |
Registered User
Join Date: Oct 2018
Posts: 324
|
__________________
AviSynth AiUpscale Last edited by Alexkral; 20th April 2021 at 17:10. |
21st April 2021, 03:22 | #3 | Link | |
Registered User
Join Date: Jul 2016
Location: Mansfield, Ohio (formerly San Jose, California)
Posts: 280
|
Weighted span-of-four interpolation -- Bounty
Quote:
Just below the diagram, vanden asks, "Is that correct ?" What is shown as 3:2 pulldown is actually 6:4 pulldown -- a quibble, I know. If, in the SVSmoothFps documentation, "smoothness" means judder, than "1mmmm1mmmm...maximum smoothness", "1mm1m1mm1m...less smoothness", and "1m11m11m11...much less smoothness" is not consistent with my findings (below) in which "2m" mode appears to have much less judder than does "1m" mode. 1m@60fps (1mm1m1mm1m...): From vanden's diagrams, 1m@60fps (as a frame timing diagram, with my calculations) is: Code:
<---------------------- 1/6 s (scale = 480 characters/s) ----------------------> [A_________________][B_________________][C_________________][D_________________] - input [AA____][AB____][AB____][BB____][BC____][CC____][CD____][CD____][DD____][DA____] - output (via interpolated motion vectors) ------><--- 11/300 s ---><----- 7/150 s ------><--- 11/300 s ---><----- 7/150 s - picture centers & picture judder: 24%, 11:14 cadence, 12 judders/s. A center (from left edge) = (1*(-1*8+4)+2*(0*8+4)+1*(1*8+4)+1*(2*8+4))/5 = 7.2 characters. B center (from left edge) = (1*(1*8+4)+1*(2*8+4)+2*(3*8+4)+1*(4*8+4))/5 = 24.8 characters. C center (from left edge) = (1*(4*8+4)+2*(5*8+4)+1*(6*8+4)+1*(7*8+4))/5 = 47.2 characters. D center (from left edge) = (1*(6*8+4)+1*(7*8+4)+2*(8*8+4)+1*(9*8+4))/5 = 64.8 characters. A to B (center-to-center) = 24.8-7.2 = 17.6 characters = 11/300 s. B to C (center-to-center) = 47.2-24.8 = 22.4 characters = 7/150 s. C to D (center-to-center) = 64.8-47.2 = 17.6 characters = 11/300 s. D to A (center-to-center) = 80-64.8+7.2 = 22.4 characters = 7/150 s. judder% = 100*(frame_width_difference)/(average_frame_width)% = 100*(22.4-17.6)/((17.6+22.4)/2)% = 24%. judder cadence = 17.6:22.4 = 11:14. Code:
<---------------------- 1/6 s (scale = 480 characters/s) ----------------------> [A_________________][B_________________][C_________________][D_________________] - input [AA____][AA____][AB____][BB____][BB____][CC____][CC____][CD____][DD____][DD____] - output (via interpolated motion vectors) 300 s ---><----- 1/25 s -----><---- 13/300 s ----><----- 1/25 s -----><----- 13/ - picture centers & picture judder: 8%, 12:13 cadence, 12 judders/s. Code:
<---------------------- 1/6 s (scale = 480 characters/s) ----------------------> [A_________________][B_________________][C_________________][D_________________] - input [AA____][AB____][AB____][BC____][BC____][CC____][CD____][CD____][DA____][DA____] - output (via interpolated motion vectors) ---><----- 1/24 s -----><----- 1/24 s -----><----- 1/24 s -----><----- 1/24 s -- - picture centers & picture judder: none Background: Vanden's diagrams suggest that all existing motion vector interpolation is solely between a preceeding frame and a following frame. To give that a name, I would call it a "span-of-3 interpolation". For example, this (1m@60fps): Code:
<---------------------- 1/6 s (scale = 480 characters/s) ----------------------> [A_________________][B_________________][C_________________][D_________________] - input [A_____][AB____][AB____][B_____][BC____][C_____][CD____][CD____][D_____][DA____] - output (via interpolated motion vectors) [AB] interpolated from [A] & [B], [BC] interpolated from [B] & [C], [CD] interpolated from [C] & [D], and [DA] interpolated from [D] & [A], plus 2 frame repeats: [AB] and [CD]. Comment: Even though the repeated frames are interpolated frames, frame repeats are not interpolation. The frame repeats produce judder. Issue 1: The SVSmoothFps documentation for 'algo:11' refers to a "dynamic median" without defining what that means. Does "dynamic median" mean 4 span-of-3 interpolations followed by 8 span-of-3 interpolations between them followed by 6 discards as follows? Code:
<---------------------- 1/6 s (scale = 480 characters/s) ----------------------> [A_________________][B_________________][C_________________][D_________________] - input [A_______][AB______][B_______][BC______][C_______][CD______][D_______][DA______] - 48fps (via span-of-3 interpolations) followed by Code:
[A_______][AB______][B_______][BC______][C_______][CD______][D_______][DA______] - 48fps (via span-of-3 interpolations) [A__][AAB][AB_][ABB][B__][BBC][BC_][BCC][C__][CCD][CD_][CDD][D__][DDA][DA_][DAA] - 72fps (via span-of-3 interpolations) followed by Code:
[A__][AAB][AB_][ABB][B__][BBC][BC_][BCC][C__][CCD][CD_][CDD][D__][DDA][DA_][DAA] - 72fps (via span-of-3 interpolations) [A_____][AB____][B_____][BBC___][BC____][C_____][CD____][D_____][DDA___][DA____] - 60fps output (via 6 discards) -><---- 149/3360 s ----><-- 131/3360 s --><---- 149/3360 s ----><-- 131/3360 s - - picture centers & picture judder: 13%, 131:149 cadence, 12 judders/s. Comment: As you can see, "dynamic median" results in judder that's more than "2m" but less than "1m". The judder is produced by the discards. Comment: When attempting to convert 2 frames to 5 frames, any process that tries to do it via nested span-of-3 interpolations is going to have to repeat (or discard) some frames. Judder is caused by the repeats (or discards). Issue 2: Because I'm a novice and didn't know better, in my original post I had this: Code:
<---------------------- 1/6 s (scale = 480 characters/s) ----------------------> [A_________________][B_________________][C_________________][D_________________] - input [AAAAA_][AAABB_][ABBBB_][BBBBC_][BBCCC_][CCCCC_][CCCDD_][CDDDD_][DDDDA_][DDAAA_] - output (via interpolated motion vectors) [AAABB_][ABBBB_] interpolated from [A] & [B], [BBBBC_][BBCCC_] interpolated from [B] & [C], [CCCDD_][CDDDD_] interpolated from [C] & [D], and [DDDDA_][DDAAA_] interpolated from [D] & [A]. There is no dynamic median. There are no frame repeats or discards. There is no judder. Comment: Weighted span-of-4 interpolations is the motion-vector equivalent of weighted span-of-4 spacial pixel interpolation (which I have done via ffmpeg's 'mix' filter). It appears that weighted span-of-4 interpolation is not available. Bounty Since professional programmers need to make a living, even for FOSS software, I will pay a bounty for the creation of weighted span-of-4 interpolation. |
|
21st April 2021, 08:48 | #4 | Link | |
Registered User
Join Date: Oct 2018
Posts: 324
|
There are more diagrams below, and a post from the dev saying that they are all "generally" correct. How accurate that is, I don't know, but the dev himself has used that same diagram in other posts (like this one) to explain this, and the Avatar images from the first post on the next page seem to confirm it.
Quote:
I think it's possible to achieve similar things by mixing interpolation and blending, as in the SVP x2 1m + madVR Smooth Motion example from the Avatar images.
__________________
AviSynth AiUpscale |
|
21st April 2021, 09:17 | #5 | Link | |||
Registered User
Join Date: Jul 2016
Location: Mansfield, Ohio (formerly San Jose, California)
Posts: 280
|
Quote:
1, The discussion was about new PTSs. Well, of course there would be new PTSs. That doesn't mean that the motion vector interpolation goes further than 1 interpolated frame between a pair of original frames (i.e. what I'm calling a "span-of-3 interpolation"). And 2, The output of mode 0 does have judder. It's very small, much much smaller than 3:2 pull-down (or 6:4 pull-down), but it is judder. If there was no repeated frames, there would be no judder. Do you agree? Quote:
Though we disagree a bit, you've been very helpful. I'm so grateful. Thanks again. --Mark. Last edited by markfilipak; 21st April 2021 at 09:20. Reason: "grateful" was "greatful". Oops. |
|||
21st April 2021, 21:00 | #6 | Link |
Registered User
Join Date: Jul 2016
Location: Mansfield, Ohio (formerly San Jose, California)
Posts: 280
|
I've reworked what I posted. I reworked it to expose the judder in the output pictures instead of the input pictures. That actually makes more sense because it's the output pictures that are seen.
1m@60fps contains 8 pictures, so 1m@60fps is 60fps that carries 48pps (pictures-per-second). 2m@60fps contains 7 pictures, so 2m@60fps is 60fps that carries 42pps. Uniform@60fps contains 6 pictures, so Uniform@60fps is 60fps that carries 36pps. The details follow. 1m@60fps -- 1mm1m1mm1m -- converts pictures-per-second from 24pps to 48pps and conveys them via 60fps. Code:
<-------------------------------- 1/6 s (scale = 480 characters/s) --------------------------------> [A______________________][B______________________][C______________________][D______________________] input, 24fps(24pps) [A_______][AB______][AB______][B_______][BC______][C_______][CD______][CD______][D_______][DA______] output, 60fps(48pps) via interpolated motion vectors 1 m m 1 m 1 m m 1 m 60 -><--- 1/40 ----><--- 1/40 ----><- 1/60 -><- 1/60 -><--- 1/40 ----><--- 1/40 ----><- 1/60 -><- 1/ center-to-center picture timing, seconds picture judder: 40%, 2:2:3:3 cadence, 12 judders/s. Code:
<-------------------------------- 1/6 s (scale = 480 characters/s) --------------------------------> [A______________________][B______________________][C______________________][D______________________] input, 24fps(24pps) [A_______][AB______][B_______][B_______][BC______][C_______][C_______][CD______][D_______][D_______] output, 60fps(42pps) via interpolated motion vectors 1 m 1 1 m 1 1 m 1 1 ----><- 1/60 -><--- 1/40 ----><--- 1/40 ----><--- 1/40 ----><--- 1/40 ----><--- 1/40 ----><-- 1/40 - center-to-center picture timing, seconds picture judder: 40%, 2:3:3:3:3:3:3 cadence, 6 judders/s. Code:
<-------------------------------- 1/6 s (scale = 480 characters/s) --------------------------------> [A______________________][B______________________][C______________________][D______________________] input, 24fps(24pps) [A_______][AB______][AB______][BC______][BC______][C_______][CD______][CD______][DA______][DA______] output, 60fps(36pps) via interpolated motion vectors 1 m m m m 1 m m m m ----><--- 1/40 ----><------ 1/30 ------><--- 1/40 ----><--- 1/40 ----><------ 1/30 ------><---- 1/40 center-to-center picture timing, seconds picture judder: 29%, 3:3:4 cadence, 12 judders/s. (not available) -- converts pictures-per-second from 24pps to 60pps and conveys them via 60fps. Code:
<-------------------------------- 1/6 s (scale = 480 characters/s) --------------------------------> [A______________________][B______________________][C______________________][D______________________] input, 24fps(24pps) [A_______][AAABB___][ABBBB___][BBBBC___][BBCCC___][C_______][CCCDD___][CDDDD___][DDDDA___][DDAAA___] output, 60fps(60pps) via interpolated motion vectors 1 m m m m 1 m m m m ----><-------- 1/24 ---------><-------- 1/24 ---------><-------- 1/24 ---------><-------- 1/24 ----- center-to-center picture timing, seconds picture judder: none Last edited by markfilipak; 21st April 2021 at 21:03. |
21st April 2021, 23:06 | #7 | Link | |
Registered User
Join Date: Sep 2007
Posts: 5,377
|
Quote:
This one should not be happening with mode:0, but I can confirm it does despite fiddling with the settings. Those duplicates shouldn't be there. It affects avs version too It might be fixed in newer versions with svpflow manager running, but I'm running older "free" version. mvtools2 works ok, both avs and vpy |
|
21st April 2021, 23:51 | #8 | Link | |||||
Registered User
Join Date: Jul 2016
Location: Mansfield, Ohio (formerly San Jose, California)
Posts: 280
|
Quote:
Those duplicates shouldn't be there? If not, what do you think should be there? Quote:
Quote:
Do you know of anyone who'd like to earn the bounty by writing this: Quote:
Last edited by markfilipak; 21st April 2021 at 23:53. |
|||||
21st April 2021, 23:58 | #9 | Link | ||||
Registered User
Join Date: Sep 2007
Posts: 5,377
|
Quote:
Quote:
Quote:
vapoursynth version too Here is an example of one variation. You would change num to 60000, den to 1001 https://forum.doom9.org/showthread.p...02#post1814002 Quote:
|
||||
22nd April 2021, 00:05 | #10 | Link |
Registered User
Join Date: Oct 2018
Posts: 324
|
I just realized that what you are looking for is exactly the same as SVP 2.5x mode:0. I can't test it now but I can't see any judder when using it at home with the SVP 4 Pro Manager, so it must be a problem with the SVPflow libs you are using.
__________________
AviSynth AiUpscale |
22nd April 2021, 00:17 | #11 | Link | |||
Registered User
Join Date: Jul 2016
Location: Mansfield, Ohio (formerly San Jose, California)
Posts: 280
|
Did you look at this? https://www.svp-team.com/forum/viewt...d=49164#p49164
My diagrams are basically those but layed on their sides. You see, I'm a hardware design engineer and I prefer diagrams that look like timing diagrams. Mine have a lot more detail, though. Quote:
Quote:
Quote:
|
|||
22nd April 2021, 00:29 | #12 | Link | |||
Registered User
Join Date: Sep 2007
Posts: 5,377
|
Quote:
Quote:
Quote:
"maximum smoothest" does not really "jive" with "duplicates" in my book None of the modes (0,1,2,3) were smooth, all had duplicates. |
|||
22nd April 2021, 00:31 | #13 | Link | |
Registered User
Join Date: Jul 2016
Location: Mansfield, Ohio (formerly San Jose, California)
Posts: 280
|
Quote:
If the diagrams by MAG79 that you showed me are correct, then my diagrams are correct. There is judder (which actually becomes very evident when SVSmoothFps.mask.area is greater then 300), but at lower 'area'values and at 60fps, the judder is difficult to see. What I do is view my test videos over a 60Hz TV via DisplayPort. The TV is 50-inch and is about a foot and a half away, so critcal analysis is fairly easy. |
|
22nd April 2021, 00:36 | #14 | Link | |
Registered User
Join Date: Jul 2016
Location: Mansfield, Ohio (formerly San Jose, California)
Posts: 280
|
Quote:
You should examine my diagrams. I spent a lot of time developing them and I think there are no errors. Did you look at MAG79's diagrams (https://www.svp-team.com/forum/viewt...d=49164#p49164)? |
|
22nd April 2021, 00:43 | #15 | Link |
Registered User
Join Date: Jul 2016
Location: Mansfield, Ohio (formerly San Jose, California)
Posts: 280
|
Hey, is VapourSynth written entirely in python? No 'C'?
Never mind. The DLLs have to be in 'C', so I'm not going to develop my own mode:0 that doesn't have judder. |
22nd April 2021, 01:12 | #16 | Link | |
Registered User
Join Date: Sep 2007
Posts: 5,377
|
Quote:
Maybe I'm misreading it? For example MI(0,1)a and MI(0,1)b are different. One is at 0.33 , the other at 0.66 Looking at it more closely, I might be getting something slightly different than you; You're describing for mode:0 [A_______][AB______][AB______][BC______][BC______][C_______][CD______][CD______][DA______][DA______] That would imply that A and AB are different. But I'm actually getting no interpolation on the duplicates. There is no telltale artifacts, or slight warping of edges, or blending, or mix I'm getting A,A,B,B,C,C,C,D,D,E But my point is duplicates are "not smooth". And we're both getting them, even if the pattern description is slightly different Last edited by poisondeathray; 22nd April 2021 at 01:15. |
|
22nd April 2021, 01:16 | #17 | Link |
Registered User
Join Date: Oct 2018
Posts: 324
|
Yes. Look at the SVP 2.5x Uniform mode diagram here, there are no duplicates, the frames are interpolated at different positions in the time scale.
The last SVPflow versions that work correctly without the SVP Manager running are posted here: https://forum.doom9.org/showthread.p...45#post1914245
__________________
AviSynth AiUpscale |
22nd April 2021, 01:30 | #18 | Link | |
Registered User
Join Date: Sep 2007
Posts: 5,377
|
Quote:
Those fix the duplicates for me, for the vpy version. It works as expected now EDIT: spoke too soon, still some problems with duplicates...checking Last edited by poisondeathray; 22nd April 2021 at 01:39. |
|
22nd April 2021, 01:49 | #19 | Link |
Registered User
Join Date: Sep 2007
Posts: 5,377
|
Just so everyone is on the same page -
Here is a test clip, "test_6frames.mp4", 6 frames 23.976p 640x640 . It's a synthetic move, 20px horizontal per frame, so you can overlay a grid and calculate how precise or what ratio the interpolated frame is. (I know you're looking at 4 frame source cycles, but I added extra, because some algorithms have a "look ahead" and examine n+1,n+2 frames) The pattern 1mmmm1mmmm indicates which original frames positions are kept (frame zero, frame six), and that is consistent with the avisynth mvtools2 way of interpolation. There are no duplicates, but there are pro/cons to either resampling all frames , or keeping some original frames. Original frames have higher quality, synthesized frames have lower quality, more blur. Pros/cons. The quality fluctuation by keeping some original frames can be percieved as a sort of flicker. It's a more drastic effect on film grain, or noisy content. "1mmmm1mmmm" can be slightly less smooth than resampling all frame completely, but resampling all frames is generally prone to more artifacts https://www.mediafire.com/file/2tdxa...tests.zip/file test_6frames.mp4 is the test source test_frc.mp4 is mvtools2 avisynth based using default settings test_mvflowfps.mp4 is the mvtools2 vpy script above test_svpflow_mode0.mp4 is the avs version of svpflow, I think 2.x something. vpy version was the same, I didn't bother including Maybe Alexkral can test if newest SVP 4.x with manager exhibits the issue |
22nd April 2021, 02:32 | #20 | Link | |||||
Registered User
Join Date: Jul 2016
Location: Mansfield, Ohio (formerly San Jose, California)
Posts: 280
|
Quote:
Code:
[A_______][AB______][AB______][BC______][BC______][C_______][CD______][CD______][DA______][DA______] :........::..................::..................::........::..................::..................: picture 1 picture 2 picture 3 picture 4 picture 5 picture 6 Quote:
I'm going to resolve this issue. I'm going to dump raw frames and compare their data. If I need help with that, will you help, poisondeathray? Quote:
Quote:
Quote:
|
|||||
|
|