Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 25th September 2016, 18:23   #1  |  Link
Navarre66
Registered User
 
Join Date: Mar 2013
Posts: 12
(Suggestion for a new filter)Generalized Video Deblurring for Dynamic Scenes

I've been trying to deblur this old VHS capture for several years now and had until now found nothing that could help. It was filmed with handheld camcorder with a lot of blur due to the motion of the camera. I finally found this Matlab code that does some major improvement, but it takes forever and a day to process a lot of video. I was hoping their was some skilled person here that might be willing to take up the challenge and convert this Matlab and C++ code to an avisynth plugin.

It was developed by Tae Hyun Kim & Kyoung Mu Lee from Seoul National University.

http://cv.snu.ac.kr/research/~VD/

https://www.youtube.com/watch?v=T6F5QzRm2lQ

Thanks in advance,
Navarre66

Last edited by Navarre66; 8th May 2018 at 15:26. Reason: Added video presentation of paper.
Navarre66 is offline   Reply With Quote
Old 6th May 2018, 21:06   #2  |  Link
Navarre66
Registered User
 
Join Date: Mar 2013
Posts: 12
Dynamic Video Deblurring using a Locally Adaptive Linear Blur Model

Tae Hyun Kim & Kyoung Mu Lee have continued developing and added correction for out of focus video. This is essential if you want to do video stabilization on hand-held video footage or the stabilization will show the video going in and out of focus. This isn't noticeable when the video is not stabilized because it is in sync with the motion-blur.

From what I can tell this code would work well with MVTools.

Perhaps pinterf would be interested in taking this up.

Latest paper.
http://cv.snu.ac.kr/publication/jour...M_PAMI2017.pdf

Source code is Matlab and C++
http://cv.snu.ac.kr/research/~VD/GVD-release_v1.zip

Thanks in advance,
Navarre66
Navarre66 is offline   Reply With Quote
Old 8th May 2018, 02:34   #3  |  Link
Navarre66
Registered User
 
Join Date: Mar 2013
Posts: 12
Here is an example of before and after with this filter that I ran myself. It isn't CSI, but it is marked improvement over the original.

It took 8 hours for 30 frames. I'm sure the speed can be improved because this code is not optimized at all.

Thanks in advance,
Navarre66
Original:

Deblurred:

Last edited by Navarre66; 8th May 2018 at 15:38. Reason: Updated images
Navarre66 is offline   Reply With Quote
Old 8th May 2018, 02:42   #4  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Have you tried FQSharp ? :- https://forum.doom9.org/showthread.php?t=120496
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 8th May 2018, 02:47   #5  |  Link
Navarre66
Registered User
 
Join Date: Mar 2013
Posts: 12
Quote:
Originally Posted by StainlessS View Post
I think I tried it at one point, but you have to provide an estimate of the blur. I never got results as good as what I just posted. The biggest problem is that the blur is not consistent.

Thanks,
Navarre66
Navarre66 is offline   Reply With Quote
Old 8th May 2018, 03:01   #6  |  Link
Navarre66
Registered User
 
Join Date: Mar 2013
Posts: 12
One of the things I forgot to make clear is that this method analyzes the motion of the video and uses that to determine the motion blur in order to reverse the motion blur. I'm pretty sure the motion estimations that are produced by MVTools could be used as input for this method.

As far as I know, FQSharp is only good for a consistent blur or if you only want to deblur a few frames as you adjust the parameters.

Navarre66
Navarre66 is offline   Reply With Quote
Old 8th May 2018, 06:31   #7  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,345
It looks very interesting, and the pdf results look good. Too bad there weren't any video samples to check for temporal consistency (or problems with flicker) - can you make any comments on that from what you've seen so far ?

Quote:
Originally Posted by Navarre66 View Post
It took 8 hours for 30 frames. I'm sure the speed can be improved because this code is not optimized at all.
You think that's a long time ?

It's nothing compared to the time required for attachments to get approved....*that* code is not optimized either ...

I can't see your pictures, but people skirt around the issue by posting to free 3rd party image hosts or file hosts
poisondeathray is offline   Reply With Quote
Old 8th May 2018, 09:22   #8  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Quote:
Originally Posted by poisondeathray View Post
You think that's a long time ?

It's nothing compared to the time required for attachments to get approved....*that* code is not optimized either ...

I can't see your pictures, but people skirt around the issue by posting to free 3rd party image hosts or file hosts
That made me laugh out aloud, nice one PDR.

Navarre66, suggest try PostImage.org, you can create an account [EDIT: free] very easily, or post anonymous if you prefer.

Paste this one (circled in red, copies to ClipBoard):


EDIT: Or the Thumbnail for forums, two options higher.
eg
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 8th May 2018 at 10:18.
StainlessS is offline   Reply With Quote
Old 8th May 2018, 15:37   #9  |  Link
Navarre66
Registered User
 
Join Date: Mar 2013
Posts: 12
I added a video presentation of the original paper that has some video clips of the original work. I also updated my own images.

Thanks,
Navarre66
Navarre66 is offline   Reply With Quote
Old 9th May 2018, 11:36   #10  |  Link
`Orum
Registered User
 
Join Date: Sep 2005
Posts: 178
Quote:
Originally Posted by Navarre66 View Post
It took 8 hours for 30 frames. I'm sure the speed can be improved because this code is not optimized at all.
Well I certainly hope it can be optimized, because that's at the point where speed alone would kill the use of the filter. However, the results look impressive, but I'd really have to see the source footage to grasp exactly how well it's performing.

EDIT: Nevermind, I missed the YouTube video at first. The results look quite good, but the output begs to then be run through Deshaker or equivalent for several scenes.

While you're probably right about some potential integration with MVTools, I can already sort of see a way to do this with the current MVTools, plus a little extra magic on the side. The basic idea would be to detect global motion blur (e.g. camera movement) via contrast detection or some other means (perhaps look at how MPEG-4 ASP does it for GMC), and then use MFlowInter() to interpolate those frames? No idea if this would work well or if it would even be practical, but it might be worth a quick test.

That said, while you mention pinterf is probably the best person for anything related to MVTools (and possibly the only one active on the forums now who fully understands its internals), I hate to ask him to do one more thing as he already does such a ridiculous amount for the community. If someone does pick this up though, I'll be sure to give it a test, though I'm not sure if I have much footage that needs a filter like this.
__________________
My filters: DupStep | PointSize

Last edited by `Orum; 9th May 2018 at 11:49.
`Orum is offline   Reply With Quote
Old 10th May 2018, 02:43   #11  |  Link
Navarre66
Registered User
 
Join Date: Mar 2013
Posts: 12
Quote:
Originally Posted by `Orum View Post
The results look quite good, but the output begs to then be run through Deshaker or equivalent for several scenes.
Yup. That is the next step. If you run a video like this through Deshaker first it just makes the blurriness even more obvious.
Navarre66 is offline   Reply With Quote
Old 10th May 2018, 06:47   #12  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,345
The results really are good, at least based on the the demo material. Much better than what we have now in terms of deconvolution , motion blur removal , including commercial offerings

If it's the real deal , I'm surprised some company like Adobe or some plugin company hasn't already worked with (or in the process of working) with the authors to commercialize it.

Quite a few operations outlined in the pdf, it looks quite complex and I think a massive undertaking. I have a feeling it would be extremely difficult to implement in avisynth.
poisondeathray is offline   Reply With Quote
Old 11th May 2018, 21:11   #13  |  Link
Motenai Yoda
Registered User
 
Motenai Yoda's Avatar
 
Join Date: Jan 2010
Posts: 709
why not try making a sharpen filter with MFlowBlur?
__________________
powered by Google Translator
Motenai Yoda is offline   Reply With Quote
Old 12th May 2018, 03:57   #14  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,345
Quote:
Originally Posted by Motenai Yoda View Post
why not try making a sharpen filter with MFlowBlur?
In what way ?

MFlowBlur won't work (as in by itself) because it doesn't accept negative values to reverse the blur. It overlays many copies of intermediate time pixels to make the blur larger, not smaller

Some other 3rd party plugins / approaches that add motion blur can use negative values (essentially subtracting blur), and they can work to a limited extent; but nothing as remarkable as the results presented here
poisondeathray is offline   Reply With Quote
Old 12th May 2018, 09:25   #15  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
McDegrainSharp() is one of my favorite script functions, works great almost always. (I include in pretty much every script)

http://forum.doom9.org/showthread.ph...45#post1737045

EDIT: Only sharpens where good matches found, so maybe of limited use here.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 12th May 2018 at 10:07.
StainlessS is offline   Reply With Quote
Old 9th June 2018, 15:09   #16  |  Link
Motenai Yoda
Registered User
 
Motenai Yoda's Avatar
 
Join Date: Jan 2010
Posts: 709
Quote:
Originally Posted by poisondeathray View Post
In what way ?

MFlowBlur won't work (as in by itself) because it doesn't accept negative values to reverse the blur. It overlays many copies of intermediate time pixels to make the blur larger, not smaller

Some other 3rd party plugins / approaches that add motion blur can use negative values (essentially subtracting blur), and they can work to a limited extent; but nothing as remarkable as the results presented here
1- do a motionblur with MFlowBlur
2- subtract input clip from motionblur
3- add difference to input clip
__________________
powered by Google Translator
Motenai Yoda is offline   Reply With Quote
Old 9th June 2018, 17:42   #17  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,345
Quote:
Originally Posted by Motenai Yoda View Post
1- do a motionblur with MFlowBlur
2- subtract input clip from motionblur
3- add difference to input clip
Maybe in theory, I think you're going to cause more problems because it does not address the original motion blur correctly - because MFlowBlur is calculation is off already from a blurred image . Adding back the diff of the additional blur will not "reverse" the original blur

If you've ever used MFlowBlur (to add motion blur) to a clean source, with clear motion paths, ideal circumstance - you should already know the accuracy is quite poor.

Or do you have an example where this approach works semi decently , even on an "ideal" test scenario ?
poisondeathray is offline   Reply With Quote
Reply

Tags
avisynth, deblur, matlab, mvtools

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 13:21.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.