another restore function

[MOmonster]

@all
Crestore is updated: Thanks to Didee who recognize a bug with a resized "clip2", there was a bugfix and the mt_masktools are now in Version 2.0a28 packed.

@Alain
For anime samples you get a better blenddetection with higher modes (4-8). For really low motion you should choose a bthresh near 100 (between 101 and 111) and not use the mthresh (set it to 0). For extrem low motion (only with postrocessed clip2 or really good quality of the source) Cdeblend(sd=true, mode>5, mil=?) will give a really heavy detection. The sd parameter isn´t implemented in Crestore and can also lead to a wrong detection (not so save).
It can also be, that some parts are overblended, no simple normconversion (-> special Cdeblendmodes).

@stg
I downloaded your samples and will have a look on it this evening.

[Fizick]

MOmonster,

Thanks for your works! I will try it.

One non-technical comment (it is general, not only to you) .

If you distribute GPL plugins binary (mt_masktools.dll), you must distribute its source too.

Here is part 3 of GPL license text:

  3. You may copy and distribute the Program (or a work based on it,
under Section 2) in object code or executable form under the terms of
Sections 1 and 2 above provided that you also do one of the following:

    a) Accompany it with the complete corresponding machine-readable
    source code, which must be distributed under the terms of Sections
    1 and 2 above on a medium customarily used for software interchange; or,

    b) Accompany it with a written offer, valid for at least three
    years, to give any third party, for a charge no more than your
    cost of physically performing source distribution, a complete
    machine-readable copy of the corresponding source code, to be
    distributed under the terms of Sections 1 and 2 above on a medium
    customarily used for software interchange; or,

    c) Accompany it with the information you received as to the offer
    to distribute corresponding source code.  (This alternative is
    allowed only for noncommercial distribution and only if you
    received the program in object code or executable form with such
    an offer, in accord with Subsection b above.)

Our case is case (c).
So, if you downloaded some GPL binary WITHOUT source (for example Avisynth2.56, which have separate source archive package), you may distribute it without a source too.

But if you downloaded some GPL binary WITH a source in one zip package, you must distribute whole package (with source).

Other possible solution - do not distribute any plugins, but links only.
It will make your archive really very small.

Third You also can solution: make their separate full plugins package at your site.

GPL is not very "free" license.

[MOmonster]

@Filzick
Oh, I didn´t know that, thanks. I´ll do that.

@stg
I had a look on your sample yesterday. And yes it´s no wonder that Crestore doesn´t work on this sample. Your sample is the result of a so called DEFT conversion, scharfis_brain might correct my. So it´s not so bad like a hybrid source.
There are some possibilities to restore this.
You can for example use Cdeblend with the right settings (mode=1,...) and than decimate to 25fps. Or you use my unDEFTmatcher. It´s a little bit older, but the results with your source were ok. This is the script I used:

a = last.tdeint(mode=1)
unDEFTmatcher(last, bob=a, MI=55)

Function unDEFTmatcher(clip clp, clip "bob", float "blenddiff", int "MI") 
{
###### PREPARATION ######
MV = default(MI,77)
global out = bob
global testclip = clp.separatefields()
global combclip = clp
global matchmask = clp.tfm(mode=0,pp=2,MI=MV,clip2=clp.greyscale())
global diff = default(blenddiff,1.15)

###### VAR.. ######
global me1 = 1.0
global mo1 = 1.0
global choose = false
global match1 = false
global matchc = false
global match0 = false
global matchb = false

###### Conditional Function Chain, evaluated from bottom to top (!) ######
c99=scriptclip(clp, "match1 == true && match0 == true && matchc == matchb ? matchmask :
	\	(choose == false ? out.selecteven() : out.selectodd())")

c5=FrameEvaluate(c99, "global choose = ((me1 < me0 ? me0/me1 : me1/me0)*diff < (mo1 < mo0 ? mo0/mo1 : mo1/mo0)) ||
	\	(match0 == true && matchex == true && matchbb == matchb) ? false : 
	\	((me1 < me0 ? me0/me1 : me1/me0) > diff*(mo1 < mo0 ? mo0/mo1 : mo1/mo0) ? true : choose)
		global match1 = AverageChromaU(matchmask) != 128.000 ? true : false
		global matchc = ChromaUDifference(matchmask,combclip) < 0.001 ? true : false")
c4=FrameEvaluate(c5, "global match0 = match1
		global matchb = matchc
		global me1 = YDifferenceToNext(testclip.selecteven())
		global mo1 = YDifferenceToNext(testclip.selectodd())")
c3=FrameEvaluate(c4, "global matchex = match0
		global matchbb = matchb
		global me0 = me1
		global mo0 = mo1")
return(c3) 
}

You need tivtc to run this function and and should maybe optimize the internal tfm settings for your source.
Scharfis_Brain also created a function to restore these sources. You find it here.
I don´t know what is the optimal way for your source. Some less duplicates will be there because the transitions (between fieldshifted and progressive, fiedshifted and blended, ...). In case of speed and quality I prefer unDEFTmatcher. But if you hate single blends on scenechanges (I don´t recognize them during playback), the other to possibilities might work better for you.

Edit: I remove mt_masktools from the zip because of the GPL licence. You can download Manao´s newest mt_masktools here.

[stg]

Thank you for checking out the sample and for the script !!

I'll try it today and see if I can make it work, I'm not very good with this yet, I'm n00b what comes to these....

I was wondering if you could paste me the exact avisynth script + the exact parameters you used with this script, while testing that clip, thank you

[MOmonster]

Originally Posted by stg

I was wondering if you could paste me the exact avisynth script + the exact parameters you used with this script, while testing that clip, thank you

I allready gave you the full script in my last post (without the loading of the necessary filters):

#here you have to load the necessary plugins: tivtc, tdeint and DGDecode

mpeg2source("...")

a = last.tdeint()
unDEFTmatcher(last, bob=a, MI=55)

#and here is the code for the function
Function ...

The only advice I can give you (can´t test it over the weekend) is to lower the cthresh and maybe also the mthresh in tfm inside the function for your source:

global matchmask = clp.tfm(mode=0,pp=2,MI=MV,clip2=clp.greyscale())
  |
  V
global matchmask = clp.tfm(mode=0,cthresh=7,mthresh=3,pp=2,MI=MV,clip2=clp.greyscale())

Hope this is clear enough now.

[stg]

Thanks

Got it working now, altough I'm still making something wrong as my clips look "wacky", sometimes repeating the same scene multiple times.

Let's see how soon I can produce good results

[MOmonster]

Originally Posted by stg

Thanks

Got it working now, altough I'm still making something wrong as my clips look "wacky", sometimes repeating the same scene multiple times.

Let's see how soon I can produce good results

This is a problem I get sometimes if I load different vobs (from different sources) in dgdecode. Create a new d2v and try it one more time and use the source and dgdecode only in one instance.

Cu, MOmonster.

[stg]

It seems I get some promising results with CDeblend, not "quite there yet" but promising.

But with unDEFTmatcher.... every time the result looks like this:

http://download.yousendit.com/B4EE2B485990C698

Tried with different sources, dunno what's wrong.

[MOmonster]

Oh sorry, there is a big mistake in my script. Of course the bob parameter want a bobbed clip. Instead of a = last.tdeint() it should be a = last.tdeint(mode=1). I hope this works.

[stg]

Ok, now the results look much better ! Thanks. Doesn't look "wacky" anymore

[MOmonster]

Another update:
Crestore is updated to 1.0v2. I included the smoothmotion conditions that gives a much better patterndetection and motion for many sources. Because this don´t work so good for 12fps sources and with wrong settings it can conflict with the basic blenddetection default it is disabled. To enable this feature set the smuse Parameter (10-50).
Also I added another function called Mrestore. It´s similar to Crestore but it works better for good quality natural sources. It doesn´t need any external filters and has the same speed as Crestore(mode=-x) (->fast).

[desta]

I'm trying this out as Restore24 just doesn't want to work for me, but I when I load my script into Vdub Mod, there is always a message saying "Choosen *rate* won't work!!" Doesn't matter what rate I use or what changes I make to other settings, it doesn't go away.

There's also still a hell of a lot of telecined/field blended frames remaining.

[MOmonster]

You have to bob the source before you use C/Mrestore. For a PAL source you first bob to 50fps and than use Crestore/Mrestore with the rate 23.976 (default), for a NTSC source you first bob to 59.94fps and than choose the rate 25 (this is only the default case).
Example:

MPEG2SOURCE("... .d2v")

tdeint(mode=1)
Mrestore()

If this doesn´t work, a small sample could help.

[desta]

Many thanks. That did sort out the "choosen rate" error. I am still getting a lot of blended frames left over though, and quite a lot of interlaced ones too. I'm very new to this kind of restoring, so it's probably a case of me doing something wrong.

I'm uploading a sample vob somewhere now. Should I just paste the link here when it's done, or pm it?

Thanks again.


edit: I've uploaded it. It's a 106MB 2 minute sample, with a lot of fast motion. Megaupload.com - sample.rar. I apologise if the link is not allowed in the forums.

[MOmonster]

Originally Posted by desta

Many thanks. That did sort out the "choosen rate" error. I am still getting a lot of blended frames left over though, and quite a lot of interlaced ones too.

There shouldn´t be interlaced Frames after tdeint(mode=1). I hope you used Crestore and not Mrestore. It was just a sample, the last version was only for a limited range of sources. Your sample is really big but I downloaded it and will have a look on.

@all
I made another update.
Crestore is now a part of Mrestore. Mrestore has a modified blendthresholding and needs now also mt_masktools.
It enables profiles for a easier use of Mrestore. For more informations, please read the Mrestore.txt.

[desta]

Originally Posted by MOmonster

There shouldn´t be interlaced Frames after tdeint(mode=1). I hope you used Crestore and not Mrestore. It was just a sample, the last version was only for a limited range of sources. Your sample is really big but I downloaded it and will have a look on it.

I realise there shouldn't be interlaced frames, which is why I mentioned it. I used Crestore. Hopefully you'll see what I mean from the sample... I get the impression they did a very bad job of telecining/blending/ntsc>pal transfer. I'm actually thinking of just returning the dvd and buying the R1 version.

I appreciate the help.

[Alain2]

Good idea the profiles Maybe you can have a look at the hybridfupp script to see how he creates and handles profiles, seems easier to read and maintain

[MOmonster]

@Alain
Maybe I´ll do it, but at the moment Mrestore is finished. I have no idea for a faster or better blenddetection (at the moment). The decimation is not so good like in restore24 but it´s really fast and the basic idea of Crestore. We´ll see what comes next. I´m more interested in programming at the moment.

@desta

tdeint(mode=1)

Mrestore(profile=2)
#Mrestore(profile=6)

This works for me. The output is fluid and the blends are gone. The strong motionblur in some scenes is more disturbing than the really less blends left (for me). These artefacts that looks similar to interlacing are compression artefacts because of the blendconversion. I don´t know if there is much you can do against (you can open a thread and ask for). Profile 2 and 6 both performs good with your source (I like profile 2 a bit more, but this is my taste), but I can´t say what will work better for the whole animation.
I think buying the r1 version is the best idea if these artefacts are too disturbing, but you should first try it, because the pattern is correct detected.

[desta]

Thanks for the advice MOmonster. I'm gonna check those profiles out and see how it looks. I'm just annoyed that Sony would go through this process to make a PAL DVD... I would've thought it made far more sense to just speed up the progressive film rather than go all 'round the houses the way they have.

Anyway.. I'm rambling, hehe. Thanks again.

[Alain2]

Originally Posted by MOmonster

@Alain
Maybe I´ll do it, but at the moment Mrestore is finished. I have no idea for a faster or better blenddetection (at the moment). The decimation is not so good like in restore24 but it´s really fast and the basic idea of Crestore. We´ll see what comes next. I´m more interested in programming at the moment.

Don't do it for me, i mentionned it just in case you didn't know of this preset method, which can help you I think if you have to dive into the code later.. Thanks for the updates btw