Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 4th August 2019, 07:51   #301  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
well, I think F3KDB_3 is stable now, hope there are no bugs left

========

F3KDB_3 updates and features


difference from F3KDB:-

1- sample_mode 3 and 4 (4 is the Default)

2- new pass parameter (2 is Default if sample_mode = 4)

3- new chroma parameter

4- work with HBD and old lsb/interleaved hack and all avs/avs+ Versions

=======

if you use it without chroma=false or still using avs 2.5 you need flash3kyuu deband 1.5.1:- x86 / x64

if you use it with chroma=false and avs 2.6 / avs+ then you need f3kdb-b98d6bc in Attachments

if you use it with old lsb/interleaved hacks then you will need dither tools and HDRCore (in Attachments)

=========

note:- if F3KDB() added sample_mode 3 and 4 and pass parameter and others then this function will update to take that into account since coding those in dll will make them faster

======

download:-
this is a old one, see the 1st OP post

pastebin and in Attachments

=====

Samples:-

with this using ConvertBits(16).F3KDB_3(pass=4).ConvertBits(8, dither=1)

-------

with this (got it from here)

F3KDB() /// F3KDB_3()

Hqdn3d_2().F3KDB() /// Hqdn3d_2().F3KDB_3()

edit: see this post for last update info
Attached Files
File Type: zip f3kdb-b98d6bc.zip (234.6 KB, 176 views)
File Type: 7z HDRCore.7z (110.1 KB, 146 views)
File Type: rar Advanced Denoising v1.78.rar (10.3 KB, 82 views)
File Type: rar Advanced Denoising v1.81.rar (10.3 KB, 106 views)
__________________
See My Avisynth Stuff

Last edited by real.finder; 23rd March 2020 at 22:43.
real.finder is offline   Reply With Quote
Old 4th August 2019, 11:00   #302  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
What is the difference between f3kdb-b98d6bc and flash3kyuu 1.5.1 or v2.0pre?

EDIT: Only avs 2.5 compatibility?
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline   Reply With Quote
Old 4th August 2019, 12:14   #303  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
Quote:
Originally Posted by ChaosKing View Post
What is the difference between f3kdb-b98d6bc and flash3kyuu 1.5.1 or v2.0pre?

EDIT: Only avs 2.5 compatibility?
flash3kyuu 1.5.1 has bug with Y8 and input_mode > 0, I think 2.0pre fix that, but f3kdb-b98d6bc was the last one by SAPikachu (2.0pre may has some bugs that fixed in b98d6bc but the main reason is b98d6bc has x64 and x86 while 2.0pre is only x86)
__________________
See My Avisynth Stuff
real.finder is offline   Reply With Quote
Old 4th August 2019, 13:12   #304  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
And is this version (F3KDB Version 2.1 with native high bit depth) supported? https://forum.doom9.org/showthread.php?t=176553
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline   Reply With Quote
Old 4th August 2019, 16:42   #305  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
Quote:
Originally Posted by ChaosKing View Post
And is this version (F3KDB Version 2.1 with native high bit depth) supported? https://forum.doom9.org/showthread.php?t=176553
not yet, I answer this here https://forum.doom9.org/showthread.p...53#post1880853

and above
Quote:
note:- if F3KDB() added sample_mode 3 and 4 and pass parameter and others then this function will update to take that into account since coding those in dll will make them faster
__________________
See My Avisynth Stuff
real.finder is offline   Reply With Quote
Old 6th August 2019, 08:59   #306  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
new update for F3KDB_3 in the same post above

new masking parameter (true by default) with mask_thr and mask_radius parameters that make it act like f3kdbmod16 / flash3kyuuMOD16

edit: new parameter added (bool lastpassmask)

edit: test sample with the same things here https://forum.doom9.org/showthread.p...77#post1880477 but with smam(filter=daa3mod(slow=4)) instead of smam(filter=daa3mod()), with all scripts updates until today

https://i.imgur.com/TQkDvyO.png

edit: new F3KDB_3_adg https://pastebin.com/e196hcN9 (not part of Advanced Denoising.avsi)
__________________
See My Avisynth Stuff

Last edited by real.finder; 13th August 2019 at 04:55.
real.finder is offline   Reply With Quote
Old 18th August 2019, 20:11   #307  |  Link
Natty
Noob
 
Join Date: Mar 2017
Posts: 221
when i try to use FastLineDarkenMod2() alone i get error
i dont know what "avsplusnumber" means
is there any way to fix it without using SMDegrain forcefully?
same issues with other plugins.
Natty is offline   Reply With Quote
Old 18th August 2019, 21:01   #308  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
Quote:
Originally Posted by Natty View Post
when i try to use FastLineDarkenMod2() alone i get error
i dont know what "avsplusnumber" means
is there any way to fix it without using SMDegrain forcefully?
same issues with other plugins.
you dont need to use SMDegrain you just need to load it or put it in autoload folder

anyway you can copy the missing functions (like AvsPlusVersionNumber) from SMDegrain in any script you use
__________________
See My Avisynth Stuff

Last edited by real.finder; 18th August 2019 at 21:23.
real.finder is offline   Reply With Quote
Old 19th August 2019, 13:15   #309  |  Link
Natty
Noob
 
Join Date: Mar 2017
Posts: 221
thanks. moved Smdegrain, masktools and rgtools from staxrip's internal folder to plugins64+ folder and its fixed and working fast now. thanks.
Natty is offline   Reply With Quote
Old 26th August 2019, 20:01   #310  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
DeHaloHmod

new update for DeHaloHmod https://pastebin.com/mGRFQjje

now can work with minideen and still can work with old deen if minideen not present

edit: it's not work with minideen yet https://forum.doom9.org/showthread.p...58#post1883458 it will use old deen until this fixed

edit: it's fixed in r3 and r4 of minideen
Attached Files
File Type: zip DeHaloHmod v2.44.zip (3.1 KB, 85 views)
__________________
See My Avisynth Stuff

Last edited by real.finder; 13th September 2019 at 01:24.
real.finder is offline   Reply With Quote
Old 6th September 2019, 18:24   #311  |  Link
dREV
Registered User
 
dREV's Avatar
 
Join Date: Jan 2019
Location: Antarctica
Posts: 74
from https://forum.doom9.org/showthread.php?p=1884083

I haven't switched to 64 bit AviSynth+ because when I did my testing RAM went up over 90% on my single 8 GB RAM, so I planning to buy another 8 GB RAM as I use the same computer to do stuff while I wait for encode to finish.

Also, when I tried to use it on 32 bit AviSynth+ it would say this on avspmod:

"I don't know what 'AvsPlusVersionNumber' means
C:\ path\plugins\ResizeX_v1.0.1_mod 9.avsi, line 70
C:\ path\plugins\DebilinearM_v1.3.1_mod_1.25.avsi, line 76"


So I just went back to how I did it normally till I get to 64 bit.


On 64 bit AviSynth+ I can't use it without the lsb hack because when I use ConvertBits(bits=16) or ConvertBits(bits=32) with
Code:
ly = DebilinearM(1280,544,thr=01,lsb_inout=true, DeResizeMT=true)
lc = nnedi3_resize16(1280*2, 544*,lsb_in=true,lsb=true,kernel_d="Spline64",kernel_u="Spline64",src_top=0.0,src_left=0.50)
lu = lc.UtoY()
lv = lc.VtoY()
YtoUV(lu,lv,ly)
and HBD it tells me on avspmod

"lsb hack is not Compatible with native high bit depth
C:\ path\plugins64+\ResizeX_v1.0.1_mod9.35.avsi, line 79
C:\ path\plugins64+\DebilinearM_v1.3.1_mod1.25.avsi, line 87"


It works fine if I use the lsb hack dither_convert_8_to_16()
You should be how to do it like that tho? Remember? https://forum.doom9.org/showthread.p...23#post1879523

Same thing when I use Gradfun3mod with HBD

Code:
ly = GradFun3mod(thr=0.35, smode=2,thr_det=2,elast=4,mode=6, yuv444=true, resizer="DebilinearMTM", lsb_in=true, lsb=true)
lc = nnedi3_resize16(1280*2, 720*2,lsb_in=true,lsb=true,kernel_d="Spline64",kernel_u="Spline64",src_top=0.0,src_left=0.50)
lu = lc.UtoY()
lv = lc.VtoY()
YtoUV(lu,lv,ly)
On avspmod it tells me:

"ConvertToYV411: 8 bit only
C:\ path\plugins64+\GradFun3mod 3.1.avsi, line 176"


Works fine if I use the lsb hack dither_convert_8_to_16() for this too.
You showed me to do it this way also, https://forum.doom9.org/showthread.p...09#post1879909

Can you show me how to do it properly cuz I don't get it as you know I'm still new to this HBD thing as I am so use to using the lsb hack. If you know how to could you edit those posts above to include the HBD please leave lsb hack too.

On 32 bit AviSynth+ version I can't get around the thing it says that it doesn't know what AvsPlusNumber is. So until I get that RAM I can't use your scripts till I switch to 64 bit version and buy another 8 GB RAM.

Or maybe I'm missing something again?

Last edited by dREV; 6th September 2019 at 18:35. Reason: stuff
dREV is offline   Reply With Quote
Old 6th September 2019, 19:38   #312  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
Quote:
Originally Posted by dREV View Post
from https://forum.doom9.org/showthread.php?p=1884083

I haven't switched to 64 bit AviSynth+ because when I did my testing RAM went up over 90% on my single 8 GB RAM, so I planning to buy another 8 GB RAM as I use the same computer to do stuff while I wait for encode to finish.
that odd, it should not use that much

Quote:
Originally Posted by dREV View Post
Also, when I tried to use it on 32 bit AviSynth+ it would say this on avspmod:

"I don't know what 'AvsPlusVersionNumber' means
C:\ path\plugins\ResizeX_v1.0.1_mod 9.avsi, line 70
C:\ path\plugins\DebilinearM_v1.3.1_mod_1.25.avsi, line 76"


So I just went back to how I did it normally till I get to 64 bit.
if you google that message you will know why, anyway you need SMDegrain.avsi in your plugins folder

Quote:
Originally Posted by dREV View Post
On 64 bit AviSynth+ I can't use it without the lsb hack because when I use ConvertBits(bits=16) or ConvertBits(bits=32) with
Code:
ly = DebilinearM(1280,544,thr=01,lsb_inout=true, DeResizeMT=true)
lc = nnedi3_resize16(1280*2, 544*,lsb_in=true,lsb=true,kernel_d="Spline64",kernel_u="Spline64",src_top=0.0,src_left=0.50)
lu = lc.UtoY()
lv = lc.VtoY()
YtoUV(lu,lv,ly)
and HBD it tells me on avspmod

"lsb hack is not Compatible with native high bit depth
C:\ path\plugins64+\ResizeX_v1.0.1_mod9.35.avsi, line 79
C:\ path\plugins64+\DebilinearM_v1.3.1_mod1.25.avsi, line 87"


It works fine if I use the lsb hack dither_convert_8_to_16()
You should be how to do it like that tho? Remember? https://forum.doom9.org/showthread.p...23#post1879523

Same thing when I use Gradfun3mod with HBD

Code:
ly = GradFun3mod(thr=0.35, smode=2,thr_det=2,elast=4,mode=6, yuv444=true, resizer="DebilinearMTM", lsb_in=true, lsb=true)
lc = nnedi3_resize16(1280*2, 720*2,lsb_in=true,lsb=true,kernel_d="Spline64",kernel_u="Spline64",src_top=0.0,src_left=0.50)
lu = lc.UtoY()
lv = lc.VtoY()
YtoUV(lu,lv,ly)
On avspmod it tells me:

"ConvertToYV411: 8 bit only
C:\ path\plugins64+\GradFun3mod 3.1.avsi, line 176"


Works fine if I use the lsb hack dither_convert_8_to_16() for this too.
You showed me to do it this way also, https://forum.doom9.org/showthread.p...09#post1879909

Can you show me how to do it properly cuz I don't get it as you know I'm still new to this HBD thing as I am so use to using the lsb hack. If you know how to could you edit those posts above to include the HBD please leave lsb hack too.

On 32 bit AviSynth+ version I can't get around the thing it says that it doesn't know what AvsPlusNumber is. So until I get that RAM I can't use your scripts till I switch to 64 bit version and buy another 8 GB RAM.

Or maybe I'm missing something again?
Of course lsb not work with HBD, you must find updated scripts or alternatives to use it with HBD, like this

Code:
ConvertBits(bits=16)
ly = DebilinearM(1280,544,thr=01, DeResizeMT=true)
lc = edi_rpow2(1280*2, 544*2,cshift="Spline64",src_top=0.0,src_left=0.50)
lu = lc.ExtractU()
lv = lc.ExtractV()
YtoUV(lu,lv,ly)
if you didn't find updated scripts or alternatives then you can use ConvertToStacked/ConvertFromStacked see here http://avisynth.nl/index.php/ConvertStacked

Code:
ConvertBits(bits=16) #if it not 16 HBD already
ly = ConvertToStacked().GradFun3mod(thr=0.35, smode=2,thr_det=2,elast=4,mode=6, yuv444=true, resizer="DebilinearMTM", lsb_in=true, lsb=true).ConvertFromStacked()
lc = edi_rpow2(1280*2, 720*2,cshift="Spline64",src_top=0.0,src_left=0.50)
lu = lc.ExtractU()
lv = lc.ExtractV()
YtoUV(lu,lv,ly)
__________________
See My Avisynth Stuff

Last edited by real.finder; 6th September 2019 at 20:45.
real.finder is offline   Reply With Quote
Old 7th September 2019, 00:08   #313  |  Link
Natty
Noob
 
Join Date: Mar 2017
Posts: 221
nnedi3 resize16 and gradfun dont support HBD i think. you can try like this

Dither_convert_8_to_16() or ConvertBits(16).ConvertToStacked()
Nnedi3_resize16(lsb_in=true, lsb=true)
GradFun3mod(lsb_in=true, lsb=true)
DitherPost(mode=0) or ConvertFromStacked().ConvertBits(8 or 10, Dither=0)
Natty is offline   Reply With Quote
Old 9th September 2019, 18:00   #314  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
days ago I start read how Nnedi3_resize16 work to either make it work with HBD or add its features to edi_rpow2, and find it use Dither_limit_dif16, first time I didnt think of it can be replaced with something else, and there are SmoothAdjust but seems it only for tvrange vs full range things, then I start study Dither_limit_dif16 and find it work almost same as my sBlackerPixel and swhiterPixel, so I use them as a base to simulate Dither_limit_dif16 and it seems work!

Code:
# base on Dither_limit_dif16 idea (not 100% same output)
function slimit_dif(clip "flt", clip "src", clip "ref", float "thr", float "elast", int "Y", int "U", int "V", bool "elastinfl")
{
thr    = default(thr,                 0.25)
elast  = default(elast,                3.0)
elast2  = elast > 1.0 ? elast * thr : elast
Y  = default(Y,           3)
U  = default(U,           3)
V  = default(V,           3)
elastinfl = default(elastinfl, false)
 
Assert(!(elast < 1), "slimit_dif: elast must be more than 1")
Assert((defined(flt) && defined(src)), "slimit_dif: both flt clip and src clip are needed")
 
ldmask = elast!=1 ? VersionNumber() < 2.6 ? mt_lutxy(defined(ref) ? ref : src,flt,"x y - abs "+string(thr)+" <= 255 x y - abs "+string(elast2)+" <= ? 255 255 "+string(elast2)+" / x y - abs * - 0 ?", Y=Y,U=U,V=V)
                    \                     : mt_lutxy(defined(ref) ? ref : src,flt,"x y - abs "+string(thr)+" scalef <= range_max x y - abs "+string(elast2)+" scalef <= ? range_max range_max "+string(elast2)+" scalef / x y - abs * - 0 ?", Y=Y,U=U,V=V,use_expr=2) : nop()

elast!=1 ? VersionNumber() < 2.6 ? smaskmerge(src,flt, elastinfl ? ldmask.mt_inflate(155) : ldmask, Y=Y,U=U,V=V) : Mt_Merge(src,flt, elastinfl ? ldmask.mt_inflate(155) : ldmask, Y=Y,U=U,V=V) : \
           VersionNumber() < 2.6 ? defined(ref) ? ref.mt_lutxyz(flt,src,"x y - abs "+string(thr)+" <= y z ?", Y=Y,U=U,V=V) : src.mt_lutxy(flt,"x y - abs "+string(thr)+" <= y x ?", Y=Y,U=U,V=V)
           \                     : defined(ref) ? ref.mt_lutxyz(flt,src,"x y - abs "+string(thr)+" scalef <= y z ?",use_expr=2, Y=Y,U=U,V=V) : src.mt_lutxy(flt,"x y - abs "+string(thr)+" scalef <= y x ?",use_expr=2, Y=Y,U=U,V=V)
}
with elast=1 both are 100% same

but as it process choma, avoid use it with float clip if you process choma for now, I will add it in Advanced Denoising.avsi soon so any update will be in Advanced Denoising.avsi not the code above
__________________
See My Avisynth Stuff

Last edited by real.finder; 11th September 2019 at 02:37.
real.finder is offline   Reply With Quote
Old 12th September 2019, 10:50   #315  |  Link
dREV
Registered User
 
dREV's Avatar
 
Join Date: Jan 2019
Location: Antarctica
Posts: 74
posted another one https://forum.doom9.org/showthread.p...73#post1884673...

I've put in SMDegrain.avsi and trying out
Code:
ConvertBits(bits=16)
ly = DebilinearM(1280,544,thr=01, DeResizeMT=true)
lc = edi_rpow2(1280*2, 544*2,cshift="Spline64",src_top=0.0,src_left=0.50)
lu = lc.ExtractU()
lv = lc.ExtractV()
YtoUV(lu,lv,ly)
Got avspmod to say:
Script error: edi_pow2 does not have a named argument "src_top"

So I went and edited the edi out to, edi_rpow2(1280*2, 544*2,cshift="Spline64").

Now gives a new error
edi_rpow2: rfactorX must be a power of 2
(C:\path\plugins\edi_rpow2+ v1.0 mod 7.avsi, line 205)


......

So tried the other one you put using

Code:
ConvertBits(bits=16) #if it not 16 HBD already
ly = ConvertToStacked().GradFun3mod(thr=0.35, smode=2,thr_det=2,elast=4,mode=6, yuv444=true, resizer="DebilinearMTM", lsb_in=true, lsb=true).ConvertFromStacked()
lc = edi_rpow2(1280*2, 720*2,cshift="Spline64",src_top=0.0,src_left=0.50)
lu = lc.ExtractU()
lv = lc.ExtractV()
YtoUV(lu,lv,ly)
Now it complains about this:

Gradfun3mod: wrong resizer value.
(C:\path\plugins\GradFunmod.avsi, line 187)


I've got all your scripts
Import("C:\path\plugins\SMDegrain+ v.3.1.2.104s.avsi")
Import("C:\path\plugins\edi_rpow2+ v1.0 mod 7.avsi")
Import("C:\path\plugins\ResizeX+_v1.0.1_mod 9.35.avsi")


Also, not sure if I like having to give up my friend, nnedi3_resize16() yet.
dREV is offline   Reply With Quote
Old 12th September 2019, 16:26   #316  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
sorry, I didn't note that edi_pow2 does not have src_top, so I will try add it in edi_pow2 or port the nnedi3_resize16 to support HBD

and it need to be like this edi_rpow2(fwidth=1280*2, fheight=544*2,cshift="Spline64") to work but will not give right output in your case since there are no src_top

and about "Gradfun3mod: wrong resizer value." are you sure you use the MT edit I did of Gradfun3mod?
__________________
See My Avisynth Stuff
real.finder is offline   Reply With Quote
Old 12th September 2019, 22:48   #317  |  Link
dREV
Registered User
 
dREV's Avatar
 
Join Date: Jan 2019
Location: Antarctica
Posts: 74
Sorry I forgot to add your modded GradFun3mod, I didn't see it on your Avisynth script page, so had to look around but found them again, https://forum.doom9.org/showthread.p...09#post1879909.

Got errors though

Code:
ConvertBits(bits=16) #if it not 16 HBD already
ly = ConvertToStacked().GradFun3mod(thr=0.35, smode=2,thr_det=2,elast=4,mode=6, yuv444=true, resizer="DebilinearMTM", lsb_in=true, lsb=true).ConvertFromStacked()
lc = edi_rpow2(1280*2, 720*2,cshift="Spline64")
lu = lc.ExtractU()
lv = lc.ExtractV()
YtoUV(lu,lv,ly)
edi_rpow2:rfactorX must be a power of 2
(C:\path\plugins\edi_rpow2+ v1.0 mod 7.avsi, line 205)


Then I put in the other one you told me to write instead but got errors too

Code:
ConvertBits(bits=16) #if it not 16 HBD already
ly = ConvertToStacked().GradFun3mod(thr=0.35, smode=2,thr_det=2,elast=4,mode=6, yuv444=true, resizer="DebilinearMTM", lsb_in=true, lsb=true).ConvertFromStacked()
lc = edi_rpow2(fwidth=1280*2, fheight=544*2,cshift="Spline64")
lu = lc.ExtractU()
lv = lc.ExtractV()
YtoUV(lu,lv,ly)
YToUV: Video height ratio does not match any internal colorspace.

But I forgot had to change edi_rpow2(fwidth=1280*2, fheight=544*2,cshift="Spline64") to edi_rpow2(fwidth=1280*2, fheight=720*2,cshift="Spline64") and it worked because of how GradFun3mod is strict as it forces to use only 1280x720 resolution and won't allow any another resolution. So for sources where I have to crop (like the one I been having trouble on) I need to use DebilinearM or something else.

When I use some functions of edi_rpow2 to use the functions of nedi3_resize16() specifically nsize=6,pscrn=4,qual=2,etype=0 seems the screenshots I take became smaller instead of be the same or increase. Odd but I found the culprit to be pscn= when using "3" or "4" but using "1" and "2" do fine.

The nnedi3 documentation
Quote:
http://avisynth.nl/index.php/Nnedi3/nnedi3_rpow2
0 - no prescreening (same as false in prior versions)
1 - original prescreener (same as true in prior versions)
2 - new prescreener level 0
3 - new prescreener level 1
4 - new prescreener level 2

Higher levels for the new prescreener result in cubic interpolation being used on fewer pixels (so are slower, but incur less error). However, the difference is pretty much unnoticeable.
Level 2 is closest to the original prescreener in terms of incurred error, but is much faster.
I wanted to use "4" but I this is alright

--------------------------------------------------------------------------------

I'm playing around with the HBD with your modded Gradfun3mod and having major issues with implementing certain functions such as
f3kdb(Y=65,Cb=65,Cr=65,grainY=65,grainC=65,mt=true,input_depth=16,output_depth=16)
Dither_add_grain16()


And I'm certain other filters that I need after resizing but basically majority of my scripts is centered around the lsb hack. It's slow trying to transition to HBD. I think I read that these filters can't use HBD and I also use
s16 = last
s16.Dither_limit_dif16 (last, thr=, elast=)

before I resize and cannot figure out how to implement this or even if I can do it in the first place with HBD.

How can i replace lsb with hbd on clean sources

Code:
#sometimes use 8 bit filter(s) here
dither_convert_8_to_16()
s16 = last
DitherPost()

Turn. #sometimes line darkening filter
#anti-aliasing filter.Turn

dither_convert_8_to_16()
s16.Dither_limit_dif16 (last, thr=0.25, elast=4.0)
SmoothGrad()

# resizer 4:4:4 DebilinearM or GradFun3mod
# grain filter
# if not using gradfun3mod resizer I use banding filter here
Dither_out()
What I'm looking for is the HBD equivalent of opening 8 bit to 16 bit and vice-versa like with the lsb hack's dither_convert_8_to_16(), DitherPost(), and Dither_out(). As whenever I use HBD with the resizer above my screen is green or is messed up when I'm trying to use the rest of my filters like Dither_add_grain16() grain filter which I need. How can I do this? I've tried what Natty posted but didn't work.


Thanks for your time in helping me out and trying to work out the other stuff. I also noticed how fast the resizer is with HBD and took less RAM on [iavspmod[/i] really interested if can get this to work completely.

Can you help me out on the AddBorders() question too if you don't mind? Have issues with H264 but it works fine with HEVC. https://forum.doom9.org/showthread.p...73#post1884673

Last edited by dREV; 13th September 2019 at 00:39. Reason: forgot important details
dREV is offline   Reply With Quote
Old 13th September 2019, 01:04   #318  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
the edit I did for Gradfun3mod not for HBD, because it use "dither tools" or better call it "lsb or Stacked tools" and it not work with HBD ofc, so you can't mix "dither tools" things with HBD or bad things happen unless you use http://avisynth.nl/index.php/ConvertStacked

about x264, you must ask someone else, anyway try put ConvertFromDoubleWidth() after Dither_out() and it will output native (HBD) 16 bit and remove --input-depth 16 (like you encode 8 bit clip)
__________________
See My Avisynth Stuff
real.finder is offline   Reply With Quote
Old 13th September 2019, 02:31   #319  |  Link
dREV
Registered User
 
dREV's Avatar
 
Join Date: Jan 2019
Location: Antarctica
Posts: 74
Quote:
Originally Posted by real.finder View Post
the edit I did for Gradfun3mod not for HBD, because it use "dither tools" or better call it "lsb or Stacked tools" and it not work with HBD ofc, so you can't mix "dither tools" things with HBD or bad things happen unless you use http://avisynth.nl/index.php/ConvertStacked

about x264, you must ask someone else, anyway try put ConvertFromDoubleWidth() after Dither_out() and it will output native (HBD) 16 bit and remove --input-depth 16 (like you encode 8 bit clip)
Alright, I may just do it the easy way or something else on the AddBorders() thing with h264.


Didn't you say you can't mix dither tools with HBD? Dither_out() is part of the dither tools. Well I tried what you said

Code:
ConvertBits(bits=16) #if it not 16 HBD already
ly = ConvertToStacked().GradFun3mod(thr=0.35, smode=2,thr_det=2,elast=4,mode=6, yuv444=true, resizer="DebilinearMTM", lsb_in=true, lsb=true).ConvertFromStacked()
lc = edi_rpow2(fwidth=1280*2, fheight=720*2,cshift="Spline64")
lu = lc.ExtractU()
lv = lc.ExtractV()
YtoUV(lu,lv,ly)
Dither_out().ConvertFromDoubleWidth()
error (I had a feeling this wasn't gonna work)
ConvertFromDoubleWidth: Input double width clip must be YV12, YV16, YV24, Y8, RGB24 or RGB32

I've tried manipulating around that and ConvertToStacked and I just can't figure it out.... I'm about to give up on this HBD thing. There's not enough information to the documentation nor examples for me to customize it like I did for lsb hack that came with the dither tools. If I can't get around this HBD to use other filters there's really no point. Sorry for wasting your time with me on that.

---------------------------------------------------------------------------------------------------

Also, I found something odd. I have some screenshot sample results of the same frame.

With DebilinearM:
DebilinearM (original script) 709 KB
DebilinearM (your version) 764 KB

With Gradfun3mod() I get:
Gradfunmod3 (original script) 794 KB
Gradfunmod3 (your version) 761 KB

Changed both the original and your version's thr from "01" instead of default "10" in the DebilinearM.avsi script as it gives too much sharpness creating halos. They also are using the lsb version not the HBD. I've manipulated scripts around and eliminating what may be the reason why it gives low output and it seems to be your modded version of nnedi3_resize16_v3.3. Everything else was the same file size except when I used that script it would go down and that one only.

Latest screenshots results were using GradFun3mod

original 781 kb
your version 755 kb

In addition when I use GradFun3mod(thr=0.35, smode=2,thr_det=2,elast=4,mode=6, yuv444=true, resizer="DebilinearM", lsb_in=true, lsb=true) file size is high (781 kb) but when I switch to GradFun3mod(thr=0.35, smode=2,thr_det=2,elast=4,mode=6, yuv444=true, resizer="DebilinearMTM", lsb_in=true, lsb=true) it goes down (to 755 kb) both using the original nnedi3_resize16_v3.3 script. However, even when I use your modded nnedi3 version together with both functions of Gradfun3mod they still give results 755 kb. Only when returning to the original nnedi3 script does it return to normal as well as using the original GradFun3mod function.

All of this was using your updated scripts from ResizeX v1.0.1 mod 9.35, edi_rpow2 v1.0 mod 7, Gradfun3mod r6 mod 3.1, DebilinearM v1.3.1 mod 1.25, etc

Seems the original Gradfun3mod output has increased detail moreso then with both the original and your version of DebilinearM also. What you think?

Last edited by dREV; 13th September 2019 at 03:43. Reason: typos
dREV is offline   Reply With Quote
Old 13th September 2019, 05:41   #320  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
I think you need to know the basic of what lsb (aka dither tools things aka Stack16) and what is HBD in avs+, why don't you read the wiki link I post many many times?!!! they can't be mixed UNLESS with http://avisynth.nl/index.php/ConvertStacked (the link you don't read or don't understand its content every time) and btw you already mixed both (even without ConvertFromDoubleWidth by using Dither_out with HBD input so the output will be 100% wrong indeed) but in wrong way

do not expect getting a result identity by 100% matching if the script or the used plugins or even the avs changed/updated, and that normal since the edit I did for GradFun3mod and DebilinearM will use yet another plugins away from original Debilinear.dll

and my edit of nnedi3_resize16 in avs+ even with lsb it will use some HBD features to get real 16bit as possible since nnedi3_resize16 with lsb don't do real 16 bit proccess in first place since nnedi3 don't has lsb support, so yes without set the nlsb in nnedi3_resize16 to false in avs+ you will get a different result and it should be a better
__________________
See My Avisynth Stuff

Last edited by real.finder; 13th September 2019 at 06:31.
real.finder is offline   Reply With Quote
Reply

Tags
avisynth, avs script, avs+, banding, generator, hbd, mod

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 14:18.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.