Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 19th September 2016, 00:11   #1  |  Link
OvejaNegra
ekTOMBE STUDIOS
 
OvejaNegra's Avatar
 
Join Date: Dec 2005
Location: Cuba
Posts: 254
help improving and scaling this anime (DVD R1 Source)

Im trying to upscale this anime to 1440 X 1080 using

Code:
nnedi3_rpow2(rfactor=2, nsize=0, nns=3, qual=1, etype=0, pscrn=1, threads=0, opt=0, fapprox=15,cshift="spline36resize",fwidth=1440,fheight=1080)
Results for now are great (suggestions welcomed) but first i want to clean a little bit.

The source has some bright halos on the RIGHT of the lines, removing them using dehalo / finedehalo gives me acceptable resuls, but no good. That without (off course) killing the lines or creating blurring.

Halos look like this (post-scale result, halos processed on original resolution- green rectangles):


After dehalo, this is the result (the best i could do)



I want to apply proToon to darken the lines, but i have a problem: Using protoon AFTER dehalo gives me some broken lines. Using protoon BEFORE dehalo gives me more clean lines, but affects dehalo results.
Is possible to "create" the protoon mask before the dehalo pass and apply it later (i don't know if i'm making the right question)?

This is the best i could do without both filters opposing each other:



Another question: The chroma has a lot of noise, i was trying to saturate a little bit but it looks horrible and noisy. Also, it adds chroma noise to dark / white areas: Is possible to create a mask with the lines, the black areas and the white areas to "protect them" during saturation"? (i know the example picture is over saturated, is intentional to show the noise).



Making it look so clean like a flash animation or a modern anime is too much but i want to try. Thanks
__________________
So, it works or not???
OvejaNegra is offline   Reply With Quote
Old 19th September 2016, 07:08   #2  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
DeHaloHmod
__________________
See My Avisynth Stuff
real.finder is offline   Reply With Quote
Old 21st September 2016, 18:28   #3  |  Link
brucethemoose
Registered User
 
Join Date: Sep 2016
Posts: 67
Would long encoding times be an issue?
brucethemoose is offline   Reply With Quote
Old 23rd September 2016, 05:08   #4  |  Link
OvejaNegra
ekTOMBE STUDIOS
 
OvejaNegra's Avatar
 
Join Date: Dec 2005
Location: Cuba
Posts: 254
Sorry for delay. No encoding times is not issue.
In the end FineDehalo gave me acceptable results and now im working with protoon.
Also im testing Satmask for saturating the dull parts of the image (like Vibrance).
I need something like that for the dark / white zones -> Making the dark zones really dark and some overbright zones, not that bright.
Also, the white zones has some blue tint, so maybe white point correction will be necesary.
Advices are always welcome.
__________________
So, it works or not???
OvejaNegra is offline   Reply With Quote
Old 23rd September 2016, 09:06   #5  |  Link
SSH4
Registered User
 
Join Date: Nov 2006
Posts: 90
First rules of up-scaling anime - never use R1 sources. Use only R2 Japan sources that have 2-3x times bigger bit-rate than 99,99% of R1 sh*t.
SSH4 is offline   Reply With Quote
Old 23rd September 2016, 15:35   #6  |  Link
brucethemoose
Registered User
 
Join Date: Sep 2016
Posts: 67
Quote:
Originally Posted by OvejaNegra View Post
Sorry for delay. No encoding times is not issue.
In the end FineDehalo gave me acceptable results and now im working with protoon.
Also im testing Satmask for saturating the dull parts of the image (like Vibrance).
I need something like that for the dark / white zones -> Making the dark zones really dark and some overbright zones, not that bright.
Also, the white zones has some blue tint, so maybe white point correction will be necesary.
Advices are always welcome.
If you really mean that (I'm talking ~2 FPS or worse), Waifu and Vine are pretty amazing:

http://www.screenshotcomparison.com/comparison/185123

(Pupils are messed up in that screenshot due to too much dehaloing, but you get the idea)

I had less luck with FineDehalo, but my halo problems were much more severe.

Last edited by brucethemoose; 23rd September 2016 at 15:39.
brucethemoose is offline   Reply With Quote
Old 30th September 2016, 05:19   #7  |  Link
OvejaNegra
ekTOMBE STUDIOS
 
OvejaNegra's Avatar
 
Join Date: Dec 2005
Location: Cuba
Posts: 254
sorry for delay.
I compared waifu with my setup, and i prefer mine, waifu looks.... wierd
What's vine? Can't find anything about it

I know about R1 VS R2 DVDs, but R1 i what i have
__________________
So, it works or not???
OvejaNegra is offline   Reply With Quote
Old 1st October 2016, 07:09   #8  |  Link
brucethemoose
Registered User
 
Join Date: Sep 2016
Posts: 67
Quote:
Originally Posted by OvejaNegra View Post
sorry for delay.
I compared waifu with my setup, and i prefer mine, waifu looks.... wierd
What's vine? Can't find anything about it

I know about R1 VS R2 DVDs, but R1 i what i have
A very, very slow VapourSynth dehaloer. I'm in the wrong forum section for that :P

Be sure to turn down denoising to 0 if you try Waifu again, or just turn it off with -1. Anything higher just messes it up, even in my source.



Anyway, my problem with dehaloing is that it blurs/breaks some dark lines. Since halos are usually white, my plan is to mask out the filter in areas below a darkness threshold, to save the black lines... maybe you could try something similar.

Last edited by brucethemoose; 1st October 2016 at 07:11.
brucethemoose is offline   Reply With Quote
Old 1st October 2016, 14:08   #9  |  Link
OvejaNegra
ekTOMBE STUDIOS
 
OvejaNegra's Avatar
 
Join Date: Dec 2005
Location: Cuba
Posts: 254
i have the same problem, i can make the halos dissapear, at the cost of blurring dark lines.
That was why i was thinking on creating a mask with the dark lines before the dehalo filters, and using that mask after to "paste" the dark lines over the filtered clip. But i'm new with the mask filters
so im trying.
__________________
So, it works or not???
OvejaNegra is offline   Reply With Quote
Old 1st October 2016, 19:09   #10  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
Quote:
Originally Posted by brucethemoose View Post
A very, very slow VapourSynth dehaloer. I'm in the wrong forum section for that :P

Be sure to turn down denoising to 0 if you try Waifu again, or just turn it off with -1. Anything higher just messes it up, even in my source.



Anyway, my problem with dehaloing is that it blurs/breaks some dark lines. Since halos are usually white, my plan is to mask out the filter in areas below a darkness threshold, to save the black lines... maybe you could try something similar.
I already did this, but the Thread writer seems ignore my post

http://avisynth.nl/index.php/DeHaloHmod < but don't have all Parameters and updates
__________________
See My Avisynth Stuff
real.finder is offline   Reply With Quote
Old 1st October 2016, 19:20   #11  |  Link
brucethemoose
Registered User
 
Join Date: Sep 2016
Posts: 67
Quote:
Originally Posted by real.finder View Post
I already did this, but the Thread writer seems ignore my post

http://avisynth.nl/index.php/DeHaloHmod < but don't have all Parameters and updates
Thanks. I'll try it too if I can import it to VS.
brucethemoose is offline   Reply With Quote
Old 2nd October 2016, 08:13   #12  |  Link
OvejaNegra
ekTOMBE STUDIOS
 
OvejaNegra's Avatar
 
Join Date: Dec 2005
Location: Cuba
Posts: 254
Quote:
I already did this, but the Thread writer seems ignore my post
Slow down please, yes i tried your filter but i was not able to remove the halos with it, maybe some advice from you will help.

Code:
DeHaloHmod(Radius=8, Str=16, Maska=false, strong=true, mode=2, thr=32, analog=false, dirty=false, smooth=false, anime=true, pel2=false, usedeen=false, maskpel2=false, thr2=32,exdehalo="FineDehalo(rx=2, ry=1, thmi=5, thma=128, thlimi=10, thlima=250, darkstr=1.0, brightstr=2.0, showmask=0, contra=0.0, excl=true, edgeproc=0.0)")
Those gave me the best results, and yes i tried with and without and external dehaloer.

Also, my goal is a little different: Make a mask from the lines, do several stages of filtering (not just dehalos) and "paste" the black lines after all that:
For example: Playing with sat mask some times make the lines around saturated objects dark red or dark blue. I suposse this is for the lower resolution chroma, but im not going to stretch this to 4:4:4

Thanks
__________________
So, it works or not???
OvejaNegra is offline   Reply With Quote
Old 2nd October 2016, 09:25   #13  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
Quote:
Originally Posted by OvejaNegra View Post
Slow down please, yes i tried your filter but i was not able to remove the halos with it, maybe some advice from you will help.

Code:
DeHaloHmod(Radius=8, Str=16, Maska=false, strong=true, mode=2, thr=32, analog=false, dirty=false, smooth=false, anime=true, pel2=false, usedeen=false, maskpel2=false, thr2=32,exdehalo="FineDehalo(rx=2, ry=1, thmi=5, thma=128, thlimi=10, thlima=250, darkstr=1.0, brightstr=2.0, showmask=0, contra=0.0, excl=true, edgeproc=0.0)")
Those gave me the best results, and yes i tried with and without and external dehaloer.

Also, my goal is a little different: Make a mask from the lines, do several stages of filtering (not just dehalos) and "paste" the black lines after all that:
For example: Playing with sat mask some times make the lines around saturated objects dark red or dark blue. I suposse this is for the lower resolution chroma, but im not going to stretch this to 4:4:4

Thanks
for the only mask EMask_dhh()

only mode=3 do mask for black lines, others just normal edges mask and with anime=true they maybe make the halo past from the source

and seems you dehalo after upscale, it's better to do it before that, and why you didn't post video sample?

edit: with simple code using your upscale pic

Code:
ImageSource("e84ll4_jpg.png")
Spline36Resize(720,480)
ConvertToYV12()
DeHaloHmod(smooth=true)
nnedi3_rpow2(rfactor=2, nsize=0, nns=3, qual=1, etype=0, pscrn=1, threads=0, opt=0, fapprox=15,cshift="spline36resize",fwidth=1440,fheight=1080)


and of course you can use your dehalo in exdehalo
__________________
See My Avisynth Stuff

Last edited by real.finder; 2nd October 2016 at 10:34.
real.finder is offline   Reply With Quote
Reply

Tags
anime, halos, improve, restore

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:38.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.