Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 10th November 2020, 19:11   #1  |  Link
TCmullet
Registered User
 
Join Date: Nov 2003
Posts: 344
Filters to cartoonize and decolor and "undetail"

Is there an Avisynth filter, free or cheap, that can take a live action video and turn it into an animated cartoon? That is, turn it into a line drawing of all the people and objects and fill in simple colors for everything? Also, converting full or simple colors to greyscales would be desirable.

Second, might there be a filter, dependent or independent of the above filter, that could create a black and white line drawing only of the video? Perhaps you're thinking I mean simply converting to greyscale. No, I include the idea of somehow allowing the level of details to be adjustable so that at the highest end of retention, you'd see full facial expressions (for example), but at the lowest end, you'd see nothing but outlines of all people and objects.

Are these too tall an order for what Avisynth can do?
TCmullet is offline   Reply With Quote
Old 10th November 2020, 19:29   #2  |  Link
butterw2
Registered User
 
Join Date: Jun 2020
Posts: 203
No idea what's natively available for Avisynth.

Some realtime gpu video shaders are available for mpc-hc.
https://forum.doom9.org/showthread.php?t=181584&page=3
- reducing color palette (posterization)
- converting to greyscale
- edge detection
They could theoretically be used in avisynth via AvisynthShader.

VLC dx11 also has a cartoon filter that you can encode with.
butterw2 is offline   Reply With Quote
Old 10th November 2020, 22:07   #3  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,439
I haven't used it for years, but there is this for VirtualDub:

http://compression.ru/video/cartoonizer/index_en.html
johnmeyer is offline   Reply With Quote
Old 11th November 2020, 12:53   #4  |  Link
butterw2
Registered User
 
Join Date: Jun 2020
Posts: 203
Based on the featured sample images, the virtualdub cartoonizer seems quite hit and miss. It should be possible to do better.
If it is possible to do a cartoon transform in photoshop, it should be possible to get a similar result in avisynth with a video scene.
Your results will however depend on the input scene and parameter values. A complex algorithm is likely required to get a temporaly stable, near pixel perfect result.

The realtime vlc cartoon filter isn't bad (Tools > filters > video effects > colors > Gradient +Edge + Cartoon). The source code is in gradient.c, so I guess it could be ported to avisynth.

More or less detail is built-in to all edge detection methods (ex: Sobel). Output is thresholded.
butterw2 is offline   Reply With Quote
Old 20th November 2020, 11:26   #5  |  Link
butterw2
Registered User
 
Join Date: Jun 2020
Posts: 203
The VLC buit-in filter uses a simple Y' (Luma) method (edge + colorize). The effect runs easily (up to 1080p input) in realtime on a single cpu core, the results are best on well lit hd video, but as can be seen on the 720p input screenshot the results are not perfect. Better results are possible with clean high bitrate 4K input.



The simple colorize method compresses smoothed Luma in 4 fixed bands. The edges (in black) are produced by a sobel filter thinned with angle processing.
Grayscale input is also handled. No tuning parameters are made available.

https://code.videolan.org/videolan/v...ter/gradient.c

I may port this to a mpv gpu pixel shader. I also have other realtime shaders for toon/comic-book stylised effects. Does anyone have suggestions on how to achieve higher quality results ?

- Luma methods are sensitive to lighting (changes). Avoid fluctuating light sources.
- Shadows and compression artifacts in dark areas also tend to show up (and fluctuate).
- A (removable) frame border artifact is present.
- Results could probably be improved by adding dithering to limit visible banding.

Last edited by butterw2; 20th November 2020 at 19:06. Reason: Tried to Force image to display at original size, and failed...
butterw2 is offline   Reply With Quote
Old 20th November 2020, 15:14   #6  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 9,145
Judging by above sample image, maybe could benefit from a little bit of blurring following the cartoonizing to soften those edges,
[probably spatial and temporal blur, or maybe HqDn3D], but then I'm not really used to looking at cartoons.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 20th November 2020, 23:22   #7  |  Link
butterw2
Registered User
 
Join Date: Jun 2020
Posts: 203
Thanks, temporal methods aren't typically possible in video player shaders, but I'll give it a look in Avisynth.
Just resizing down the screenshot to 75% gives quite a nice result. Of course screenshots only tell part of the story...

Ideally, I would also like to be able to upscale from 720p to 1080p. Edge detection will perform better on the upscaled image, but it might also require some final smoothing.
---

Here's another lightweight shader bPosterize(levels=4) applied to the smoothed image, which performs a slightly improved color Posterization in rgb (helps avoid greens in skintones).



PS: Is there a way to scale image display size on the forum ?

Last edited by butterw2; 22nd November 2020 at 15:21.
butterw2 is offline   Reply With Quote
Old 21st November 2020, 01:00   #8  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,439
There is a completely different approach to this, one which gives you a LOT more flexibility than any of the "cartoonizer" plugins. I've used it myself, so I know it works.

What you do is export your video as an image sequence. Then, open one of those images in Photoshop (or your favorite image editor -- I use an ancient program called Photoimpact). Use the image editor's plugins to do the cartooning. Mine has such a plugin. You can use more than one plugin if you don't get quite the look you are going for using just one plugin.

Then, use the batch feature in your photo editor to apply that same sequence of changes to all the other photos that you exported. Almost any decent photo editor has a batch facility that will let you do this.

Finally, reassemble the images back into a video. You can do this with most NLEs. Virtualdub can also do it.
johnmeyer is offline   Reply With Quote
Old 21st November 2020, 01:31   #9  |  Link
butterw2
Registered User
 
Join Date: Jun 2020
Posts: 203
If such a Photoshop plugin that can do cartooning as desired (different art styles are possible), it means it could also be implemented in Avisynth. It seems there just hasn't been any great interest to do so, so far.
Considerations on source and output resolution, lighting, output palette, parameter adjustment and temporal stability would still be relevant though.

Simple comic book style shaders are quite popular for realtime video. I'm not sure how they compare to the best pro-plugins probably available as Premiere or similar add-ons.
butterw2 is offline   Reply With Quote
Old 21st November 2020, 12:18   #10  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 9,145
Anybody know how Polar Express was done ? [thnik that is the name of the Tom Hanks thing].
(Maybe I get it when we come out of Lockdown).
Maybe was hand painted [over live action], with some kind of auto copy for similar frames.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 21st November 2020, 16:58   #11  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,629
Kuwahara for avisynth (x86 only , but can run in mp_pipeline x64)
https://forum.doom9.org/showthread.php?t=170788
http://www.wilbertdijkhof.com/Kuwahara_v11.zip
poisondeathray is offline   Reply With Quote
Old 21st November 2020, 17:33   #12  |  Link
butterw2
Registered User
 
Join Date: Jun 2020
Posts: 203
bStipple comic book shader (mpv .hook and .hlsl):



This effect uses an increased contrast black and white image, adds blue noise stippling (which is useful to cover up artifacts) and then blends the result with the color image.

Works well with low-res input <=720p.
The code is very simple, except for the part relating to noise generation which wouldn't be needed with an external blue noise .png (the shader can be used to generate this texture, but higher quality implementations might be available).

Last edited by butterw2; 22nd November 2020 at 22:22. Reason: +details
butterw2 is offline   Reply With Quote
Old 22nd November 2020, 15:02   #13  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 9,145
butterw2, I like that effect, nifty
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 22nd November 2020, 15:40   #14  |  Link
butterw2
Registered User
 
Join Date: Jun 2020
Posts: 203
bHatch (.hlsl, .hook) shader maps an increased contrast input image (6 levels) to 4 color palette with 2 hatching levels.


EDIT: was (4 levels, 2 solid colors + hatching) https://raw.githubusercontent.com/bu...size_x1280.jpg .
Shader was used post-resize (1080p) and the result was resized down to 720p providing some smoothing in edges and palette.
It's probably possible to figure out a better combination of colors and hatching (reverse hatch \\ can be used). It would also be possible to combine selected colors, dots/hatching, edges from the different methods presented here.
Because this shader inherently causes some loss of detail, it works best on close-ups (ex: Tiktok videos).

For reference, the source frame (720p)

Last edited by butterw2; 23rd November 2020 at 18:53. Reason: updated result
butterw2 is offline   Reply With Quote
Old 24th November 2020, 13:03   #15  |  Link
butterw2
Registered User
 
Join Date: Jun 2020
Posts: 203
Pencil-sketch with Edge_Sharpen shader (edge threshold: 0.15, bg: grey200)


Different edge detection algorithm exists (with an ajustable edge threshold), some can be sensitive to source artifacts.

Frei-chen Edge detection (more detailed):
https://raw.githubusercontent.com/bu...x1280_grey.png
butterw2 is offline   Reply With Quote
Old 24th November 2020, 19:56   #16  |  Link
TCmullet
Registered User
 
Join Date: Nov 2003
Posts: 344
Gosh, thank you all for the contributions! I'm glad you all got more out of this than I did! It turns out that the project I hoped to tackle may not be possible, for reasons totally "political" of sorts. But if I did become able to do it, let me ask you further. Butterw2, would it be possible to virtually eliminate all or most details that are not part of an "edge"? Basically, the outlines of people and objects are what is wanted, and perhaps other edges like clothing necklines, etc. I want to see that this is a human and probably the gender (if obvious), but not their identity nor details of what things look like OTHER than their shapes via the edges. Almost like a silhouette of sort. (But not a black-filled silhouette.) Need to see what people are doing without any details of their own showing on them or the objects they are handling. This might be like a movie's storyboarding, I think. Maybe even more obfuscated than that. Basically, animated outline drawings.
TCmullet is offline   Reply With Quote
Old 24th November 2020, 22:45   #17  |  Link
butterw2
Registered User
 
Join Date: Jun 2020
Posts: 203
It's possible to threshold higher but then parts will go missing (the dark lines will remain). The outlines are clean with anime, but with real life video content, it's more messy.

EDIT: to keep only the thick dark lines, something like this would be required:
edge_detect(threshold: keep only the dark lines)
erode, median filter, dilate: eliminate thin lines

Last edited by butterw2; 25th November 2020 at 10:10.
butterw2 is offline   Reply With Quote
Old 24th November 2020, 23:20   #18  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,439
The process I outlined above can certainly do what you ask. Here is an example of one frame:

Original image


Image with just pen outline


I forgot to remove the last little bit of background, but with the image editing program it is trivial to do that.

Getting only edges from an image can be tricky. I chose this image because the background is fuzzed out and shows what happens when the edges become less distinct.

This can be automated. It took me less than a minute to do this, and you can process the images pretty quickly, depending on the resolution.
johnmeyer is offline   Reply With Quote
Old 25th November 2020, 20:30   #19  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,192
Cool thread, some weeks ago I was looking around for some hlsl filters that I could use in AviSynth, the selection is limited . BTW, thanks butterw2, I came across your filters .

Here are two examples of things I found.

Using AviSynthShader with the following hlsl filters:

Cartoon Shader by Andrew Price:


Gameboy shader by kchapelier:

Reel.Deel is offline   Reply With Quote
Old 26th November 2020, 17:21   #20  |  Link
butterw2
Registered User
 
Join Date: Jun 2020
Posts: 203
Dark Sobel_Edges + Saturated Color (cartoon enhanced reality):


bSobel_Edge.hlsl
- T_Sobel 0.15
- SatFactor 1.8
butterw2 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 01:08.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.