Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
26th February 2023, 03:43 | #1 | Link |
Registered User
Join Date: Mar 2012
Location: Texas
Posts: 1,672
|
ONNX models to use with avs-mlrt
PyTorch models converted to ONNX to use with AviSynth+'s avs-mlrt plugin. Models were converted with chaiNNer (fp32 data type). Let me know of missing models to convert and I'll add them to the package.
Download: onnx-models-rd Other repositories that include onnx models:
Model database Basic usage Code:
FFVideoSource("YUV420Source.mkv") ConvertBits(32) ConvertToPlanarRGB() mlrt_ncnn(network_path="1x_Dotzilla_Compact_80k_net_g.onnx", builtin=false) If you have a low end GPU and trying to use the larger models, it helps to use the tilesize, overlap and fp16 options. Code:
mlrt_ncnn(network_path=model, builtin=false, fp16=true, tilesize_w=width/4, tilesize_h=height/4, overlap_w=8, overlap_h=8) Model conversion Download chaiNNer and install the required dependencies. Load the corresponding nodes, connect them and then hit run by either pressing the play button at the top or F5. Last edited by Reel.Deel; 25th May 2023 at 21:57. Reason: add links and restructure |
26th February 2023, 03:57 | #2 | Link |
Registered User
Join Date: Nov 2009
Posts: 2,371
|
Woow, so great, ''thumbs up'', these are my go to for anime and manga. Still searching a good one for photos though.
Code:
Old Anime Upscale: 1x_Dotzilla_Compact_80k_net_g.pth (dot crawl and rainbowing) 2X_DigitalFilmV5_Lite.pth (sharpener for soft lines, no need to downscale before AI) 2x_AnimeClassics_UltraLite_510K.pth (less reconstruction than above, similar to Waifu but for Anime. Color Shift) 2x_LD-Anime_Skr_v1.0.pth (for ringing, rainbowing, aliasing) Manga Upscale (B/W): 4x_eula_digimanga_bw_v1_860k.pth Magazines? 4x-UniScaleV2_Moderate
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread |
26th February 2023, 04:03 | #4 | Link | |
Registered User
Join Date: Jan 2018
Posts: 2,168
|
Quote:
|
|
26th February 2023, 05:06 | #5 | Link | |
Registered User
Join Date: Mar 2012
Location: Texas
Posts: 1,672
|
Quote:
Edit: if anyone has the "DeGif" model please share it. The link to it is not working right now (Public Transfer Limit Exceeded). Last edited by Reel.Deel; 27th February 2023 at 08:45. Reason: Delete link, see 1st post |
|
26th February 2023, 05:17 | #6 | Link |
Registered User
Join Date: Sep 2007
Posts: 5,598
|
Usually the models are quite narrow in scope; typically they are only useful on the data set that they were trained on, or very similar material. But you can try it...
Which version ? There are older checkpoint versions and the most recent version I have is "2x_NMKD-DeGIF_210000_G.pth" |
26th February 2023, 05:22 | #8 | Link |
Registered User
Join Date: Sep 2007
Posts: 5,598
|
I mention it because sometimes an earlier checkpoint produces better results. Here are all the checkpoints for DeGif that I have. Credits to NMKD (n00mkrad)
https://www.mediafire.com/file/2jzoq...+DeGIF.7z/file Just curious : 1) how long does it take to convert ? 2) are results bit identical to the pytorch version? |
26th February 2023, 05:33 | #9 | Link | |
Registered User
Join Date: Mar 2012
Location: Texas
Posts: 1,672
|
Quote:
1) Depends on the size of the model. Small ones are quick (few seconds), the largest one I converted (64mb) took around a minute. 2) I have not compared them to the original version. It would be interesting to do so. But while researching how to convert between the two formats I only saw that mentioned once or twice. See here: https://www.reddit.com/r/MachineLear...model_to_onnx/ and here https://pytorch.org/docs/stable/onnx.html#limitations I'm all ears if anyone has any suggestions. This is the interface in ChaiNNer: I'm sure there more options when using PyTorch directly but I have not experimented with that. Edit: can someone test the 2x_LD-Anime_Skr_v1.0.pth model? It crashes on me. Last edited by Reel.Deel; 26th February 2023 at 07:29. |
|
26th February 2023, 11:56 | #11 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,218
|
For denoise process (in physical temporal dimension) we need models for motion compensation. The framerate interpolation models are only partially good for this processing. Better neural network engines for motion compensation need accept several input frames in a sequence (not only 2 frames as we have in RIFE now).
The 'training condition' for motion compensation process is simple enough: For given number of frames with marked 'current' frame it is required to create 'current' frame view using only 'other' frames. In simplest form with 2 frame only input - we have only 'current' and 'other' frame. So engine need to reconstruct 'current' frame using only samples data from 'other' frame (it can use 'current' frame only as 'objects positions reference'). It may be even more simple in compare with 'framerate conversion' process where engine have 2 frames 'before and after' and 'current' frame to reconstruct at time of model training. One issue I found when trying to use online AI upscalers - they may not understand the depth of field blur and start to attempt to make out of focus objects sharper too. So for simple 2D drawing without textures like anime it work for sharp lines restoration from aliasing and all frame deblur about good. But for 3D real world lens projection to 2D buffer + different type (and DOF blur) textures it may fail significantly. Last edited by DTL; 26th February 2023 at 12:11. |
26th February 2023, 15:34 | #12 | Link | |
Registered User
Join Date: Nov 2009
Posts: 2,371
|
Quote:
Does anyone know of a good inpaint model? I only know 1x_NMKD-YandereInpaint_375000_G and 1x_sudo_inpaint_PartialConv2D_424000_G but not sure which one might be better for opaque logos on live action content. 1x_JPEG_60-80.pth can also be good for deblocking old codec sources, need to compare with 1x-Kim2091_DeJpeg.onnx.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread Last edited by Dogway; 26th February 2023 at 16:36. |
|
26th February 2023, 16:10 | #13 | Link | |
Registered User
Join Date: Dec 2005
Location: Sweden
Posts: 721
|
Quote:
Edit: if someone knows how to do the training bit I can provide with before/after tests in UHD (Sony does 6k then downscales internally to 3840x2160 and it is so crisp with minimal halo when internal sharpness is turned off). Last edited by anton_foy; 26th February 2023 at 16:15. |
|
26th February 2023, 17:14 | #14 | Link | ||
Registered User
Join Date: Sep 2007
Posts: 5,598
|
Quote:
https://github.com/geekyutao/Image-Inpainting https://github.com/zengyh1900/Awesome-Image-Inpainting The problem I found with single image/spatial inpainting algos, is the flicker and temporal inconsistencies (you'd have to apply some temporal smoothing after) Many of them are difficult to get working (at least for me, on Windows) . An interesting one that I actually got to work, and is video inpainting (temporal +/- 3 frames) is OPN . Worked ok for some sources, not for others. YMMV https://github.com/seoungwugoh/opn-demo Chainner lists support for 2 spatial algos that I haven't tried yet; but they should "work" since the GUI has direct support Lama https://github.com/advimman/lama MAT https://github.com/fenglinglwb/MAT Quote:
There are typically catergorized into single image vs. video . "Video" algorithms takes >1 frame into account. All the ESRGAN / Real-ESRGAN derivatives are single image only. Examples of "video" algorithms that use "propogation", "alignment" at runtime are VRT, RVRT, BasicVSR++, RealBasicVSR. At runtime is important because feature are matched in adjacent frames in the specific input video. Beware video algorithms take lots of memory. Some of them have "tiling" options built in, but the results can be worse especially for operations like deblurring. The whole frame in +/- n frames should be compared on some tasks, otherwise you get discontinunities at the tile boundaries I posted some examples of BasicVSR/BasicVSR++ in a few threads here and other forums. The provided model works on "textbook" clean cases. A big problem for is degradation step wasn't included , so it's not as generalizable to real world, compression cases. RealBasicVSR adds degradation, but the results are quite blurry / detail loss |
||
26th February 2023, 17:22 | #15 | Link | ||
Registered User
Join Date: Sep 2007
Posts: 5,598
|
Quote:
Did you try the tiling options ? Quote:
And I tried a few ONNX converted models , they were bit identical to the pytorch version so far... |
||
26th February 2023, 17:34 | #16 | Link |
Registered User
Join Date: Jul 2022
Posts: 5
|
Would like to see
__________________
Check out my repo for stupid and/or possibly useful avs scripts: https://github.com/DonCanjas/Avisynth-Functions |
26th February 2023, 17:57 | #18 | Link |
Registered User
Join Date: Sep 2007
Posts: 5,598
|
Ok, some very minor differences (only detectable on amplified differences) between avs, chainner using onnx and vs. pytorch in some cases, when settings are the same (e.g. tiles) . I think some of them might be due to the way it's converted to float or perhaps [0,1] clipping. The differences are so minor they are nothing to worry about
|
26th February 2023, 18:01 | #19 | Link |
Registered User
Join Date: Sep 2007
Posts: 5,598
|
I think so, the requirements say Vulkan device. I do not see a "CPU" mode anywhere
https://github.com/Asd-g/avs-mlrt |
26th February 2023, 18:05 | #20 | Link | |
Registered User
Join Date: Jul 2018
Posts: 1,218
|
Quote:
Also it shows TensorRT at NVIDIA cards can run RIFE about 2x faster in compare with ncnn/Vulkan implementation. So RIFE (IFNet ?) can be compiled (?) to different hardware APIs ? As for only storing frames in memory - the 1920x1080 frame in 12bytes per sample RGB 32bit float is about 24 MB only. So 1/4 of 1GB onboard RAM can keep 10 frames and it is very good already to study how motion is going between several frames. I post an issue on some RIFE development project at github about possible changing RIFE algorithm for better denoise process - https://github.com/megvii-research/E...IFE/issues/304 but still no one reply. May be it require to search someone with understanding how RIFE work so the required changes can be implemented. Last edited by DTL; 26th February 2023 at 18:13. |
|
Thread Tools | Search this Thread |
Display Modes | |
|
|