or maybe we dont need GANs to generate photo realistic image upscales..

[feisty2]

I went on and now experimented on 1/9 of my super resolution neural net, 5 million parameters roughly trained out of total 45 million parameters, I only trained it with 3 actual images (augmented to 18 images via flipping, mirroring and stuff) for 20 epochs of backprop, then tested the neural net on 4 images that it had never seen before

test results
small:

upscaled:


small:

upscaled:


small:

upscaled:


small:

upscaled:


the results already look pretty photo realistic to me, I aint got no GAN in my neural net, no perceptual loss either, I simply picked L2 (aka MSE) as the loss function (PSNR friendly) and I honestly wasn't expecting something this good, the results are almost as photo realistic as the results of GANs, and the chance of shit going south should be much smaller than GANs since the neural net was optimized for L2, which is very loyal to the ground truth,,,

so maybe we don't need GANs to have photo realistic upscales, all we need is a better neural net structure (the structural design of my neural net serves a special physical significance, it's not about brainlessly stacking layers and blocks like some other models)

[poisondeathray]

the textures don't quite look right , keep on training it f2 . 5M not enuf

It's progress and it looks good, but there is a "coarseness" to the fine details that is reminiscent of nnedi2/3 upscaling artifacts . The fine details are what differentiates this approach

There are some issues with the bridge /railing that weren't there on your earlier example (look almost like field/interlace artifacts)

[feisty2]

Originally Posted by poisondeathray

the textures don't quite look right , keep on training it f2 . 5M not enuf

any way I can make GPUs 100x faster..? I got tesla p100 in school lab but it aint fast enough for my neural net and training set,,, I'm desperate for better hardware

well, it doesn't look quite rite for mainly 2 reasons, and the number of parameters is actually not one of them, first, training set of 3 effective images is too small, also, 20 epochs of backprop aint the shit for a large neural net to converge, I'd say probably at least 300 epochs

[cwk]

Originally Posted by feisty2

any way I can make GPUs 100x faster..? I got tesla p100 in school lab but it aint fast enough for my neural net and training set,,, I'm desperate for better hardware

How about a kickstarter campaign to rent a GPU-heavy instance in AWS for a couple of days? I'd contribute if it got you closer to a fully-trained instance.

[Socia]

Originally Posted by cwek

Wrong thread, edcrfv94. Anyway, short answer: That's not what it was made for. However, there are special cases where it might still do what you want. E.g. if the video you're playing was upscaled by the studio with a soft/bad upscaler (e.g. many UHD Blu-Rays were upscaled with Catrom), then you could try downscaling it to half resolution first, then upscale it again with NGU. That might then remove some aliasing and produce thinner lines. Follow-up please in the madVR thread.

That's an awesome idea imo. How much would something like that cost?

[cwk]

Originally Posted by Socia

That's an awesome idea imo. How much would something like that cost?

Last I checked, Google has the best GPU offerings, including the P100 nvidia chips. The P100s are roughly $750/month as an add-on to any other Google Compute Engine instance. So, roughly $900-$1200/month for a single GPU depending on your instance type. You could add additional GPUs to an instance depending on your need.

Pricing for the GPUs are here: https://cloud.google.com/compute/pricing#gpus

[feisty2]

Originally Posted by poisondeathray

It's progress and it looks good, but there is a "coarseness" to the fine details that is reminiscent of nnedi2/3 upscaling artifacts . The fine details are what differentiates this approach

speaking of nnedi3






no, this is not what I would call, photo realistic

[Snap]

Always enjoy seeing NN image scaler examples. Looking great.

Originally Posted by feisty2

speaking of nnedi3

no, this is not what I would call, photo realistic

It's interesting comparing them at similar sharpness. Lacks the same finer details which make up most of the photo realism but does have cleaner edges in places (the vertical bridge posts/sign for ex).

[poisondeathray]

Originally Posted by feisty2

speaking of nnedi3

no, this is not what I would call, photo realistic

Exactly! That's the main differentiating factor, the very 1st thing I said in the other thread that impressed me. The "coarseness" result common to nnedi3 type approaches vs. the fine details when you introduced this in the other thread

This version has some coarseness features too (definitely better than nnedi3), but going back now I can see why - the other pics in the other thread were at lower dimensions

So I'm thinking you can improve on it

[poisondeathray]

I don't know the details of what it takes to train... Can you pause and restart for example ? Does it take 1 contiguous session ? Can you train it overnight at school for example for a few weeks ?



It looks good, I just think you can do better

Any thoughts as to why would this version generate those bridge horizontal artifacts ?

[feisty2]

Originally Posted by poisondeathray

I don't know the details of what it takes to train... Can you pause and restart for example ? Does it take 1 contiguous session ? Can you train it overnight at school for example for a few weeks ?

yes, but I did the math, with tesla p100 it's gonna take approx 8 yrs if I wanna train the full model with the full training set for 300 epochs of backprop

It looks good, I just think you can do better

its gonna be better for sure, I'm just surprised that for now it already looks closer to the results of GANs in general (yes there're shitty details and textures) rather than "traditional" stuff like nnedi3 or waifu2x

Any thoughts as to why would this version generate those bridge horizontal artifacts ?

simple, training set too small and 20 epochs of backprop too few

[raffriff42]

The bridge has minor artifacts (oversharpening and/or residual interlacing) that are exaggerated (or at least, not smoothed) in f2's images

[CkJ]

Could you test on these images?






I'm curious to see your approach compares with NGU Sharp.

[feisty2]

there's zero point to compare this with NGU just yet, cuz it's doomed to introduce hell lot more artifacts than NGU for now, cuz the neural net is not yet fully trained, actually make that, yet just "barely trained"
anyways, the castle image, the source image is way too soft and doesn't have a lot of details,, so the result is just mediocre quality and suffers from mild level of artifacts


the cartoon image is 100% pointless, it's not photo realistic to begin with
the result of lighthouse image suffers from crappy artifacts..

[CkJ]

Well, it looks promising. How about the speed? Is it possible to use on videos or just available for images? And will you release an avisynth plugin or it will be a commerical software? Thanks.

[feisty2]

Originally Posted by CkJ

Well, it looks promising. How about the speed? Is it possible to use on videos or just available for images? And will you release an avisynth plugin or it will be a commerical software? Thanks.

it's free and open source, but I will only release the source code and the exact design of the neural net after I submitted my paper to CVPR, sounds a bit snobbish but... trust me I'm really not snobbish like that, I just need this paper to get me thru my applications for graduate schools, if I release the source code rite now someone mite silently steal my idea and I'm fucked for good (the neural net mentioned in this thread is very different from the one in my super resolution github repo, so the code is not yet public)

it works for videos (temporally stable), but it's slow

[wonkey_monkey]

feisty2, can you make anything of this?

https://github.com/DmitryUlyanov/deep-image-prior

Seems to be about using untrained neural nets for noise reduction, inpainting, and upscaling. One of the test images from this thread is in the paper.

[poisondeathray]

Originally Posted by davidhorman

feisty2, can you make anything of this?

https://github.com/DmitryUlyanov/deep-image-prior

Seems to be about using untrained neural nets for noise reduction, inpainting, and upscaling. One of the test images from this thread is in the paper.

Yummy!

Looks promising too

Does it lend itself more easily to vapoursynth since both use python ?

[feisty2]

Originally Posted by poisondeathray

Yummy!

Looks promising too

Does it lend itself more easily to vapoursynth since both use python ?

no, cuz there's no numpySource, also u gonna need GPU versions of TensorFlow or PyTorch or stuff like that, all extremely hard to install, u mite as well wanna shoot urself after 1000 error msgs pop outta nowhere and ur installation of any of these things is still failure, CPU versions r easy to install but they r useless, too damn fucking slow, I'm talking about 100x slower
and amd gpus r fucked too, all require nvidia gpus

python does rule the machine learning field, so basically implementations for all recent papers r coded in python

[poisondeathray]

Originally Posted by feisty2

no, cuz there's no numpySource, also u gonna need GPU versions of TensorFlow or PyTorch or stuff like that, all extremely hard to install, u mite as well wanna shoot urself after 1000 error msgs pop outta nowhere and ur installation of any of these things is still failure, CPU versions r easy to install but they r useless, too damn fucking slow, I'm talking about 100x slower
and amd gpus r fucked too, all require nvidia gpus

python does rule the machine learning field, so basically implementations for all recent papers r coded in python

Ok thanks for explaining

No 1 clicky, or even semi-easy solution for a script

Back to training then...