Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
|
|
Thread Tools | Search this Thread | Display Modes |
19th January 2018, 17:09 | #21 | Link |
Registered Developer
Join Date: Sep 2006
Posts: 9,140
|
Looks promising. It definitely has a more "high-res" look to it than any other non-GAN algorithms I've seen so far. Of course a key question will be how much the artifacts will go down when fully trained.
As much as I like comparisons to NGU, it's probably not very fair, because NGU was created to run in real time. @feisty, how long does it take to upscale one 1080p video frame (using the Tesla), just to get a first impression about speed? (And if I may suggest, try using L1 loss instead of L2 loss.) |
20th January 2018, 04:57 | #22 | Link |
I'm Siri
Join Date: Oct 2012
Location: void
Posts: 2,633
|
@cwk
thx but my school is now funding me 10 tesla p100 gpus for ntire2018 @madshi I tried L1 loss before and it somehow failed to converge for my neural net, I tried to keep L1 loss and experimented tons of weird shit like "residual scaling", "warm up very low learning rate", "gradient clipping", ..., all failed to make L1 loss converge, things went back normal as I switched to L2 loss later, it just converged without any other extra bullshit it takes a few secs to upscale 512x512 to 1024x1024 on tesla p100 |
20th January 2018, 10:33 | #23 | Link |
Registered Developer
Join Date: Sep 2006
Posts: 9,140
|
Hmmm... So about a minute for upscaling one 1080p frame?
Strange thing, never had any trouble with L1 yet. Of course L2 helps achieve better PSNR, which might be beneficial for your paper. L1 usually looks better to my eyes, though. You could try Huber or Charbonnier loss as alternatives. Or you could try something like 10% L2 and 90% L1. Maybe those 10% L2 could help "guiding" L1 with your net? |
21st January 2018, 10:24 | #25 | Link |
Registered Developer
Join Date: Sep 2006
Posts: 9,140
|
Looks even "finer structured" than before now. It does look increasingly similar to GAN upscaled images. Are you still training with only 3 images? I wonder if training with such a small dataset might make your NN behave somewhat more GAN like? My understanding of GAN is that (very simplified) the NN learns a tendency to pick exactly one most likely solution, instead of averaging a number of probable solutions. So I wonder if your NN has such a large capacity and such a small dataset, maybe it ends up having exactly one solution for each problem that is far more likely than the others, resulting in something which looks more like GAN? Maybe if you train with a larger dataset, the typical look of averaged NNs comes back to some extent, because the NN then has more possibly matching solutions, leading to a higher use of averaging?
Just wildly speculating, of course. Would also like to see how your algo does on this image: http://madshi.net/clown.png I like the clown and castle images because they have a good mixture of high contrast geometric features and nature (trees, bushes etc). A good algo should do well on both. E.g. adding weird dot-crawl artifacts etc can make random textures (like trees, bushes etc) look even more realistic, but can harm elsewhere... |
21st January 2018, 11:59 | #26 | Link |
I'm Siri
Join Date: Oct 2012
Location: void
Posts: 2,633
|
well, the training set contains 3 actual images but augmented to total 18 images, the uncompressed binary of the training set is around 1GB (after augmentation and overlapped slicing) and the file size of the weights is around 20MB, so no way it coulda memorized all training samples, but yeah it's been overfitting cuz the training set is still way too small, I'll expand the training set to a much larger one for ntire 2018 and then I'll see how it goes, maybe ur rite |
21st January 2018, 12:44 | #27 | Link |
Registered Developer
Join Date: Sep 2006
Posts: 9,140
|
One thing the clown and castle images are also good for is to check how consistent the upscaling algo is in keeping line thinning roughly constant. E.g. if you look at the white van in the clown image, it has a stripe in the middle. Ideally an image upscaling algo should thin this stripe the same way from left to right. You can see in your upscaling result that the stripe is partially thinned very much and partially not thinned at all. For comparison, here's NGU:
NGU Sharp Low Quality: NGU Sharp Very High Quality: In Low Quality, the stripe is not thinned equally, either, and not thinned much at all. In Very High Quality, the result is not perfect, but quite ok. Do you have good training images? If not, I can send you some nice ones. |
21st January 2018, 12:53 | #28 | Link | ||
I'm Siri
Join Date: Oct 2012
Location: void
Posts: 2,633
|
Quote:
Quote:
|
||
22nd January 2018, 11:58 | #30 | Link | |
I'm Siri
Join Date: Oct 2012
Location: void
Posts: 2,633
|
Quote:
so it shares similar behaviors with Huber loss but it's a bit better cuz it's analytic, and any order of its derivative is also analytic, it's prettier cuz it got no non-differentiable singularities and Huber loss does (edit: the Huber loss itself is differentiable but its derivate is not, it's not differentiable for higher orders) Last edited by feisty2; 22nd January 2018 at 12:49. |
|
24th January 2018, 19:19 | #33 | Link |
Registered Developer
Join Date: Sep 2006
Posts: 9,140
|
It's not exactly the same image (e.g. the clouds are in a different position), but I suppose it doesn't matter much. Generally, I think it's a really good test image. So we could create a low-res version of this groundtruth and then use that instead of the other image. Of course a good question would be which downscaling algo should be used for that.
|
24th January 2018, 19:28 | #34 | Link | |
I'm Siri
Join Date: Oct 2012
Location: void
Posts: 2,633
|
Quote:
|
|
24th January 2018, 22:36 | #35 | Link |
Registered Developer
Join Date: Sep 2006
Posts: 9,140
|
Yeah, almost all papers test with catrom. Which makes sense because it makes PSNR/SSIM results more comparable. But for real life use it would be interesting to test the final NN with different downscaling methods, too.
|
30th January 2018, 09:05 | #37 | Link |
Registered Developer
Join Date: Sep 2006
Posts: 9,140
|
Wrong thread, edcrfv94. Anyway, short answer: That's not what it was made for. However, there are special cases where it might still do what you want. E.g. if the video you're playing was upscaled by the studio with a soft/bad upscaler (e.g. many UHD Blu-Rays were upscaled with Catrom), then you could try downscaling it to half resolution first, then upscale it again with NGU. That might then remove some aliasing and produce thinner lines. Follow-up please in the madVR thread.
|
26th February 2018, 09:40 | #39 | Link | |
Registered User
Join Date: Aug 2017
Posts: 1
|
Quote:
Last edited by Socia; 2nd November 2021 at 17:21. |
|
5th March 2018, 23:29 | #40 | Link | |
Registered User
Join Date: Jan 2004
Location: earth, barely
Posts: 96
|
Quote:
Pricing for the GPUs are here: https://cloud.google.com/compute/pricing#gpus |
|
Thread Tools | Search this Thread |
Display Modes | |
|
|