Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Announcements and Chat > General Discussion

Reply
 
Thread Tools Search this Thread Display Modes
Old 16th November 2017, 16:42   #61  |  Link
zub35
Registered User
 
Join Date: Oct 2016
Posts: 49
madshi Try it, I can make mistakes with the choice of filters. File to download in the post above.
p.s. Private Messages

UPD: From a technical, it is better to create a new neural network, comparing frame by frame the source file with compression (several thousand variations) for restoration / removal of artifacts (given the different codecs and settings)
And then, to apply this neural network to upscale. Either combine them into one big neural network.

Based on the above. Provided that a stable algorithm. Add to the container (mkv) of the video file (or individual *.neural file), the minimum data needed for fast/realtime work of the neural network on the players.
At the same time, don't even have to create a new video standard, and apply them to the existing AVC or HEVC. Accordingly, retaining the ability to play videos where there is no power to the neural network.

Last edited by zub35; 16th November 2017 at 18:32.
zub35 is offline   Reply With Quote
Old 16th November 2017, 22:34   #62  |  Link
ABDO
Registered User
 
Join Date: Dec 2016
Posts: 55
Quote:
Originally Posted by madshi View Post
I find NGU-AA to be not only more detailed, but it also has less ringing artifacts and looks more natural at the same time! I guess I should ask (in the madVR thread) once more if NNEDI3 is still needed/useful.
for me i never used NNEDI3 since NGU-AA come to madvr, NGU-AA looks more natural as you said and much faster in the same time
Quote:
Would be interesting to see how it would handle typical movie sources. Usually these types of algos are too slow for real time use, though.
yeah, it is too slow for real time use,i will traning the network soon if author did not put the pretraning moudel and i wish it give good result in typical movie sources.

Quote:
How does it compare to madVR's "remove compression artifacts"?
it is equal to madvr RCA-6 Values, but madvr also much much faster, i will up some comparison image soon.
edit:
All Images upscaling with NGU Sharp VH

jpg Source
https://postimg.org/image/pcaefvqfd/

madvr RCA6-high 1080p
https://postimg.org/image/qtvurl0cp/

letsenhance-anti-jpeg source
https://postimg.org/image/qv5skkewp/

letsenhance-anti-jpeg 1080p
https://postimg.org/image/ay70o19pl/

i think while (RCA*6) totally treat jpeg artifacts as efficiency as (letsenhance-anti-jpeg), but it is clear that (madvr-RCA) excel in save more sharpen and texture detail ��, With Note that (madvr- RCA) give this result in real time, though for now definitely (madvr-RCA) does best job than (letsenhance-anti-jpeg) since we do not have any control over letsenhance-anti-jpeg strength.(sorry if i miss understand your question right)

Last edited by ABDO; 17th November 2017 at 03:00.
ABDO is offline   Reply With Quote
Old 17th November 2017, 14:43   #63  |  Link
zub35
Registered User
 
Join Date: Oct 2016
Posts: 49
ABDO He was referring to - apply filters to remove artifacts from compressed video of my example crf=25 and upload them in a png and jpeg and compare them with the results without filters removal of artifacts.

But I think (not tested) that removing artifacts, will not allow the algorithm to recover the details. Since the compress-artifacts contains information about the original content.
The presence of an artifact indicates that this place was something else, unlike those places where artifacts are not.

Last edited by zub35; 17th November 2017 at 14:47.
zub35 is offline   Reply With Quote
Old 17th November 2017, 16:45   #64  |  Link
ABDO
Registered User
 
Join Date: Dec 2016
Posts: 55
Quote:
Originally Posted by zub35 View Post
ABDO He was referring to - apply filters to remove artifacts from compressed video of my example crf=25 and upload them in a png and jpeg and compare them with the results without filters removal of artifacts.
i am sorry, i realy did not understand the question right.
Quote:
But I think (not tested) that removing artifacts, will not allow the algorithm to recover the details. Since the compress-artifacts contains information about the original content.
The presence of an artifact indicates that this place was something else, unlike those places where artifacts are not.
yeah, Unfortunately i am not sure, as i not technical man or programer, but as you said the technology is extremely promising, so i think the future will bring an Improvements to it.
ABDO is offline   Reply With Quote
Old 22nd November 2017, 16:41   #65  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
Quote:
Originally Posted by madshi View Post
Very good, thank you! I've de-rotated/de-mirrored your images and here they are for easy comparison:

no modification - | - rotated left - | - rotated right - | - mirrored horizontally

If you compare these images, you'll see that the texture changes a lot in all 4 images, it's completely different in each frame. It still has an overall similar look to it, but the changes are still much bigger than any dithering, so in motion this will look extremely noisy/unstable.

For still images it might not matter too much, but for video this type of "texture hallucination" is IMHO currently not feasible in motion, because it is not stable when the image content changes slightly. I'm not sure if the algorithm could be changed to fix this problem. I kind of doubt it because the algo by design doesn't even try to restore the original texture (which is technically impossible, anyway), it just tries to hallucinate a texture which hopefully has a similar look to the texture the original hi-res image had before downscaling. So the algo is by design not able to maintain a stable "position" of the texture in motion.

Even worse, if you look at the very bottom of the image, the alphalt texture is changing its brightness very strongly from frame to frame, this will actually produce visible flickering in motion. That said, these brightness fluctuations should be fixable with better neural network training.
the loss function defined in SRGAN has 2 sections, content loss and adversarial loss, the content loss is defined as a perceptual loss which is a high level VGG feature loss rather than pixel loss (SAD/MSE), the adversarial loss would try to make the reconstructed image look as close to a native high res image in general as possible and probably 80% of the magic comes from this section, so this section stays put, now a different content loss function would not affect the "hi res" magic much, but would determine the level of the "richness" of details in the generated image, and here u could remove the perceptual loss function and replace it with MSE, the perceptual loss function gives rich but unstable details while MSE gives blurry but stable details, adversarial loss paired with MSE would give u a slightly blurry and stable result that still looks native high resolution in general, I guess u could try with this and see if it works out alright
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 22nd November 2017, 16:49   #66  |  Link
madshi
Registered Developer
 
Join Date: Sep 2006
Posts: 9,072
Thanks. Do you happen to know any papers/projects which do it that way? Would like to see if the resulting images look pleasing enough to make it worth spending the time.
madshi is offline   Reply With Quote
Old 22nd November 2017, 17:31   #67  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
Quote:
Originally Posted by madshi View Post
Thanks. Do you happen to know any papers/projects which do it that way? Would like to see if the resulting images look pleasing enough to make it worth spending the time.
page 8 of the original SRGAN paper, the author compared the results of different loss functions

SRResNet: MSE
SRGAN-MSE: adversarial loss + MSE (what I suggested)
SRGAN-VGG22: adversarial loss + lower level of perceptual loss
SRGAN-VGG54: adversarial loss + higher level of perceptual loss
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 22nd November 2017, 17:54   #68  |  Link
madshi
Registered Developer
 
Join Date: Sep 2006
Posts: 9,072
Thanks, doesn't look too bad. Might be worth a try. It does seem to do some sort of texture sharpening, though. I wonder what that will do to video sources with compression artifacts...
madshi is offline   Reply With Quote
Old 11th May 2018, 19:21   #69  |  Link
zub35
Registered User
 
Join Date: Oct 2016
Posts: 49
neural network in restoration old films
(rus) https://yandex.ru/blog/company/oldfilms
example: 1 2 3 4

Last edited by zub35; 11th May 2018 at 19:46.
zub35 is offline   Reply With Quote
Old 13th May 2018, 02:07   #70  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,498
Quote:
Originally Posted by zub35 View Post
neural network in restoration old films
(rus) https://yandex.ru/blog/company/oldfilms
example: 1 2 3 4
This is cool, thanks

Any more info on what process was used ? Some things lost in google translation too
poisondeathray is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 15:03.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.