NTIRE2017 Super-resolution Challenge: SNU_CVLab

klapse

Central Scrutinizer
Joined
Aug 30, 2012
Messages
1,932
Location
Germany
NTIRE2017 Super-resolution Challenge: SNU_CVLab

Super-resolution is the modern term given to a new generation of neural-network based image upscaling algorithms. I've been interested in this for quite some time and these by far the most impressive results to have graced my screen.




Scroll down in the linked github page for more example pictures; enjoy!

https://github.com/LimBee/NTIRE2017
 

levi

Still fresh, damnit!
Joined
Oct 6, 2008
Messages
11,489
Location
Somewhere off the coast of the EU
Possibly, although running a recombinant neural network (as I believe the terminology goes) as a ffmpeg filter might run quite slowly. Even the downscale filter taking a 720p input and scaling it down to 600p for playback on my EeePC doesn't quite run in realtime on the fastest machine I currently own, so I can't see myself using it like that any time soon, but maybe in a few years machines that can do this in reasonable timeframes will become more commonplace.
 

Autian

Still Fresh
Joined
Jul 19, 2017
Messages
37
Age
21
Location
Germany
Well, in ffmpeg motion detecion filters were implemented recently for FPS increases in the output video. It has around 8 algorithms to choose from, a few of them being dead slow. So I don't see why it should not get into ffmpeg.
I have an Eee PC too (1001HA) and I was surprised how slow it is compared to my main system with an E8600. Winamp takes 3% CPU time on the latter with the DSP plugin running my multiband compressor algorithm. On my Eee its around 35%, even without visualizers. I know it's a netbook, it has lower clock frequencies, an older CPU architecture and poor cooling, but still :D
So I would not do any demanding video processing on it anyway :)
 

klapse

Central Scrutinizer
Joined
Aug 30, 2012
Messages
1,932
Location
Germany
It shouldn't be too long before there's FOSS software to super-resolution old SD video which takes into account information from multiple frames. NN Super-resolution of individual frames creates distracting artefacts which vary between frames.

If any of y'alls finds a project that creates Full-HD video out of SD sources that's significantly better than bilinear scaling, I'd be quite interested to see it.

Here's an example of what I get from a 1970s film with EnhanceNet.

[doublepost=1524302970,1524301891][/doublepost]
Enhance!

Interesting stuff. I guess the HR patch is actually some high resolution source image, before blurring has been applied. Becuase otherwise it seems to win every time!
Afaik it's the source image, which they then downscale (4x?), then the super-resolution algorithms upscale that smaller pic and we compare the results to the original.
 

FBnil

Ready to Champion the Pyra to the World...
Joined
Dec 14, 2012
Messages
2,857
Location
Yurp
@klapse: That is pretty awesome. Just the buildings behind the blinds is a bit weird. But the face is awesomely restored. I also notice that it tries to unblur what should be blurred... so I'm not sure about motion pictures, but static pictures... wow...
 

klapse

Central Scrutinizer
Joined
Aug 30, 2012
Messages
1,932
Location
Germany
@klapse: That is pretty awesome. Just the buildings behind the blinds is a bit weird. But the face is awesomely restored. I also notice that it tries to unblur what should be blurred... so I'm not sure about motion pictures, but static pictures... wow...
I think the effect is nicest on the lampshade reflections. Will we someday get movie upscalers that create consistently pleasing results? I'm inclined to think so. It's a fascinating thing to play with.
 

FBnil

Ready to Champion the Pyra to the World...
Joined
Dec 14, 2012
Messages
2,857
Location
Yurp
I think the effect is nicest on the lampshade reflections. Will we someday get movie upscalers that create consistently pleasing results? I'm inclined to think so. It's a fascinating thing to play with.
I think so too. You know those services where you take a picture with the tower of Pisa and remove part of the image (because there is a person there), and these algorithms fill the space in using the best replacement? I think that when that tower of Pisa is modeled in 3D, it can be used to detect the exact viewpoint and angle and be used to "enhance" the picture using it's database of highres images and 3D rendering. But it would look weird if it can find a highres of a few objects and not others, thus mixing highres with lowres...
 

FBnil

Ready to Champion the Pyra to the World...
Joined
Dec 14, 2012
Messages
2,857
Location
Yurp

Oh, it's an annual thing!



Some of the entries have a PPTX that barely fits on a CD!
 

erico

Advanced Member
Joined
Oct 25, 2011
Messages
1,650
Age
43
Location
Brasil
Website
sites.google.com
I think the effect is nicest on the lampshade reflections. Will we someday get movie upscalers that create consistently pleasing results? I'm inclined to think so. It's a fascinating thing to play with.
Can be done manually, so it will be done automatically at some point.
I love this kind of stuff.
 
Top