Best Solution for Single Image Super-Resolution Using a Generative Adversarial Network Deep Learning

As there is sudden growth in the popularity of image-based applications, improving image content becomes crucial. There are many hardware-based solutions but image super-resolution is more of a software approach.

Single Image Super Resolution: Problem Statement

Super-resolution basically includes enhancing the low-resolution (LR) image to high-resolution(HR). It has been a topic of research and still an ongoing process in technology. There are many methods for achieving high-resolution enlargements for pixel- based images.

Single Image Super Resolution is a method of deep learning. It has its applications in the fields of surveillance, forensics, medical imaging, satellite imaging, and consumer photography.

If you wish to restore an image into a high-resolution (HR) image correctly, it is necessary to infer high-frequency components of a low-resolution image. In applications like video surveillance, forensic investigation, face recognition, medical diagnosis, satellite images, and pattern recognition, it is extremely important to extract information from the images. In this process, if the image is enlarged beyond a certain limit, it could result in a blurred image with no crucial information. Hardware limitations of sensors are one of the main cause behind this problem.

We decided to try various methods to maximize our PSNR performance. We analyzed SRResNet which is one of the successful methods of super-resolution with a convolutional neural network. It was the first method of deep learning to surpass all other traditional methods for super-resolution. It is also a state-of-art, deep learning super-resolution method. The proposed method uses a convolutional neural network (like SRCNN)to map interpolated LR patches to desired HR patches.

There has been a dramatic development in the accuracy and speed of single image super-resolution by using faster and deeper convolutional neural networks. But the main problem still remains as it is in the picture. How to recover the finer texture details while super-resolving at large upscaling factors?

The answer to this could be Super-resolution GAN. It applies a deep network in combination a challenger network to produce higher resolution images.

SRGAN - Super-resolution GAN

In this method, a high-resolution image (HR) is downsampled to a low-resolution image (LR). A GAN generator helps in un-sampling LR image to a super-resolution image (SR). A discriminator is used to differentiate the HR images and send back the GAN loss to train the discriminator and generator.

We propose a perceptual loss function which consists of an adversarial loss and a content loss. The adversarial loss pushes our solution to the natural image manifold using a discriminator network that is trained to differentiate between the super-resolved images and original photo-realistic images. In addition, we use a content loss motivated by perceptual similarity instead of similarity in pixel space.

Fig 1: The diagram explains how Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network is implemented in Keras.

Fig 1

Fig2: Here is how the Generator and Discriminator network functions:

Fig 2

Fig3: It shows how to train the generator and discriminator to identified an image as real or fake.

Fig 3

The formulas used in to calculate Perpetual loss, Adversarial loss and Content loss:Perpetual Loss:

Adversarial Loss:

Content Loss:

Conclusion:

We could say that the Super-Resolution obtained using Generative Adversarial Networks worked the best for generating high-resolution images. SRGAN provides more details as compared to the similar design without GAN (SRResNet). SRGAN proves to be more appealing to the human eye. It is the first framework which is capable of achieving natural images for 4x upscaling factors.

You can see the success with the help of below images.

The first image is LR provided as an input and the second image is the output that we get as an HR.

Coruscate has been using this technology to provide businesses with the best super-resolution solutions to help them achieve great heights.

Reference: https://arxiv.org/abs/1609.04802