DEEP LEARNING BASED APPROACH TO IMAGE DEBLURRING AND SUPER-RESOLUTION

Abstract

Image deblurring and super-resolution, as representative problems of image restoration, have been studied for decades. Due to their wide range of applications, numerous techniques have been proposed to tackle these problems, still inspiring innovations for better performance. Recently, deep learning has become a powerful framework for many image processing tasks including restoration. In particular, generative adversarial networks (GANs), proposed by Goodfellow et al. [1], have demonstrated remarkable performances on generating plausible images. However, training GANs for image restoration is a non-trivial task. This research investigates optimization schemes for GANs that benefit image quality improvement, by providing meaningful training objective functions. To serve the purpose of improving image quality, training GANs needs suitable constraints for navigating towards specific optimization direction. Hence, the method of image quality constrained GANs is proposed by introducing effective image quality measures into the training objective or loss function. Compared with the existing methods, experimental results show that the proposed image quality losses are capable of achieving improved results on GAN models. To further improve the proposed constrained GAN method, a multi-objective training scheme is proposed, termed as HypervolGAN. By integrating with a quality evaluation metric originated from multiple objective optimization problem, the training objective function of GAN is further modified. HypervolGAN provides an efficient alternative for simultaneous optimization of multiple training objectives, and automatically tunes the weights for each objective during the training. It is shown to further improve on the image quality with fine-tuned multi-objectives and their weights. Both proposed methods have been demonstrated to be effective in improving image quality, on extensive quantitative and qualitative results provided in this thesis. Moreover, both methods are straightforward, simple to adopt, and flexible to be adapted according to applications. Therefore, the proposed methods are easy to generalise for improving performance of other image restoration tasks or to be embedded in other deep learning models.

Date of Award	1 Aug 2022
Original language	English
Awarding Institution	The University of Manchester
Supervisor	Hujun Yin (Supervisor) & Frank Podd (Supervisor)