eduzhai > Applied Sciences > Engineering >

iSeeBetter Spatio-temporal video super-resolution using recurrent generative back-projection networks

  • Save

... pages left unread,continue reading

Document pages: 11 pages

Abstract: Recently, learning-based models have enhanced the performance of single-imagesuper-resolution (SISR). However, applying SISR successively to each videoframe leads to a lack of temporal coherency. Convolutional neural networks(CNNs) outperform traditional approaches in terms of image quality metrics suchas peak signal to noise ratio (PSNR) and structural similarity (SSIM). However,generative adversarial networks (GANs) offer a competitive advantage by beingable to mitigate the issue of a lack of finer texture details, usually seenwith CNNs when super-resolving at large upscaling factors. We presentiSeeBetter, a novel GAN-based spatio-temporal approach to videosuper-resolution (VSR) that renders temporally consistent super-resolutionvideos. iSeeBetter extracts spatial and temporal information from the currentand neighboring frames using the concept of recurrent back-projection networksas its generator. Furthermore, to improve the "naturality " of thesuper-resolved image while eliminating artifacts seen with traditionalalgorithms, we utilize the discriminator from super-resolution generativeadversarial network (SRGAN). Although mean squared error (MSE) as a primaryloss-minimization objective improves PSNR SSIM, these metrics may not capturefine details in the image resulting in misrepresentation of perceptual quality.To address this, we use a four-fold (MSE, perceptual, adversarial, andtotal-variation (TV)) loss function. Our results demonstrate that iSeeBetteroffers superior VSR fidelity and surpasses state-of-the-art performance.

Please select stars to rate!

         

0 comments Sign in to leave a comment.

    Data loading, please wait...
×