1. bookVolume 28 (2018): Issue 4 (December 2018)
Journal Details
License
Format
Journal
eISSN
2083-8492
First Published
05 Apr 2007
Publication timeframe
4 times per year
Languages
English
access type Open Access

Impact of Low Resolution on Image Recognition with Deep Neural Networks: An Experimental Study

Published Online: 11 Jan 2019
Page range: 735 - 744
Received: 05 Dec 2017
Accepted: 12 Apr 2018
Journal Details
License
Format
Journal
eISSN
2083-8492
First Published
05 Apr 2007
Publication timeframe
4 times per year
Languages
English
Abstract

Due to the advances made in recent years, methods based on deep neural networks have been able to achieve a state-of-the-art performance in various computer vision problems. In some tasks, such as image recognition, neural-based approaches have even been able to surpass human performance. However, the benchmarks on which neural networks achieve these impressive results usually consist of fairly high quality data. On the other hand, in practical applications we are often faced with images of low quality, affected by factors such as low resolution, presence of noise or a small dynamic range. It is unclear how resilient deep neural networks are to the presence of such factors. In this paper we experimentally evaluate the impact of low resolution on the classification accuracy of several notable neural architectures of recent years. Furthermore, we examine the possibility of improving neural networks’ performance in the task of low resolution image recognition by applying super-resolution prior to classification. The results of our experiments indicate that contemporary neural architectures remain significantly affected by low image resolution. By applying super-resolution prior to classification we were able to alleviate this issue to a large extent as long as the resolution of the images did not decrease too severely. However, in the case of very low resolution images the classification accuracy remained considerably affected.

Keywords

Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mane, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viegas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y. and Zheng, X. (2016). TensorFlow: Large-scale machine learning on heterogeneous distributed systems, arXiv: 1603.04467.Search in Google Scholar

Bevilacqua, M., Roumy, A., Guillemot, C. and Alberi-Morel, M.L. (2012). Low-complexity single-image super-resolution based on nonnegative neighbor embedding, British Machine Vision Conference (BMVC), Guildford, UK.10.5244/C.26.135Search in Google Scholar

Chang, H., Yeung, D.-Y. and Xiong, Y. (2004). Super-resolution through neighbor embedding, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR, Washington, DC, USA, Vol. 1, pp. I-I.Search in Google Scholar

da Costa, G.B.P., Contato, W.A., Nazare, T.S., Neto, J.E. and Ponti, M. (2016). An empirical study on the effects of different types of noise in image classification tasks, arXiv: 1609.02781.Search in Google Scholar

Dodge, S. and Karam, L. (2016). Understanding how image quality affects deep neural networks, 8th International Conference on Quality of Multimedia Experience (QoMEX), Lisbon, Portugal, pp. 1-6.Search in Google Scholar

Dong, C., Loy, C.C., He, K. and Tang, X. (2014). Learning a deep convolutional network for image super-resolution, European Conference on Computer Vision, Zurich, Switzerland, pp. 184-199.Search in Google Scholar

Dong, C., Loy, C.C. and Tang, X. (2016). Accelerating the super-resolution convolutional neural network, European Conference on Computer Vision, Amsterdam, The Netherlands, pp. 391-407.Search in Google Scholar

Dutta, A., Veldhuis, R.N. and Spreeuwers, L.J. (2012). The impact of image quality on the performance of face recognition, 33rd WIC Symposium on Information Theory in the Benelux, Boekelo, The Netherlands, pp. 141-148.Search in Google Scholar

Freeman, W.T., Jones, T.R. and Pasztor, E.C. (2002). Example-based super-resolution, IEEE Computer Graphics and Applications 22(2): 56-65.10.1109/38.988747Search in Google Scholar

Goodfellow, I., Bengio, Y. and Courville, A. (2016). Deep Learning, MIT Press, Cambridge, MA.Search in Google Scholar

Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. and Bengio, Y. (2014). Generative adversarial nets, in Z. Ghahramani et al. (Eds.), Advances in Neural Information Processing Systems, Curran Associates, Inc., Red Hook, NY, pp. 2672-2680.Search in Google Scholar

He, K., Zhang, X., Ren, S. and Sun, J. (2016). Deep residual learning for image recognition, Proceedings of the IEEE10.1109/CVPR.2016.90Search in Google Scholar

Huang, J.-B., Singh, A. and Ahuja, N. (2015). Single image super-resolution from transformed self-exemplars, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, pp. 5197-5206.Search in Google Scholar

Johnson, J., Karpathy, A. and Fei-Fei, L. (2016). DenseCap: Fully convolutional localization networks for dense captioning, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, VN, USA, pp. 4565-4574.Search in Google Scholar

Karahan, S., Yildirum, M.K., Kirtac, K., Rende, F.S., Butun, G. and Ekenel, H.K. (2016). How image degradations affect deep CNN-based face recognition, International Conference of the Biometrics Special Interest Group (BIOSIG), Darmstadt, Germany, pp. 1-5.Search in Google Scholar

Kim, J., Kwon Lee, J. and Mu Lee, K. (2016). Accurate image super-resolution using very deep convolutional networks, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, pp. 1646-1654.Search in Google Scholar

Kingma, D. and Ba, J. (2014). Adam: A method for stochastic optimization, arXiv: 1412.6980.Search in Google Scholar

Koziarski, M. and Cyganek, B. (2017). Image recognition with deep neural networks in presence of noise-dealing with and taking advantage of distortions, Integrated Computer- Aided Engineering 24(4): 337-349.10.3233/ICA-170551Search in Google Scholar

Krizhevsky, A., Sutskever, I. and Hinton, G.E. (2012). ImageNet classification with deep convolutional neural networks, Neural Information Processing Systems, Lake Tahoe, CA, USA, pp. 1097-1105.Search in Google Scholar

Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z. and Shi, W. (2016). Photo-realistic single image super-resolution using a generative adversarial network, arXiv: 1609.04802.Search in Google Scholar

Lim, B., Son, S., Kim, H., Nah, S. and Lee, K.M. (2017). Enhanced deep residual networks for single image super-resolution, IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshop, Honolulu, HI, USA, Vol. 1, p. 3.Search in Google Scholar

Long, J., Shelhamer, E. and Darrell, T. (2015). Fully convolutional networks for semantic segmentation, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, pp. 3431-3440.Search in Google Scholar

Mao, X., Shen, C. and Yang, Y.-B. (2016). Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections, Neural Information Processing Systems, Barcelona, Spain, pp. 2802-2810.Search in Google Scholar

Martin, D., Fowlkes, C., Tal, D. and Malik, J. (2001). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics, 8th IEEE International Conference on Computer Vision, ICCV 2001, Vancouver, Canada, Vol. 2, pp. 416-423.Search in Google Scholar

Peng, X., Hoffman, J., Stella, X.Y. and Saenko, K. (2016). Fine-to-coarse knowledge transfer for low-res image classification, IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA, pp. 3683-3687.Search in Google Scholar

Ronneberger, O., Fischer, P. and Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation, International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, pp. 234-241.Search in Google Scholar

Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C. and Fei-Fei, L. (2015). ImageNet large scale visual recognition challenge, International Journal of Computer Vision 115(3): 211-252.10.1007/s11263-015-0816-ySearch in Google Scholar

Sanchez, A., Moreno, A.B., Velez, D. and V´elez, J.F. (2016). Analyzing the influence of contrast in large-scale recognition of natural images, Integrated Computer-Aided Engineering 23(3): 221-235.10.3233/ICA-160516Search in Google Scholar

Schmidhuber, J. (2015). Deep learning in neural networks: An overview, Neural Networks 61(1): 85-117.10.1016/j.neunet.2014.09.003Search in Google Scholar

Schulter, S., Leistner, C. and Bischof, H. (2015). Fast and accurate image upscaling with super-resolution forests, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, pp. 3791-3799.Search in Google Scholar

Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D. and Wang, Z. (2016). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, pp. 1874-1883.Search in Google Scholar

Simonyan, K. and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition, arXiv: 1409.1556.Search in Google Scholar

Sun, Y., Wang, X. and Tang, X. (2013). Deep convolutional network cascade for facial point detection, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, pp. 3476-3483.Search in Google Scholar

Tai, Y., Yang, J. and Liu, X. (2017). Image super-resolution via deep recursive residual network, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, pp. 2790-2798.Search in Google Scholar

Timofte, R., De Smet, V. and Van Gool, L. (2014). A+: Adjusted anchored neighborhood regression for fast super-resolution, Asian Conference on Computer Vision, Singapore, Singapore, pp. 111-126.Search in Google Scholar

Vasiljevic, I., Chakrabarti, A. and Shakhnarovich, G. (2016). Examining the impact of blur on recognition by convolutional networks, arXiv: 1611.05760.Search in Google Scholar

Yang, J., Wright, J., Huang, T.S. and Ma, Y. (2010). Image super-resolution via sparse representation, IEEE Transactions on Image Processing 19(11): 2861-2873.10.1109/TIP.2010.2050625Search in Google Scholar

Zeyde, R., Elad, M. and Protter, M. (2010). On single image scale-up using sparse-representations, International Conference on Curves and Surfaces, Paris, France, pp. 711-730.Search in Google Scholar

Zou, W.W. and Yuen, P.C. (2012). Very low resolution face recognition problem, IEEE Transactions on Image Processing 21(1): 327-340.10.1109/TIP.2011.2162423Search in Google Scholar

Recommended articles from Trend MD

Plan your remote conference with Sciendo