1. bookVolume 32 (2022): Issue 1 (March 2022)
Journal Details
License
Format
Journal
eISSN
2083-8492
First Published
05 Apr 2007
Publication timeframe
4 times per year
Languages
English
access type Open Access

Performance Analysis of a Dual Stage Deep Rain Streak Removal Convolution Neural Network Module with a Modified Deep Residual Dense Network

Published Online: 31 Mar 2022
Volume & Issue: Volume 32 (2022) - Issue 1 (March 2022)
Page range: 111 - 123
Received: 09 Jul 2021
Accepted: 26 Jan 2022
Journal Details
License
Format
Journal
eISSN
2083-8492
First Published
05 Apr 2007
Publication timeframe
4 times per year
Languages
English
Abstract

The visual appearance of outdoor captured images is affected by various weather conditions, such as rain patterns, haze, fog and snow. The rain pattern creates more degradation in the visual quality of the image due to its physical structure compared with other weather conditions. Also, the rain pattern affects both foreground and background image information. The removal of rain patterns from a single image is a critical process, and more attention is given to remove the structural rain pattern from real-time rain images. In this paper, we analyze the single image deraining problem and present a solution using the dual stage deep rain streak removal convolutional neural network. The proposed single image deraining framework primarily consists of three main blocks: a derain streaks removal CNN (derain SRCNN), a modified residual dense block (MRDB), and a six-stage scale feature aggregation module (3SFAM). The ablation study is conducted to evaluate the performance of various modules available in the proposed deraining network. The robustness of the proposed deraining network is evaluated over the popular synthetic and real-time data sets using four performance metrics such as the peak signal-to-noise ratio (PSNR), the feature similarity index (FSIM), the structural similarity index measure (SSIM), and the universal image quality index (UIQI). The experimental results show that the proposed framework outperforms both synthetic and real-time images compared with other state-of-the-art single image deraining approaches. In addition, the proposed network takes less running and training time.

Keywords

Barnum, P.C., Narasimhan, S. and Kanade, T. (2009). Analysis of rain and snow in frequency space, International Journal of Computer Vision 86(2): 256, DOI: 10.1007/s11263-008-0200-2.10.1007/s11263-008-0200-2 Search in Google Scholar

Chen, Y. and Wang, W. (2020). Recursive modified dense network for single-image deraining, Journal of Electronic Imaging 29(3): 10–12.10.1117/1.JEI.29.3.033006 Search in Google Scholar

Ding, X., Chen, L., Zheng, X., Huang, Y. and Zeng, D. (2016). Single image rain and snow removal via guided L0 smoothing filter, Multimedia Tools and Applications 75(5): 2697–2712, DOI: 10.1007/s11042-015-2657-7.10.1007/s11042-015-2657-7 Search in Google Scholar

Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X. and Paisley, J. (2017). Removing rain from single images via a deep detail network, Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, USA, pp. 1715–1723. Search in Google Scholar

Garg, K. and Nayar, S.K. (2004). Detection and removal of rain from videos, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, Washington, USA, Vol. 1, pp. I–I. Search in Google Scholar

Gu, S., Meng, D., Zuo, W. and Zhang, L. (2017). Joint convolutional analysis and synthesis for sparse representation for single image layer separation, 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, pp. 1717–1725. Search in Google Scholar

He, K., Zhang, X., Ren, S. and Sun, J. (2016). Deep residual learning for image recognition, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, pp. 770–778. Search in Google Scholar

Huang, G., Liu, Z., Van Der Maaten, L. and Weinberger, K.Q. (2017). Densely connected convolutional networks, Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, USA, pp. 2261–2269. Search in Google Scholar

Jayaraman, T. and Chinnusamy, G.S. (2020a). Analysis of deep rain streaks removal convolutional neural network-based post-processing techniques in HEVC encoder, Journal of Circuits, Systems and Computers 30(2): 1–21, Paper no. 2150020.10.1142/S0218126621500201 Search in Google Scholar

Jayaraman, T. and Chinnusamy, G.S. (2020b). Investigation of filtering of rain streaks affected video sequences under various quantisation parameter in HEVC encoder using an enhanced V-BM4D algorithm, IET Image Processing 14(2): 337–347.10.1049/iet-ipr.2018.6005 Search in Google Scholar

Kang, L., Lin, C. and Fu, Y. (2012). Automatic single-image-based rain streaks removal via image decomposition, IEEE Transactions on Image Processing 21(4): 1742–1755.10.1109/TIP.2011.217905722167628 Search in Google Scholar

Kim, J., Lee, C., Sim, J. and Kim, C. (2013). Single-image deraining using an adaptive nonlocal means filter, IEEE International Conference on Image Processing, Melbourne, Australia, pp. 914–917. Search in Google Scholar

Kou, F., Chen, W., Wen, C. and Li, Z. (2015). Gradient domain guided image filtering, IEEE Transactions on Image Processing 24(11): 4528–4539.10.1109/TIP.2015.246818326285153 Search in Google Scholar

Kowal, M., Skobel, M., Gramacki, A. and Korbicz, J. (2021). Breast cancer nuclei segmentation and classification based on a deep learning approach, International Journal of Applied Mathematics and Computer Science 31(1): 85–106, DOI: 10.34768/amcs-2021-0007. Search in Google Scholar

Koziarski, M. and Cyganek, B. (2018). Impact of low resolution on image recognition with deep neural networks: An experimental study, International Journal of Applied Mathematics and Computer Science 28(4): 735–744, DOI: 10.2478/amcs-2018-0056.10.2478/amcs-2018-0056 Search in Google Scholar

Li, P., Tian, J., Tang, Y., Wang, G. and Wu, C. (2020). Model-based deep network for single image deraining, IEEE Access 8(1): 14036–14047.10.1109/ACCESS.2020.2965545 Search in Google Scholar

Li, X., Wu, J., Lin, Z., Liu, H. and Zha, H. (2018). Recurrent squeeze-and-excitation context aggregation net for single image deraining, Proceedings of the European Conference on Computer Vista (ECCV), Amsterdam, The Netherlands, pp. 254–269. Search in Google Scholar

Li, Y., Tan, R.T., Guo, X., Lu, J. and Brown, M.S. (2016). Rain streak removal using layer priors, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, pp. 2736–2744. Search in Google Scholar

Lin, C.Y., Tao, Z., Xu, A.S., Kang, L.W. and Akhyar, F. (2020). Sequential dual attention network for rain streak removal in a single image, IEEE Transactions on Image Processing 29(1): 9250–9265.10.1109/TIP.2020.302540232976099 Search in Google Scholar

Papiez, A., Badie, C. and Polanska, J. (2019). Machine learning techniques combined with dose profiles indicate radiation response biomarkers, International Journal of Applied Mathematics and Computer Science 29(1): 169–178, DOI: 10.2478/amcs-2019-0013.10.2478/amcs-2019-0013 Search in Google Scholar

Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L.,Desmaison, A.,Köpf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J. and Chintala, S. (2019). PyTorch: An imperative style, high-performance deep learning library, arXiv 1912.01703 (NeurIPS). Search in Google Scholar

Ren, D., Shang, W., Zhu, P., Hu, Q., Meng, D. and Zuo, W. (2020). Single image deraining using bilateral recurrent network, IEEE Transactions on Image Processing 29(1): 6852–6863.10.1109/TIP.2020.2994443 Search in Google Scholar

Ren, D., Zuo, W., Hu, Q., Zhu, P. and Meng, D. (2019). Progressive image deraining networks: A better and simpler baseline, IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, USA pp. 3937–3946. Search in Google Scholar

Sharma, P.K., Basavaraju, S. and Sur, A. (2021). High-resolution image de-raining using conditional GAN with sub-pixel upscaling, Multimedia Tools and Applications 80(1): 1075–1094, DOI: 10.1007/s11042-020-09642-7.10.1007/s11042-020-09642-7 Search in Google Scholar

Sheikh, H.R. and Bovik, A.C. (2006). Image information and visual quality, IEEE Transactions on Image Processing 15(2): 430–444.10.1109/TIP.2005.859378 Search in Google Scholar

Thiyagarajan, J. and Gowri Shankar, C. (2020). Quality improvement and performance analysis of high efficiency video coding under high quantization parameters and rain streaks, Signal, Image and Video Processing 14(2): 387–395, DOI: 10.1007/s11760-019-01565-7.10.1007/s11760-019-01565-7 Search in Google Scholar

Wang, C., Zhang, M., Su, Z., Yao, G., Wang, Y., Sun, X. and Luo, X. (2019). From coarse to fine: A stage-wise deraining net, IEEE Access 7(1): 84420–84428.10.1109/ACCESS.2019.2922549 Search in Google Scholar

Wang, M., Chen, L., Liang, Y., Hao, Y., He, H. and Li, C. (2020a). Single image rain removal with reusing original input squeeze-and-excitation network, IET Image Processing 14(8): 1467–1474.10.1049/iet-ipr.2019.0716 Search in Google Scholar

Wang, M., Chen, L., Liang, Y., Huang, H. and Cai, R. (2020b). Deep learning method for rain streaks removal from single image, Journal of Engineering 2020(13): 555–560.10.1049/joe.2019.1197 Search in Google Scholar

Wang, Y., Gong, D., Yang, J., Shi, Q., van den Hengel, A., Xie, D. and Zeng, B. (2020c). Deep Single Image Deraining via Modeling Haze-like Effect, IEEE Transactions on Multimedia 23(1): 1–1.10.1109/TMM.2020.3013383 Search in Google Scholar

Wang, Y., Zhang, D. and Dai, G. (2020d). Classification of high resolution satellite images using improved U-Net, International Journal of Applied Mathematics and Computer Science 30(3): 399–413, DOI: 10.34768/amcs-2020-0030. Search in Google Scholar

Wang, Z. and Bovik, A.C. (2002). A universal image quality index, IEEE Signal Processing Letters 9(3): 81–84.10.1109/97.995823 Search in Google Scholar

Wang, Z., Bovik, A.C., Sheikh, H.R. and Simoncelli, E.P. (2004). Image quality assessment: From error visibility to structural similarity, IEEE Transactions on Image Processing 13(4): 600–612.10.1109/TIP.2003.819861 Search in Google Scholar

Wei, W., Meng, D., Zhao, Q., Xu, Z. and Wu, Y. (2018). Semi-supervised transfer learning for image rain removal, 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2018, Long Beach, USA, pp. 3877–3886. Search in Google Scholar

Wu, S. and Zhou, J. (2020). MSFA-Net: A network for single image deraining, Journal of Physics: Conference Series 1584(1), Paper no. 012047. Search in Google Scholar

Yang, W., Tan, R.T., Feng, J., Guo, Z., Yan, S. and Liu, J. (2020). Joint rain detection and removal from a single image with contextualized deep networks, IEEE Transactions on Pattern Analysis and Machine Intelligence 42(6): 1377–1393.10.1109/TPAMI.2019.289579330703011 Search in Google Scholar

Yang, W., Tan, R.T., Feng, J., Liu, J., Guo, Z. and Yan, S. (2017). Deep joint rain detection and removal from a single image, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, USA, pp. 1685–1694. Search in Google Scholar

Zhang, H. and Patel, V.M. (2018). Density-aware single image de-raining using a multi-stream dense network, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Juan, USA, pp. 695–704. Search in Google Scholar

Zhang, L., Zhang, L., Mou, X. and Zhang, D. (2011). FSIM: A feature similarity index for image quality assessment, IEEE Transactions on Image Processing 20(8): 2378–2386.10.1109/TIP.2011.210973021292594 Search in Google Scholar

Zhang, Y., Tian, Y., Kong, Y., Zhong, B. and Fu, Y. (2018). Residual dense network for image super-resolution, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, pp. 2472–2481. Search in Google Scholar

Recommended articles from Trend MD

Plan your remote conference with Sciendo