An investigation of rain streak removal models based on expert experience and deep learning
DOI:
https://doi.org/10.54097/hset.v56i.9812Keywords:
Computer vision technology, rain streak removal models, deep learningAbstract
Computer vision technology has a wide range of applications in today's society, and image rain removal is of great importance in outdoor vision capture. Today's image de-rain techniques are divided into video de-rain, and image de-rain, with the image de-rain task being more difficult than the video de-rain task due to the lack of a time factor. Current image rain removal methods are divided into three main types: filter-based methods, a priori knowledge-based methods and deep learning methods. Although these methods can achieve the image rain removal requirements to a certain extent, there is still no highly generalized method that can better solve the image rain removal problem in all cases. This paper first considers a filter-based approach, which takes less time to run but is difficult to remove cleanly for complex rain streaks. Secondly, this paper examines an a priori knowledge-based approach, which requires the study of how rain images are constructed as a priori knowledge and then uses the existing prior knowledge to remove rain from the images. This approach has a high reliance on prior knowledge and poor generalization. Finally, this paper investigates a deep learning-based method that requires a large number of supervised samples for training and has a better rain removal effect, but ignores the prior knowledge of rain streaks and is prone to overfitting. Based on these three methods we collected some data and experimental results in this field and summarized and analyzed them, giving reasons for the strengths and weaknesses of these three models and presenting new perspectives for future improvements in image de-rain methods.
Downloads
References
Comaniciu D, Ramesh V, Meer P. Kernel-based object tracking[J]. Pattern Analysis & Machine Intelligence, 2003, 25(5):564-575.
Itti L. A model of saliency-based visual attention for rapid scene analysis[J]. IEEE Trans, 1998, 20.
Farenzena M, Bazzani L, Perina A, et al. Person re-identification by symmetry-driven accumulation of local features[C]// Proc IEEECONFERENCE on Computer Vision & Patternrecognition. IEEE, 2010:2360-2367.
Shehata M S, Cai J, Badawy W M, et al. Video-Based Automatic Incident Detection for Smart Roads: The Outdoor Environmental Challenges Regarding False Alarms[J]. IEEE Transactions on Intelligent Transportation Systems, 2008, 9(2):349-360.
Fu Y H, Kang L W, Lin C W, et al. Single-frame-based rain removal via image decomposition[C]// IEEE International Conference on Acoustics. IEEE, 2014.
Jing X, Wei Z, Peng L, et al. Removing rain and snow in a single image using guided filter[C]// 2012 IEEE International Conference on Computer Science and Automation Engineering (CSAE). 0.
Zheng X, Liao Y, Guo W, et al. Single-image-based rain and snow removal using multi-guided filter[J]. Springer Berlin Heidelberg, 2013.
Ding X, Chen L, Zheng X, et al. Single image rain and snow removal via guided L0 smoothing filter[J]. Multimedia Tools & Applications, 2016, 75(5):2697-2712.
Huang S Y, Xu Y T, Yang Y, et al. Image deraining algorithm based on morphological filtering and convolution sparse coding[J]. Chinese Journal of Image Graphics, 2022, 27(5): 1522-1536
Sun S H, Fan S P, Wang Y C F. Exploiting image structural similarity for single image rain removal[C]//2014 IEEE International Conference on Image Processing (ICIP). IEEE, 2014: 4482-4486.
Luo Y, Xu Y, Ji H. Removing rain from a single image via discriminative sparse coding[C]//Proceedings of the IEEE international conference on computer vision. 2015: 3397-3405.
Li Y, Tan R T, Guo X, et al. Rain streak removal using layer priors[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 2736-2744.
Zhu L, Fu C W, Lischinski D, et al. Joint bi-layer optimization for single-image rain streak removal[C]//Proceedings of the IEEE international conference on computer vision. 2017: 2526-2534.
Mu P, Chen J, Liu R, et al. Learning bilevel layer priors for single image rain streaks removal[J]. IEEE Signal Processing Letters, 2018, 26(2): 307-311.
Sun G, Shao H, Cattani C. A priori-guided multi-layer rain-aware network for single image deraining[J]. Knowledge-Based Systems, 2022, 235: 107613.
Fu X, Huang J, Ding X, et al. Clearing the skies: A deep network architecture for single-image rain removal[J]. IEEE Transactions on Image Processing, 2017, 26(6): 2944-2956.
Zhang H, Sindagi V, Patel V M. Image de-raining using a conditional generative adversarial network[J]. IEEE transactions on circuits and systems for video technology, 2019, 30(11): 3943-3956.
Li X, Wu J, Lin Z, et al. Recurrent squeeze-and-excitation context aggregation net for single image deraining[C]//Proceedings of the European conference on computer vision (ECCV). 2018: 254-269.
Ren D, Zuo W, Hu Q, et al. Progressive image deraining networks: A better and simpler baseline[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 3937-3946.
Wei W, Meng D, Zhao Q, et al. Semi-supervised transfer learning for image rain removal[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 3877-3886.
Yu Y, Yang W, Tan Y P, et al. Towards Robust Rain Removal Against Adversarial Attacks: A Comprehensive Benchmark Analysis and Beyond[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 6013-6022.
Liang X, Zhao F. Single-Image Rain Removal Network Based on an Attention Mechanism and a Residual Structure[J]. IEEE Access, 2022.orks[J]. IEEE Transactions on Industrial Electronics, 2019, 67(8): 6473-6482.
Liu P, Xu J, Liu J F, et al. An Algorithm for Real-time Analysis of Rain-affected Videos[J]. ACTA AUTOMATICA SINICA, 2010, 36(10):1371-1378.
Nair V, Hinton G E. Rectified Linear Units Improve Restricted Boltzmann Machines Vinod Nair[C]// Proceedings of the 27th International Conference on Machine Learning (ICML-10), June 21-24, 2010, Haifa, Israel. 2010.
Yang W, Tan R T, Feng J, et al. Deep joint rain detection and removal from a single image[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 1357-1366.
Fu X, Huang J, Zeng D, et al. Removing rain from single images via a deep detail network[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 3855-3863.
Zhang H, Patel V M. Density-aware single image de-raining using a multi-stream dense network[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 695-704.
Li R, Cheong L F, Tan R T. Heavy rain image restoration: Integrating physics model and conditional adversarial learning[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 1633-1642.
Hu X, Fu C W, Zhu L, et al. Depth-attentional features for single-image rain removal[C]//Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition. 2019: 8022-8031.
Wang T, Yang X, Xu K, et al. Spatial attentive single-image deraining with a high quality real rain dataset[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 12270-12279.
Li S, Araujo I B, Ren W, et al. Single image deraining: A comprehensive benchmark analysis[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 3838-3847.
Gu S, Meng D, Zuo W, et al. Joint convolutional analysis and synthesis sparse representation for single image layer separation[C]//Proceedings of the IEEE International Conference on Computer Vision. 2017: 1708-1716.
Wang H, Wu Y, Li M, et al. A survey on rain removal from video and single image[J]. arXiv preprint arXiv:1909.08326, 2019.
Yang W, Tan R T, Feng J, et al. Joint rain detection and removal from a single image with contextualized deep networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2019, 42(6): 1377-1393.
Kang L W, Lin C W, Fu Y H. Automatic Single-Image-Based Rain Streaks Removal via Image Decomposition[J]. IEEE Transactions on Image Processing, 2011, 21(4):1742-1755.
Eigen D, Krishnan D, Fergus R. Restoring an Image Taken through a Window Covered with Dirt or Rain[C]// IEEE International Conference on Computer Vision. IEEE, 2014.
Dong C, Chen C L, He K, et al. Image Super-Resolution Using Deep Convolutional Networks[J]. arXiv e-prints, 2014(2).
Shang Z, Wu J. Research progress on single image rain removal technology[C]// 2020 IEEE 5th Information Technology and Mechatronics Engineering Conference (ITOEC). IEEE, 2020.
Li P, Tian J, Tang Y, et al. Model-Based Deep Network for Single Image Deraining[J]. IEEE Access, 2020, 8:14036-14047.
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
