Abstract
Visual detection and classification of water and waterbodies provide important information needed for managing water resources systems and infrastructure, such as developing flood early warning systems and drought management. But water itself is a challenging object for visual analysis because it is shapeless, colorless, and transparent. Therefore, detecting, tracking, and localizing water in different visual environments are difficult tasks. Computer vision (CV) techniques provide powerful tools for image processing and high-level scene analysis. Despite the complexities associated with water in visual scenes, there are still some physical differences, such as color, turbidity, and turbulence, affected by surrounding settings, which can potentially support CV modeling to cope with the visual processing challenges of water. The goal of this study is to introduce a new image data set, ATLANTIS Texture (ATeX), which represents various water textures of different waterbodies, and evaluate the performance of deep learning (DL) models for classification purposes on ATeX. Experimental results show that among DL models, EffNet-B7, EffNet-B0, GoogLeNet, and ShuffleNet provide the highest precision, recall, and F1 score. However, by considering the training time, total number of parameters, and total memory occupied by these models, ShuffleNet is presented as the most efficient DL network for water classification. Finally, results from this study suggest that ATeX provides a new benchmark to investigate existing challenges in the field of image analysis, in particular for water, which can help both water resources engineers and the computer vision community.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
The ATeX data set, models, and codes developed and used in this study are available online in the GitHub repository (https://github.com/smhassanerfani/atex). Moreover, the ATeX Wiki (https://github.com/smhassanerfani/atex/wiki) documents the guidelines for the list of waterbodies considered in the ATeX data set.
References
Cordts, M., M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele. 2016. “The cityscapes dataset for semantic urban scene understanding.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 3213–3223. New York: IEEE.
Cui, Y., M. Jia, T.-Y. Lin, Y. Song, and S. Belongie. 2019. “Class-balanced loss based on effective number of samples.” In Proc., IEEE/CVF Conf. on Computer Vision and Pattern Recognition, 9268–9277. New York: IEEE.
Deng, J., W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. 2009. “Imagenet: A large-scale hierarchical image database.” In Proc., 2009 IEEE Conf. on Computer Vision and Pattern Recognition, 248–255. New York: IEEE.
Eltner, A., P. O. Bressan, T. Akiyama, W. N. Gonçalves, and J. Marcato Junior. 2021. “Using deep learning for automatic water stage measurements.” Water Resour. Res. 57 (3): e2020WR027608. https://doi.org/10.1029/2020WR027608.
Erfani, S. M. H., Z. Wu, X. Wu, S. Wang, and E. Goharian. 2022. “Atlantis: A benchmark for semantic segmentation of waterbody images.” Environ. Modell. Software 149: 105333. https://doi.org/10.1016/j.envsoft.2022.105333.
Everingham, M., L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman. 2010. “The PASCAL visual object classes (VOC) challenge.” Int. J. Comput. Vision 88 (2): 303–338. https://doi.org/10.1007/s11263-009-0275-4.
Fawcett, T. 2006. “An introduction to ROC analysis.” Pattern Recognit. Lett. 27 (8): 861–874. https://doi.org/10.1016/j.patrec.2005.10.010.
Gebrehiwot, A., L. Hashemi-Beni, G. Thompson, P. Kordjamshidi, and T. E. Langan. 2019. “Deep convolutional neural network for flood extent mapping using unmanned aerial vehicles data.” Sensors 19 (7): 1486. https://doi.org/10.3390/s19071486.
He, K., X. Zhang, S. Ren, and J. Sun. 2016. “Deep residual learning for image recognition.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 770–778. New York: IEEE.
Howard, A. G., M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam. 2017. “MobileNets: Efficient convolutional neural networks for mobile vision applications.” Preprint submitted April 17, 2017. https://arxiv.org/abs/1704.04861.
Huang, G., Z. Liu, L. Van Der Maaten, and K. Q. Weinberger. 2017. “Densely connected convolutional networks.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 4700–4708. New York: IEEE.
Huang, G. B., M. Mattar, T. Berg, and E. Learned-Miller. 2008. “Labeled faces in the wild: A database forstudying face recognition in unconstrained environments.” In Workshop on faces in real-life images: Detection, alignment, and recognition. Lyon, France: HAL.
Iandola, F. N., S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer. 2016. “SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < 0.5 MB model size.” Preprint submitted February 24, 2016. https://arxiv.org/abs/1602.07360.
LeCun, Y., Y. Bengio, and G. Hinton. 2015. “Deep learning.” Nature 521 (7553): 436–444. https://doi.org/10.1038/nature14539.
Li, F.-F. 2017. “CS231n: Convolutional neural networks for visual recognition.” Accessed July, 2021. https://cs231n.github.io/convolutional-networks/.
Lin, T.-Y., P. Goyal, R. Girshick, K. He, and P. Dollár. 2017. “Focal loss for dense object detection.” In Proc., IEEE Int. Conf. on Computer Vision, 2980–2988. New York: IEEE.
Lo, S.-W., J.-H. Wu, F.-P. Lin, and C.-H. Hsu. 2015. “Visual sensing for urban flood monitoring.” Sensors 15 (8): 20006–20029. https://doi.org/10.3390/s150820006.
Ma, N., X. Zhang, H.-T. Zheng, and J. Sun. 2018. “ShuffleNet v2: Practical guidelines for efficient CNN architecture design.” In Proc., European Conf. on Computer Vision (ECCV), 116–131. Cham, Switzerland: Springer Cham.
Maier, H. R., and G. C. Dandy. 2000. “Neural networks for the prediction and forecasting of water resources variables: A review of modelling issues and applications.” Environ. Modell. Software 15 (1): 101–124. https://doi.org/10.1016/S1364-8152(99)00007-9.
Mosavi, A., P. Ozturk, and K.-W. Chau. 2018. “Flood prediction using machine learning models: Literature review.” Water 10 (11): 1536. https://doi.org/10.3390/w10111536.
Moy de Vitry, M., S. Kramer, J. D. Wegner, and J. Leitão. 2019. “Scalable flood level trend monitoring with surveillance cameras using a deep convolutional neural network.” Hydrol. Earth Syst. Sci. Discuss. 23 (11): 4621–4634. https://doi.org/10.5194/hess-23-4621-2019.
Mzurikwao, D., M. U. Khan, O. W. Samuel, J. Cinatl, M. Wass, M. Michaelis, G. Marcelli, and C. S. Ang. 2020. “Towards image-based cancer cell lines authentication using deep neural networks.” Sci. Rep. 10 (1): 1–15. https://doi.org/10.1038/s41598-020-76670-6.
Neuhold, G., T. Ollmann, S. Rota Bulo, and P. Kontschieder. 2017. “The mapillary vistas dataset for semantic understanding of street scenes.” In Proc., IEEE Int. Conf. on Computer Vision, 4990–4999. New York: IEEE.
Pally, R., and S. Samadi. 2022. “Application of image processing and convolutional neural networks for flood image classification and semantic segmentation.” Environ. Modell. Software 148 (Feb): 105285. https://doi.org/10.1016/j.envsoft.2021.105285.
Peterson, K. T., V. Sagan, and J. J. Sloan. 2020. “Deep learning-based water quality estimation and anomaly detection using Landsat-8/Sentinel-2 virtual constellation and cloud computing.” GIScience Remote Sens. 57 (4): 510–525. https://doi.org/10.1080/15481603.2020.1738061.
Powers, D. M. 2020. “Evaluation: From precision, recall and f-measure to roc, informedness, markedness and correlation.” Preprint, submitted October 11, 2020. https://arxiv.org/abs/2010.16061.
Razavi, S. 2021. “Deep learning, explained: Fundamentals, explainability, and bridgeability to process-based modeling.” Environ. Modell. Software 144 (Oct): 105159. https://doi.org/10.1016/j.envsoft.2021.105159.
Sandler, M., A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen. 2018. “MobileNetv2: Inverted residuals and linear bottlenecks.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 4510–4520. New York: IEEE.
Sarp, S., M. Kuzlu, M. Cetin, C. Sazara, and O. Guler. 2020. “Detecting floodwater on roadways from image data using mask-R-CNN.” In Proc., 2020 Int. Conf. on Innovations in Intelligent SysTems and Applications (INISTA), 1–6. New York: IEEE.
Sazara, C., M. Cetin, and K. M. Iftekharuddin. 2019. “Detecting floodwater on roadways from image data with handcrafted features and deep transfer learning.” In Proc., 2019 IEEE Intelligent Transportation Systems Conference (ITSC), 804–809. New York: IEEE.
Schmidhuber, J. 2015. “Deep learning in neural networks: An overview.” Neural Networks 61: 85–117. https://doi.org/10.1016/j.neunet.2014.09.003.
Shen, C. 2018. “A transdisciplinary review of deep learning research and its relevance for water resources scientists.” Water Resour. Res. 54 (11): 8558–8593. https://doi.org/10.1029/2018WR022643.
Simonyan, K., and A. Zisserman. 2014. “Very deep convolutional networks for large-scale image recognition.” Preprint submitted September 4, 2014. https://arxiv.org/abs/1409.1556.
Simpson, A. L., et al. 2019. “A large annotated medical image dataset for the development and evaluation of segmentation algorithms.” Preprint submitted February 25, 2019. https://arxiv.org/abs/1902.09063.
Szegedy, C., W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. 2015. “Going deeper with convolutions.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 1–9. New York: IEEE.
Tan, M., and Q. Le. 2019. “EfficientNet: Rethinking model scaling for convolutional neural networks.” In Proc., Int. Conf. on Machine Learning, PMLR, 6105–6114. Brookline, MA: Microtome Publishing.
Xie, S., R. Girshick, P. Dollár, Z. Tu, and K. He. 2017. “Aggregated residual transformations for deep neural networks.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 1492–1500. New York: IEEE.
Yu, F., H. Chen, X. Wang, W. Xian, Y. Chen, F. Liu, V. Madhavan, and T. Darrell. 2020. “Bdd100k: A diverse driving dataset for heterogeneous multitask learning.” In Proc., IEEE/CVF Conf. on Computer Vision and Pattern Recognition, 2636–2645. New York: IEEE.
Zagoruyko, S., and N. Komodakis. 2016. “Wide residual networks.” Preprints submitted May 23, 2016. https://arxiv.org/abs/1605.07146.
Zhang, E., L. Liu, and L. Huang. 2019. “Automatically delineating the calving front of Jakobshavn Isbræ from multitemporal TerraSAR-X images: A deep learning approach.” Cryosphere 13 (6): 1729–1741. https://doi.org/10.5194/tc-13-1729-2019.
Zhang, X., X. Zhou, M. Lin, and J. Sun. 2018. “Shufflenet: An extremely efficient convolutional neural network for mobile devices.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 6848–6856. New York: IEEE.
Zhao, H., J. Shi, X. Qi, X. Wang, and J. Jia. 2017. “Pyramid scene parsing network.” Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 2881–2890. New York: IEEE.
Zhou, B., H. Zhao, X. Puig, T. Xiao, S. Fidler, A. Barriuso, and A. Torralba. 2019. “Semantic understanding of scenes through the ade20k dataset.” Int. J. Comput. Vision 127 (3): 302–321. https://doi.org/10.1007/s11263-018-1140-0.
Information & Authors
Information
Published In
Journal of Water Resources Planning and Management
Volume 148 • Issue 11 • November 2022
Copyright
© 2022 American Society of Civil Engineers.
History
Received: Dec 8, 2021
Accepted: Jul 7, 2022
Published online: Sep 14, 2022
Published in print: Nov 1, 2022
Discussion open until: Feb 14, 2023
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.