Anomaly Detection and Classification in Water Distribution Networks Integrated with Hourly Nodal Water Demand Forecasting Models and Feature Extraction Technique
Publication: Journal of Water Resources Planning and Management
Volume 148, Issue 11
Abstract
Effective detection and classification of abnormalities, such as pipe bursts, leakage, illegal water use, and sensor failures, are critical for assisting water utilities in decision making, rapid response, and minimizing damage and disruption. This work presents a new flow data-based anomaly detection and classification method in water distribution networks. The method first establishes hourly nodal water demand forecasting models, then uses a unique integration of feature extraction technique of flow curve and convolutional neural network method to enable anomaly detection and classification from continually updated time window flow data. Verification progress from real and synthetic data of the case network shows that the proposed method can identify four common types of abnormal patterns in a fast and reliable manner with high recognition accuracy. The established models have self-learning capabilities, can process flow data in real time, and do not require hydraulic models to assist in analysis, which can be promising for wide practical applications in smart management of water distribution systems.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request, including the monitoring flow data, weather data, XGBoost code, and CNN code.
Acknowledgments
This work was jointly supported by the National Natural Science Foundation of China (51778178 and 51978203) and the National Key Research and Development Program of China (2016YFC0802402 and 2018YFC0406201-3).
References
Abdulshaheed, A., F. Mustapha, and A. Ghavamian. 2017. “A pressure-based method for monitoring leaks in a pipe distribution system: A review.” Renewable Sustainable Energy Rev. 69 (Mar): 902–911. https://doi.org/10.1016/j.rser.2016.08.024.
Al-Zahrani, M. A., and A. Abo-Monasar. 2015. “Urban residential water demand prediction based on artificial neural networks and time series models.” Water Resour. Manage. 29 (10): 3651–3662. https://doi.org/10.1007/s11269-015-1021-z.
Ares-Milián, M. J., M. Quiñones-Grueiro, C. Verde, and O. Llanes-Santiago. 2021. “A leak zone location approach in water distribution networks combining data-driven and model-based methods.” Water 13 (20): 2924. https://doi.org/10.3390/w13202924.
Breiman, L. 2001. “Random forests.” Mach. Learn. 45 (1): 5–32. https://doi.org/10.1023/A:1010933404324.
Chen, T., and C. Guestrin. 2016. “XGBoost: A scalable tree boosting system.” In Proc., 22nd ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, 785–794. New York: Association for Computing Machinery. https://doi.org/10.1145/2939672.2939785.
Chen, T., T. He, M. Benesty, V. Khotilovich, and Y. Tang. 2017. “XGBoost: Extreme gradient boosting (R package version 0.6-4).” Accessed January 5, 2017. https://CRAN.R-project.org/package=xgboost.
Christodoulou, S. E., E. Kourti, and A. Agathokleous. 2017. “Water loss detection in water distribution networks using wavelet change-point detection.” Water Resour. Manage. 31 (3): 979–994. https://doi.org/10.1007/s11269-016-1558-5.
Colombo, A. F., P. Lee, and B. W. Karney. 2009. “A selective literature review of transient-based leak detection methods.” J. Hydro-environ. Res. 2 (4): 212–227. https://doi.org/10.1016/j.jher.2009.02.003.
Datta, S., and S. Sarkar. 2016. “A review on different pipeline fault detection methods.” J. Loss Prev. Process Ind. 41 (May): 97–106. https://doi.org/10.1016/j.jlp.2016.03.010.
de Souza Groppo, G., M. A. Costa, and M. Libânio. 2019. “Predicting water demand: A review of the methods employed and future possibilities.” Water Supply 19 (8): 2179–2198. https://doi.org/10.2166/ws.2019.122.
Donkor, E. A., T. A. Mazzuchi, R. Soyer, and J. Alan Roberson. 2014. “Urban water demand forecasting: Review of methods and models.” J. Water Resour. Plann. Manage. 140 (2): 146–159. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000314.
Du, X.-X., W. Zeng, M. F. Lambert, L. Chen, and E. Jing Hu. 2021. “Approach for near-real-time pipe burst detection, localization, and quantification with low data transmission and sampling rates.” J. Water Resour. Plann. Manage. 147 (7): 04021032. https://doi.org/10.1061/(ASCE)WR.1943-5452.0001380.
Easterling, D. R., G. A. Meehl, C. Parmesan, S. A. Changnon, T. R. Karl, and L. O. Mearns. 2000. “Climate extremes: Observations, modeling, and impacts.” Science 289 (5487): 2068–2074. https://doi.org/10.1126/science.289.5487.2068.
Fan, C., C. Song, K. Liu, L. Ke, B. Xue, T. Chen, C. Fu, and J. Cheng. 2021. “Century-scale reconstruction of water storage changes of the largest lake in the Inner Mongolia Plateau using a machine learning approach.” Water Resour. Res. 57 (2): e2020WR028831. https://doi.org/10.1029/2020WR028831.
Farley, B., S. R. Mounce, and J. B. Boxall. 2010. “Field testing of an optimal sensor placement methodology for event detection in an urban water distribution network.” Urban Water J. 7 (6): 345–356. https://doi.org/10.1080/1573062X.2010.526230.
Fuqua, D., and T. Razzaghi. 2020. “A cost-sensitive convolution neural network learning for control chart pattern recognition.” Expert Syst. Appl. 150 (Jul): 113275. https://doi.org/10.1016/j.eswa.2020.113275.
Gauri, S. K. 2010. “Control chart pattern recognition using feature-based learning vector quantization.” Int. J. Adv. Manuf. Technol. 48 (9): 1061–1073. https://doi.org/10.1007/s00170-009-2354-7.
Gauri, S. K., and S. Chakraborty. 2009. “Recognition of control chart patterns using improved selection of features.” Comput. Ind. Eng. 56 (4): 1577–1588. https://doi.org/10.1016/j.cie.2008.10.006.
Ghiassi, M., D. Zimbra, and H. Saidane. 2008. “Urban water demand forecasting with a dynamic artificial neural network model.” J. Water Resour. Plann. Manage. 134 (2): 138–146. https://doi.org/10.1061/(ASCE)0733-9496(2008)134:2(138).
Guo, G., S. Liu, Y. Wu, J. Li, R. Zhou, and X. Zhu. 2018. “Short-term water demand forecast based on deep learning method.” J. Water Resour. Plann. Manage. 144 (12): 04018076. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000992.
Guo, G., X. Yu, S. Liu, Z. Ma, Y. Wu, X. Xu, X. Wang, K. Smith, and X. Wu. 2021. “Leakage detection in water distribution systems based on time–frequency convolutional neural network.” J. Water Resour. Plann. Manage. 147 (2): 04020101. https://doi.org/10.1061/(ASCE)WR.1943-5452.0001317.
Hassan, A., M. S. N. Baksh, A. M. Shaharoun, and H. Jamaluddin. 2003. “Improved SPC chart pattern recognition using statistical features.” Int. J. Prod. Res. 41 (7): 1587–1603. https://doi.org/10.1080/0020754021000049844.
House-Peters, L. A., and H. Chang. 2011. “Urban water demand modeling: Review of concepts, methods, and organizing principles.” Water Resour. Res. 47 (5): 1–15. https://doi.org/10.1029/2010WR009624.
Hu, Z., B. Chen, W. Chen, D. Tan, and D. Shen. 2021. “Review of model-based and data-driven approaches for leak detection and location in water distribution systems.” Water Sci. Technol. Water Supply 21 (7): 3282–3306. https://doi.org/10.2166/ws.2021.101.
Hutton, C., and Z. Kapelan. 2015. “Real-time burst detection in water distribution systems using a Bayesian demand forecasting methodology.” Procedia Eng. 119 (Jan): 13–18. https://doi.org/10.1016/j.proeng.2015.08.847.
Islam, M. S., R. Sadiq, M. J. Rodriguez, A. Francisque, H. Najjaran, and M. Hoorfar. 2011. “Leakage detection and location in water distribution systems using a fuzzy-based methodology.” Urban Water J. 8 (6): 351–365. https://doi.org/10.1080/1573062X.2011.617829.
Jung, D., D. Kang, J. Liu, and K. Lansey. 2015. “Improving the rapidity of responses to pipe burst in water distribution systems: A comparison of statistical process control methods.” J. Hydroinf. 17 (2): 307–328. https://doi.org/10.2166/hydro.2014.101.
Jung, D., and J. H. Kim. 2017. “Robust meter network for water distribution pipe burst detection.” Water 9 (11): 820. https://doi.org/10.3390/w9110820.
Jung, D., and K. Lansey. 2015. “Water distribution system burst detection using a nonlinear Kalman filter.” J. Water Resour. Plann. Manage. 141 (5): 04014070. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000464.
Lecun, Y., Y. Bengio, and G. Hinton. 2015. “Deep learning.” Nature 521 (7553): 436–444. https://doi.org/10.1038/nature14539.
Li, R., H. Huang, K. Xin, and T. Tao. 2015. “A review of methods for burst/leakage detection and location in water distribution systems.” Water Sci. Technol. Water Supply 15 (3): 429–441. https://doi.org/10.2166/ws.2014.131.
Loureiro, D., C. Amado, A. Martins, D. Vitorino, A. Mamade, and S. T. Coelho. 2016. “Water distribution systems flow monitoring and anomalous event detection: A practical approach.” Urban Water J. 13 (3): 242–252. https://doi.org/10.1080/1573062X.2014.988733.
Mitchell, R., and E. Frank. 2017. “Accelerating the XGBoost algorithm using GPU computing.” Peer J. Comput. Sci. 3 (Jul): e127. https://doi.org/10.7717/peerj-cs.127.
Mouatadid, S., and J. Adamowski. 2017. “Using extreme learning machines for short-term urban water demand forecasting.” Urban Water J. 14 (6): 630–638. https://doi.org/10.1080/1573062X.2016.1236133.
Mounce, S. R., R. B. Mounce, and J. B. Boxall. 2010. “Novelty detection for time series data analysis in water distribution systems using support vector machines.” J. Hydroinf. 13 (4): 672–686. https://doi.org/10.2166/hydro.2010.144.
Puust, R., Z. Kapelan, D. A. Savic, and T. Koppel. 2010. “A review of methods for leakage management in pipe networks.” Urban Water J. 7 (1): 25–45. https://doi.org/10.1080/15730621003610878.
Romano, M., and Z. Kapelan. 2014. “Adaptive water demand forecasting for near real-time management of smart water distribution systems.” Environ. Modell. Software 60 (7): 265–276. https://doi.org/10.1016/j.envsoft.2014.06.016.
Romano, M., Z. Kapelan, and D. A. Savić. 2014. “Automated detection of pipe bursts and other events in water distribution systems.” J. Water Resour. Plann. Manage. 140 (4): 457–467. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000339.
Stephens, M., J. Gong, C. Zhang, A. Marchi, L. Dix, and M. F. Lambert. 2020. “Leak-before-break main failure prevention for water distribution pipes using acoustic smart water technologies: Case study in Adelaide.” J. Water Resour. Plann. Manage. 146 (10): 05020020. https://doi.org/10.1061/(ASCE)WR.1943-5452.0001266.
Tian, D., C. J. Martinez, and T. Asefa. 2016. “Improving short-term urban water demand forecasts with reforecast analog ensembles.” J. Water Resour. Plann. Manage. 142 (6): 04016008. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000632.
Tiwari, M. K., and J. F. Adamowski. 2017. “An ensemble wavelet bootstrap machine learning approach to water demand forecasting: A case study in the city of Calgary, Canada.” Urban Water J. 14 (2): 185–201. https://doi.org/10.1080/1573062X.2015.1084011.
Wu, Y., and S. Liu. 2017. “A review of data-driven approaches for burst detection in water distribution systems.” Urban Water J. 14 (9): 972–983. https://doi.org/10.1080/1573062X.2017.1279191.
Wu, Y. P., and S. M. Liu. 2020. “Burst detection by analyzing shape similarity of time series subsequences in district metering areas.” J. Water Resour. Plann. Manage. 146 (1): 04019068. https://doi.org/10.1061/(ASCE)WR.1943-5452.0001141.
Wu, Z. Y., and Y. K. He. 2021. “Time series data decomposition-based anomaly detection and evaluation framework for operational management of smart water grid.” J. Water Resour. Plann. Manage. 147 (9): 04021059. https://doi.org/10.1061/(ASCE)WR.1943-5452.0001433.
Wu, Z. Y., and P. Sage. 2006. “Water loss detection via genetic algorithm optimization-based model calibration.” In Proc., Water Distribution Systems Analysis Symp., 1–11. Reston, VA: ASCE. https://doi.org/10.1061/40941(247)180.
Ye, G., and R. A. Fenner. 2011. “Kalman filtering of hydraulic measurements for burst detection in water distribution systems.” J. Pipeline Syst. Eng. Pract. 2 (1): 14–22. https://doi.org/10.1061/(ASCE)PS.1949-1204.0000070.
Zeng, W., J. Gong, A. R. Simpson, B. S. Cazzolato, A. C. Zecchin, and M. F. Lambert. 2020. “Paired-IRF method for detecting leaks in pipe networks.” J. Water Resour. Plann. Manage. 146 (5): 04020021. https://doi.org/10.1061/(ASCE)WR.1943-5452.0001193.
Zhou, X., Z. Tang, W. Xu, F. Meng, X. Chu, K. Xin, and G. Fu. 2019. “Deep learning identifies accurate burst locations in water distribution networks.” Water Res. 166 (Dec): 115058. https://doi.org/10.1016/j.watres.2019.115058.
Zubaidi, S. L., S. K. Gharghan, J. Dooley, R. M. Alkhaddar, and M. Abdellatif. 2018. “Short-term urban water demand prediction considering weather factors.” Water Resour. Manage. 32 (14): 4527–4542. https://doi.org/10.1007/s11269-018-2061-y.
Information & Authors
Information
Published In
Journal of Water Resources Planning and Management
Volume 148 • Issue 11 • November 2022
Copyright
© 2022 American Society of Civil Engineers.
History
Received: Sep 22, 2021
Accepted: Jul 7, 2022
Published online: Sep 9, 2022
Published in print: Nov 1, 2022
Discussion open until: Feb 9, 2023
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.