Time Series Data Decomposition-Based Anomaly Detection and Evaluation Framework for Operational Management of Smart Water Grid
Publication: Journal of Water Resources Planning and Management
Volume 147, Issue 9
Abstract
With the increasing adoption of advanced meter infrastructure (AMI), smarter sensors, and temporary and/or permanent data loggers, it is imperative to leverage data analytics methods with hydraulic modeling to improve the quality and efficiency of water service. One important task is to timely detect and evaluate anomaly events so that corresponding actions can be taken to prevent and mitigate the impact of possible water service disruption, which may be caused by the anomaly incidents including but not limited to pipe bursts and unauthorized water usages. In this paper, a comprehensive analysis framework is developed for anomaly event detection and evaluation by developing an integrated solution, which is implemented in multiple components including: (1) data-preprocess or cleansing to eliminate and correct error data records; (2) decomposition of time series data to ensure data stationarity; (3) outlier detection by statistical process control methods with stationary time series; (4) classification of system anomaly events by either correlation analysis of high-flow events with low-pressure events or high-flow outliers with low-pressure outliers; and (5) quantitative evaluation of the system anomaly events with field reported leak incidents. The solution framework has been applied to the water supply zone that is permanent monitored with the flow meter at the inlet and 12 pressure stations throughout the zone with more than 8,000 pipes. Analysis has been conducted with one-year monitoring data and 106 historical leak records, which are employed to validate 526 detected anomaly events. Among them, a 75% true positive rate has been achieved and 90% of 106 field events have been successfully detected with a lead time of more than 24 h. The results obtained indicate that the developed solution method is effective at facilitating the operational management of a smart water grid by maximizing the return of investment in continuously monitoring water distribution networks.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
All data, models, or code generated or used during the study are proprietary or confidential in nature and may only be provided with restrictions.
Acknowledgments
Authors would like to thank the colleagues from Water Supply Network Department of public utility agency for providing the datasets, hydraulic model and insightful responses to our enquiries during the study. Their support and assistance are critical for authors to complete the research, thus gratefully appreciated. The contents of this paper reflect the views of the authors, who are responsible for the facts and accuracy of the results presented herein. The contents do not reflect the official views or policies of the utility. This project is supported by the Singapore National Research Foundation under its Competitive Research Programme (CRP) (Water) and administered by PUB (PUB-1804-0087), Singapore’s national water agency.
References
Alegre, H., W. Hirner, J. M. Baptista, and R. Parena. 2000. “Performance indicators for water supply services.” In IWA manual of best practice. London: IWA Publishing.
Cleveland, R. B., W. S. Cleveland, J. E. Mcrae, and I. Terpenning. 1990. “STL: A seasonal-trend decomposition procedure based on loess.” J. Off. Stat. 6 (1): 3–73.
Collins, R. 2017. CCWI 2017: Computing and Control in the Water Industry Conference. Sheffield, UK: Univ. of Sheffield. https://doi.org/10.15131/shef.data.c.3867985.v1.
Hochenbaum, J., A. Kejariwal, and O. S. Vallis. 2017. “Automatic anomaly detection in the cloud via statistical learning.” Preprint, submitted April 24, 2017. http://arxiv.org/abs/1704.07706.
Huttona, C. J., and Z. Kapelan. 2015. “Real-time burst detection in water distribution systems using a Bayesian demand forecasting methodology.” Procedia Eng. 119: 13–18.
Jung, D., and K. Lansey. 2015. “Water distribution system burst detection using a nonlinear kalman filter.” J. Water Resour. Plann. Manage. 141 (5): 04014070. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000464.
Li, Q., Z. Y. Wu, and A. Rahman. 2019. “Evolutionary deep learning with extended Kalman filter for effective prediction modeling and efficient data assimilation.” J. Comput. Civ. Eng. 33 (3): 04019014. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000835.
Loureiro, D., C. Amado, A. Martins, D. Vitorino, A. Mamade, and S. T. Coelho. 2016. “Water distribution systems flow monitoring and anomalous event detection: A practical approach.” Urban Water J. 13 (3): 242–252. https://doi.org/10.1080/1573062X.2014.988733.
MacDonald, G., and C. D. Yates. 2005. “DMA design and implementation, a North American context.” In Proc., Leakage 2005 Conf. Halifax, NS: IWA Publishing.
Mounce, S., J. Boxall, and J. Machell. 2010. “Development and verification of an online artificial intelligence system for detection of bursts and other abnormal flows.” J. Water Resour. Plann. Manage. 136 (5): 309–318. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000030.
Mounce, S., A. Day, A. Wood, A. D. Khan, P. Widdop, and J. Machell. 2002. “A neural network approach to burst detection.” Water Sci. Technol. 45 (4–5): 237–246. https://doi.org/10.2166/wst.2002.0595.
Mounce, S., A. Khan, A. Wood, A. D. Day, P. Widdop, and J. Machell. 2003. “Sensor-fusion of hydraulic data for burst detection and location in a treated water distribution system.” Inf. Fusion 4 (3): 217–229. https://doi.org/10.1016/S1566-2535(03)00034-4.
Mounce, S. R., and J. Boxall. 2011. “Online monitoring and detection.” In Water loss reduction, edited by Z. Y. Wu. Exton, PA: Bentley Institute Press.
Mounce, S. R., R. B. Mounce, and J. B. Boxall. 2011. “Novelty detection for time series data analysis in water distribution systems using support vector machines.” J. Hydroinf. 13 (4): 672–686. https://doi.org/10.2166/hydro.2010.144.
Mounce, S. R., R. B. Mounce, T. Jackson, J. Austin, and J. B. Boxall. 2014. “Pattern matching and associative artificial neural networks for water distribution system time series data analysis.” J. Hydroinf. 16 (3): 617–632. https://doi.org/10.2166/hydro.2013.057.
Mourad, M., and J. L. Bertrand-Krajewski. 2002. “A method for automatic validation of long time series of data in urban hydrology.” Water Sci. Technol. 45 (4–5): 263–270. https://doi.org/10.2166/wst.2002.0601.
PUB (Public Utilities Board). 2016. “Managing the water distribution network with a smart water grid.” Smart Water 1 (Dec): 4. https://doi.org/10.1186/s40713-016-0004-4.
Rodriguez, A., and A. Laio. 2014. “Clustering by fast search and find of density peaks.” Science 344 (6191): 1492–1496. https://doi.org/10.1126/science.1242072.
Romano, M., Z. Kapelan, and D. A. Savic. 2014a. “Automated detection of pipe bursts and other events in water distribution systems.” J. Water Resour. Plann. Manage. 140 (4): 457–467. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000339.
Romano, M., Z. Kaplena, and D. A. Savic. 2014b. “Evolutionary algorithm and expectation maximization strategies for improved detection of pipe bursts and other events in water distribution systems.” J. Water Resour. Plann. Manage. 140 (5): 572–584. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000347.
Shewhart, W. A. 1939. Statistical method from the viewpoint of quality control. Washington, DC: Graduate School.
Tao, T., H. D. Huang, F. Li, and K. L. Xin. 2014. “Burst detection using an artificial immune network in water-distribution systems.” J. Water Resour. Plann. Manage. 140 (10): 04014027. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000405.
Taormina, R., S. Galelli, N. O. Tippenhauer, E. Salomons, A. Ostfeld, D. G. Eliades, and Z. Ohar. 2018. “Battle of the attack detection algorithms: disclosing cyber attacks on water distribution networks.” J. Water Resour. Plann. Manage. 144 (8): 04018048. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000969.
WEF (Water Environment Federation). 2017. “Intelligent water systems: The path to a smart utility.” Accessed May 15, 2019. https://www.wef.org/globalassets/assets-wef/direct-download-library.
Wen, Q., J. Gao, X. Song, L. Sun, H. Xu, and S. Zhu. 2019. “RobustSTL: A robust seasonal-trend decomposition algorithm for long time series.” Accessed October 1, 2020. https://arxiv.org/pdf/1812.01767.pdf.
Wu, Y., and S. Liu. 2017. “a review of data-driven approaches for burst detection in water distribution systems.” Urban Water J. 14 (9): 972–983. https://doi.org/10.1080/1573062X.2017.1279191.
Wu, Y., and S. Liu. 2018. “Distance-based burst detection using multiple pressure sensors in district metering areas.” J. Water Resour. Plann. Manage. 144 (11): 06018009. https://doi.org/10.1061/(ASCE)WR.1943-5452.0001001.
Wu, Y., S. Liu, K. Smith, and X. Wang. 2018a. “Using correlation between data from multiple monitoring sensors to detect bursts in water distribution systems” J. Water Resour. Plann. Manage. 144 (2): 04017084. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000870.
Wu, Z. Y. 2009. “A unified approach for leakage detection and extended period model calibration of water distribution systems.” Urban Water J. 6 (1): 53–67. https://doi.org/10.1080/15730620802541631.
Wu, Z. Y. 2011. “Leakage detection via hydraulic model calibration.” In Water loss reduction, edited by Z. Y. Wu. Exton, PA: Bentley Institute Press.
Wu, Z. Y., Y. He, and Q. Li. 2018b. “Comparing deep learning with statistical control methods for anomaly detection.” In Proc., 1st Int. WDSA/CCWI 2018 Joint Conf. Reston, VA: ASCE. https://ojs.library.queensu.ca/index.php/wdsa-ccw/article/view/12495.
Wu, Z. Y., P. Sage, and D. Turtle. 2010. “Pressure dependent leakage detection approach and its application to district water systems.” J. Water Resour. Plann. Manage. 136 (1): 116–128. https://doi.org/10.1061/(ASCE)0733-9496(2010)136:1(116).
Ye, G., and A. Fenner. 2014. “weighted least square with expectation-maximization algorithm for burst detection in U.K. Water distribution systems.” J. Water Resour. Plann. Manage. 140 (4): 417–424. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000344.
Information & Authors
Information
Published In
Journal of Water Resources Planning and Management
Volume 147 • Issue 9 • September 2021
Copyright
© 2021 American Society of Civil Engineers.
History
Received: Aug 20, 2020
Accepted: Apr 1, 2021
Published online: Jul 7, 2021
Published in print: Sep 1, 2021
Discussion open until: Dec 7, 2021
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.