Water Supply Pipeline Operation Anomaly Mining and Spatiotemporal Correlation Study
Publication: Journal of Pipeline Systems Engineering and Practice
Volume 15, Issue 4
Abstract
The recurrent manifestation of anomalies in water supply network systems exerts a profound influence on individuals’ daily lives. Despite this impact, contemporary research on urban water supply networks reveals a conspicuous lack in the thorough examination of spatiotemporal patterns and the relevance of these anomalies. This investigation meticulously scrutinizes anomalies within a specified segment of the water supply pipe network located in a county in southwest China. Clustering algorithms [-means and density-based spatial clustering of applications with noise (DBSCAN)] and statistical methods (standard deviation) identify anomalous water pressure. Subsequently, the Apriori algorithm is utilized to extract association rules for different types of anomalies, and these rules are compared with user similarity, quantified through standard Euclidean distance. The key findings are as follows. First, anomalies in water pressure are predominantly concentrated in May, September, and November. On a 24-h scale, the highest incidence of anomalies occurs between 6:00 a.m. and 9:00 a.m. Areas with the highest anomaly occurrence are primarily situated near the city center and the railway station. Second, correlation rules exist among occurrences of anomalous values at various monitoring sites within the study area. In concrete terms, identical water pressure abnormal types frequently co-occur (confidence level , support level ) at diverse monitoring sites, with this correlation linked to the types of users around the monitoring sites. Finally, the categorization of anomalies results in significantly enhanced accuracy in correlation rule outcomes, surpassing the comprehensive analysis of anomalies overall.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code used during the study were provided by Chengdu Tongfei Technology Co. Direct request for these materials may be made to the provider as indicated in the Acknowledgments.
Acknowledgments
The authors would like to acknowledge the Science and Technology Cooperation Project of CNPC-SWPU Innovation Alliance (No. 2020CX020000).
References
Abokifa, A. A., K. Haddad, C. Lo, and P. Biswas. 2019. “Real-time identification of cyber-physical attacks on water distribution systems via machine learning–based anomaly detection techniques.” J. Water Resour. Plann. Manage. 145 (1): 04018089. https://doi.org/10.1061/(ASCE)WR.1943-5452.0001023.
Aflori, C., and M. Craus. 2007. “Grid implementation of the Apriori algorithm.” Adv. Eng. Software 38 (5): 295–300. https://doi.org/10.1016/j.advengsoft.2006.08.011.
Ansari, E., M. H. Sadreddini, S. M. H. Mirsadeghi, M. Keshtkaran, and R. Wallace. 2018. “TFI-Apriori: Using new encoding to optimize the Apriori algorithm.” Intell. Data Anal. 22 (4): 807–827. https://doi.org/10.3233/IDA-173473.
Arafa, A., N. El-Fishawy, M. Badawy, and M. Radad. 2022. “RN-SMOTE: Reduced noise SMOTE based on DBSCAN for enhancing imbalanced data classification.” J. King Saud Univ. Comput. Inf. Sci. 34 (8): 5059–5074. https://doi.org/10.1016/j.jksuci.2022.06.005.
Aslan, M. E., and S. Onut. 2022. “Detection of outliers and extreme events of ground level particulate matter using DBSCAN algorithm with local parameters.” Water Air Soil Pollut. 233 (6): 203. https://doi.org/10.1007/s11270-022-05679-6.
Belkhouja, T., Y. Yan, and J. R. Doppa. 2023. “Dynamic time warping based adversarial framework for time-series domain.” IEEE Trans. Pattern Anal. Mach. Intell. 45 (6): 7353–7366. https://doi.org/10.1109/TPAMI.2022.3224754.
Bruno, O. M., and L. D. F. Costa. 2004. “A parallel implementation of exact Euclidean distance transform based on exact dilations.” Microprocess. Microsyst. 28 (3): 107–113. https://doi.org/10.1016/j.micpro.2004.01.001.
Bushra, A. A., and G. Yi. 2021. “Comparative analysis review of pioneering DBSCAN and successive density-based clustering algorithms.” IEEE Access 9 (Apr): 87918–87935. https://doi.org/10.1109/ACCESS.2021.3089036.
Chen, Q., J. Qu, R. Liu, and W. Li. 2008. “Modelling of ageing leakage patterns in Beijing’s water supply pipe network.” China Water Wastewater 11 (Apr): 52–56. https://doi.org/10.3321/j.issn:1000-4602.2008.11.014.
Cho, H. Y., J. H. Oh, K. O. Kim, and J. S. Shim. 2013. “Outlier detection and missing data filling methods for coastal water temperature data.” J. Coastal Res. 165 (10065): 1898–1903. https://doi.org/10.2112/SI65-321.1.
Chu, Z. 2019. “Research on leakage location of water supply network based on hydraulic model.” Master’s thesis, Heifei, Anhui Jianzhu Univ.
Civera, M., L. Sibille, L. Zanotti Fragonara, and R. Ceravolo. 2023. “A DBSCAN-based automated operational modal analysis algorithm for bridge monitoring.” Measurement 208 (Feb): 112451. https://doi.org/10.1016/j.measurement.2023.112451.
Fang, S., L. Huang, Y. Wan, W. Sun, and J. Xu. 2020. “Outlier detection for water supply data based on joint auto-encoder.” CMC-Comput. Mater. Continua 64 (1): 541–555. https://doi.org/10.32604/cmc.2020.010066.
Feng, S., and W. Xiao. 2008. “An improved DBSCAN clustering algorithm.” J. China Univ. Min. Technol. 1 (Sep): 105–111.
Giustolisi, O., D. Savić, M. Romano, and D. Laucelli. 2016. “Detecting anomalies in water distribution networks using EPR modelling paradigm.” J. Hydroinf. 18 (3): 409–427. https://doi.org/10.2166/hydro.2015.113.
Gomes, S. C., S. Vinga, and R. Henriques. 2021. “Spatiotemporal correlation feature spaces to support anomaly detection in water distribution networks.” Water 13 (18): 2551. https://doi.org/10.3390/w13182551.
Hu, X., Y. Han, B. Yu, Z. Geng, and J. Fan. 2021. “Novel leakage detection and water loss management of urban water supply network using multiscale neural networks.” J. Cleaner Prod. 278 (Jan): 123611. https://doi.org/10.1016/j.jclepro.2020.123611.
Hu, X. L., L. C. Zhang, and Z. X. Wang. 2018. “An adaptive smartphone anomaly detection model based on data mining.” EURASIP J. Wireless Commun. Networking 2018 (1): 1–10. https://doi.org/10.1186/s13638-018-1158-6.
Hu, Z., W. Chen, H. Wang, P. Tian, and D. Shen. 2022. “Integrated data-driven framework for anomaly detection and early warning in water distribution system.” J. Cleaner Prod. 373 (Nov): 133977. https://doi.org/10.1016/j.jclepro.2022.133977.
Huang, C., W. Li, X. Zhang, Y. Sun, and H. Wei. 2018. “Pattern recognition of water use anomalies in urban water supply pipe network area.” J. Yunnan Univ. 40 (5): 879–885.
Huang, X., T. Ma, C. Liu, and S. Liu. 2023. “GriT-DBSCAN: A spatial clustering algorithm for very large databases.” Pattern Recognit. 142 (Oct): 109658. https://doi.org/10.1016/j.patcog.2023.109658.
Jian, C., J. Gao, and Y. Xu. 2022. “Anomaly detection and classification in water distribution networks integrated with hourly nodal water demand forecasting models and feature extraction technique.” J. Water Resour. Plann. Manage. 148 (11): 04022059. https://doi.org/10.1061/(ASCE)WR.1943-5452.0001616.
Kammoun, M., A. Kammoun, and M. Abid. 2022. “Experiments based comparative evaluations of machine learning techniques for leak detection in water distribution systems.” Water Supply 22 (1): 628–642. https://doi.org/10.2166/ws.2021.248.
Kim, S., S. Jun, and D. Jung. 2022. “Ensemble CNN model for effective pipe burst detection in water distribution systems.” Water Resour. Manage. 36 (13): 5049–5061. https://doi.org/10.1007/s11269-022-03291-1.
Kuang, H., R. Qin, M. He, X. He, R. Duan, C. Guo, and X. Meng. 2021. “An association rules-based method for outliers cleaning of measurement data in the distribution network.” Front. Energy Res. 9 (Oct): 730058. https://doi.org/10.3389/fenrg.2021.730058.
Li, J., and B.-L. Lu. 2009. “An adaptive image Euclidean distance.” Pattern Recognit. 42 (3): 349–357. https://doi.org/10.1016/j.patcog.2008.07.017.
Li, L., H. Chen, and A. M. Ahmed. 2023a. “Artificial intelligence and internet of things-based leak detection method for the water supply network.” Int. Trans. Electr. Energy Syst. 2023 (Aug): 1–11. https://doi.org/10.1155/2023/3443047.
Li, L., Y. Wu, Y. Huang, B. Li, F. Miao, and Z. Deng. 2023b. “Optimized Apriori algorithm for deformation response analysis of landslide hazards.” Comput. Geosci. 170 (Jan): 105261. https://doi.org/10.1016/j.cageo.2022.105261.
Li, S., N. Qin, D. Huang, D. Huang, and L. Ke. 2020. “Damage localization of stacker’s track based on EEMD-EMD and DBSCAN cluster algorithms.” IEEE Trans. Instrum. Meas. 69 (5): 1981–1992. https://doi.org/10.1109/TIM.2019.2919375.
Li, W., W. Ling, S. Liu, J. Zhao, R. Liu, Q. Chen, Z. Qiang, and J. Qu. 2011. “Development of systems for detection, early warning, and control of pipeline leakage in drinking water distribution: A case study.” J. Environ. Sci. 23 (11): 1816–1822. https://doi.org/10.1016/S1001-0742(10)60577-3.
Liu, H., J. Chen, J. Dy, and Y. Fu. 2023. “Transforming complex problems into K-means solutions.” IEEE Trans. Pattern Anal. Mach. Intell. 45 (7): 9149–9168. https://doi.org/10.1109/TPAMI.2023.3237667.
Liu, M., J. Yang, S. Li, Z. Zhou, E. Fan, and W. Zheng. 2022. “Robust GMM least square twin K-class support vector machine for urban water pipe leak recognition.” Expert Syst. Appl. 195 (Jun): 116525. https://doi.org/10.1016/j.eswa.2022.116525.
Majid, A., J. E. van Zyl, and J. W. Hall. 2022. “The influence of temporal variability and reservoir management on demand-response in the water sector.” Appl. Energy 305 (Jan): 117808. https://doi.org/10.1016/j.apenergy.2021.117808.
Mendoza García, C. D., and C. J. Navarro Gómez. 2022. “Study of domestic water consumption in intermittent supply of the Riberas de Sacramento sector in Chihuahua, Mexico.” Water Supply 22 (4): 4728–4743. https://doi.org/10.2166/ws.2022.135.
Nofal, S., A. Alfarrarjeh, and A. Abu Jabal. 2022. “A use case of anomaly detection for identifying unusual water consumption in Jordan.” Water Supply 22 (1): 1131–1140. https://doi.org/10.2166/ws.2021.210.
Olukanmi, P., F. Nelwamondo, T. Marwala, and B. Twala. 2022. “Automatic detection of outliers and the number of clusters in k-means clustering via Chebyshev-type inequalities.” Neural Comput. Appl. 34 (8): 5939–5958. https://doi.org/10.1007/s00521-021-06689-x.
Pala, O. 2023. “A new objective weighting method based on robustness of ranking with standard deviation and correlation: The ROCOSD method.” Inf. Sci. 636 (Jul): 118930. https://doi.org/10.1016/j.ins.2023.04.009.
Papi, R., S. Attarchi, A. Darvishi Boloorani, and N. Neysani Samany. 2022. “Knowledge discovery of Middle East dust sources using Apriori spatial data mining algorithm.” Ecol. Inf. 72 (Dec): 101867. https://doi.org/10.1016/j.ecoinf.2022.101867.
Peng, W., J. Gao, S. Hu, T. Zhang, Y. Jing, and Y. Pang. 2020. “Study on the improvement of pressure-driven model based on residents’ water use behaviour.” Water Wastewater Eng. 56 (S2): 53–58. https://doi.org/10.13789/j.cnki.wwe1964.2020.S2.009.
Qahtani, T. A., M. S. Yaakob, N. Yidris, S. Sulaiman, and K. A. Ahmad. 2020. “A review on water leakage detection method in the water distribution network.” J. Adv. Res. Fluid Mech. Therm. Sci. 68 (2): 152–163. https://doi.org/10.37934/arfmts.68.2.152163.
Qi, Z., F. Zheng, D. Guo, T. Zhang, Y. Shao, T. Yu, K. Zhang, and H. R. Maier. 2018. “A comprehensive framework to evaluate hydraulic and water quality impacts of pipe breaks on water distribution systems.” Water Resour. Res. 54 (10): 8174–8195. https://doi.org/10.1029/2018WR022736.
Raj, S., D. Ramesh, and K. K. Sethi. 2020. “A spark-based Apriori algorithm with reduced shuffle overhead.” J. Supercomput. 77 (1): 133–151. https://doi.org/10.1007/s11227-020-03253-7.
Ray, B., S. Ghosh, S. Ahmed, R. Sarkar, and M. Nasipuri. 2021. “Outlier detection using an ensemble of clustering algorithms.” Multimedia Tools Appl. 81 (2): 2681–2709. https://doi.org/10.1007/s11042-021-11671-9.
Schubert, E., J. Sander, M. Ester, H. P. Kriegel, and X. Xu. 2017. “DBSCAN revisited, revisited.” ACM Trans. Database Syst. 42 (3): 1–21. https://doi.org/10.1145/3068335.
Selvi, C., and E. Sivasankar. 2018. “A novel adaptive genetic neural network (AGNN) model for recommender systems using modified k-means clustering approach.” Multimedia Tools Appl. 78 (11): 14303–14330. https://doi.org/10.1007/s11042-018-6790-y.
Shrifan, N. H. M. M., M. F. Akbar, and N. A. M. Isa. 2022. “An adaptive outlier removal aided k-means clustering algorithm.” J. King Saud Univ. Comput. Inf. Sci. 34 (8): 6365–6376. https://doi.org/10.1016/j.jksuci.2021.07.003.
Singh, P. K., E. Othman, R. Ahmed, A. Mahmood, H. Dhahri, and P. Choudhury. 2021. “Optimized recommendations by user profiling using apriori algorithm.” Appl. Soft Comput. 106 (Jul): 107272. https://doi.org/10.1016/j.asoc.2021.107272.
Tashman, Z., C. Gorder, S. Parthasarathy, M. M. Nasr-Azadani, and R. Webre. 2020. “Anomaly detection system for water networks in Northern Ethiopia using Bayesian inference.” Sustainability 12 (7): 2897. https://doi.org/10.3390/su12072897.
Ultsch, A., and J. Lotsch. 2022. “Euclidean distance-optimized data transformation for cluster analysis in biomedical data (EDOtrans).” BMC Bioinf. 23 (1): 233. https://doi.org/10.1186/s12859-022-04769-w.
Wang, J., Z. Cao, G. Xu, J. Yang, and A. Kareem. 2023. “An adaptive kriging method based on K-means clustering and sampling in n-ball for structural reliability analysis.” Eng. Comput. 40 (2): 378–410. https://doi.org/10.1108/EC-12-2021-0705.
Wu, Z. Y., and Y. He. 2021. “Time series data decomposition-based anomaly detection and evaluation framework for operational management of smart water grid.” J. Water Resour. Plann. Manage. 147 (9): 04021059 https://doi.org/10.1061/(ASCE)WR.1943-5452.0001433.
Yan, J., and T. Tao. 2022. “Unsupervised anomaly detection in hourly water demand data using an asymmetric encoder–decoder model.” J. Hydrol. 613 (Aug): 128389. https://doi.org/10.1016/j.jhydrol.2022.128389.
Yan, L., F. Qian, and W. Li. 2018. “Research on key parameters operation range of central air conditioning based on binary K-means and apriori algorithm.” Energies 12 (1): 102. https://doi.org/10.3390/en12010102.
Yang, M.-S., and I. Hussain. 2023. “Unsupervised multi-view K-means clustering algorithm.” IEEE Access 11 (Aug): 13574–13593. https://doi.org/10.1109/ACCESS.2023.3243133.
Yu, H. 2021. “Apriori algorithm optimization based on Spark platform under big data.” Microprocess. Microsyst. 80 (Jul): 103528. https://doi.org/10.1016/j.micpro.2020.103528.
Yu, T., X. Chen, W. Yan, Z. Xu, and M. Ye. 2023. “Leak detection in water distribution systems by classifying vibration signals.” Mech. Syst. Signal Process. 185 (Feb): 109810. https://doi.org/10.1016/j.ymssp.2022.109810.
Yuan, X. 2017. “An improved Apriori algorithm for mining association rules.” In Proc., AIP Conf. Shanghai, China: Shanghai Pujiang Program.
Zhang, P., Y. Wang, L. Liang, X. Li, and Q. Duan. 2020. “Short-term wind power prediction using GA-BP neural network based on DBSCAN algorithm outlier identification.” Processes 8 (2): 157. https://doi.org/10.3390/pr8020157.
Zhao, Z., Z. Zhao, and P. Zhang. 2023. “A new method for identifying industrial clustering using the standard deviational ellipse.” Sci. Rep. 13 (1): 578. https://doi.org/10.1038/s41598-023-27655-8.
Zubaidi, S. L., S. Ortega-Martorell, H. Al-Bugharbee, I. Olier, K. S. Hashim, S. K. Gharghan, P. Kot, and R. Al-Khaddar. 2020. “Urban water demand prediction for a city that suffers from climate change and population growth: Gauteng province case study.” Water 12 (7): 1885. https://doi.org/10.3390/w12071885.
Information & Authors
Information
Published In
Journal of Pipeline Systems Engineering and Practice
Volume 15 • Issue 4 • November 2024
Copyright
© 2024 American Society of Civil Engineers.
History
Received: Sep 16, 2023
Accepted: Feb 9, 2024
Published online: Jul 9, 2024
Published in print: Nov 1, 2024
Discussion open until: Dec 9, 2024
ASCE Technical Topics:
- Algorithms
- Continuum mechanics
- Correlation
- Dynamics (solid mechanics)
- Engineering fundamentals
- Engineering mechanics
- Infrastructure
- Mathematics
- Pipe networks
- Pipeline systems
- Pipelines
- Pipes
- Pressure (type)
- Solid mechanics
- Statistics
- Water and water resources
- Water management
- Water pipelines
- Water pressure
- Water supply
- Water supply systems
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.