Technical Papers
Sep 25, 2020

Drinking Water Distribution System Network Clustering Using Self-Organizing Map for Real-Time Demand Estimation

Publication: Journal of Water Resources Planning and Management
Volume 146, Issue 12

Abstract

Consumer demand estimation is a key step in real-time drinking water system (DWS) modeling used for demand forecasting, optimal operations, and water quality management. Consumer nodes in a DWS are generally clustered to reduce the number of unknown demands to be estimated from a limited number of measurement locations. A clustering methodology using the self-organizing map (SOM) is presented, which groups consumer nodes based on sensitivity of measurements to perturbations in the consumer demands and through the use of exogenous consumer information representative of, for example, socioeconomic information. The SOM algorithm not only developed demand clusters, but also provided intuitive visualization of the high-dimensional sensitivity space, which can provide important visual clues about the clustering problem such as the maximum number of clusters that can reasonably be formed and sharpness of the clusters. When applied to an example network, the sensitivity-based SOM clusters improved the performance in representing the observed measurements and demand estimate uncertainty, but reduced the performance in representing the overall network hydraulics relative to the actual clusters. Incorporating exogenous information about the actual clusters demonstrated the potential for providing trade-offs between representing the limited observed hydraulic information and the overall network hydraulics. The results from the SOM algorithm clearly demonstrate a need for clustering approaches that incorporate network-specific information (e.g., measurement locations, sensitivity information, and exogenous data) to develop demand estimates that are capable of representing observed information while adequately capturing overall system dynamics.

Get full access to this article

View all available purchase options and get full access to this article.

Data Availability Statement

All data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request. The items that are required (and available upon request) to support the findings are (1) software codes, and (2) EPANET 2.0 network files.

Acknowledgments

The authors would like to gratefully acknowledge the partial funding support provided by the NSF CBET Directorate, Environmental Engineering Program, through Award No.1511959.

References

AghaeiRad, A., N. Chen, and B. Ribeiro. 2017. “Improve credit scoring using transfer of learned knowledge from self-organizing map.” Neural Comput. Appl. 28 (6): 1329–1342. https://doi.org/10.1007/s00521-016-2567-2.
AWWA (American Water Works Association). 2005. Computer modeling of water distribution systems. 2nd ed. Denver: AWWA.
Barreto, G. A., and A. F. Araujo. 2004. “Identification and control of dynamical systems using the self-organizing map.” IEEE Trans. Neural Networks 15 (5): 1244–1259. https://doi.org/10.1109/TNN.2004.832825.
Bascià, A., and T. Tucciarelli. 2003. “Simultaneous zonation and calibration of pipe network parameters.” J. Hydraul. Eng. 129 (5): 394–403. https://doi.org/10.1061/(ASCE)0733-9429(2003)129:5(394).
Brentan, B., G. Meirelles, E. Luvizotto, and J. Izquierdo. 2018. “Hybrid SOM+k-means clustering to improve planning, operation and management in water distribution systems.” Environ. Modell. Software 106 (Aug): 77–88. https://doi.org/10.1016/j.envsoft.2018.02.013.
Chen, J., and D. L. Boccelli. 2018. “Forecasting hourly water demands with seasonal autoregressive models for real-time application.” Water Resour. Res. 54 (2): 879–894. https://doi.org/10.1002/2017WR022007.
Davidson, J., F. Bouchart, S. Cavill, and P. Jowitt. 2005. “Real-time connectivity modeling of water distribution networks to predict contamination spread.” J. Comput. Civ. Eng. 19 (4): 377–386. https://doi.org/10.1061/(ASCE)0887-3801(2005)19:4(377).
Díaz, S., J. González, and R. Mínguez. 2015. “Observability analysis in water transport networks: Algebraic approach.” J. Water Resour. Plann. Manage. 142 (4): 4015071. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000621.
Do, N. C., A. R. Simpson, J. W. Deuerlein, and O. Piller. 2016. “Calibration of water demand multipliers in water distribution systems using genetic algorithms.” J. Water Resour. Plann. Manage. 142 (11): 04016044. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000691.
Eliades, D. G., and M. M. Polycarpou. 2012. “Water contamination impact evaluation and source-area isolation using decision trees.” J. Water Resour. Plann. Manage. 138 (5): 562–570. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000203.
Gibson, P. B., S. E. Perkins-Kirkpatrick, P. Uotila, A. S. Pepler, and L. V. Alexander. 2017. “On the use of self-organizing maps for studying climate extremes.” J. Geophys. Res. Atmos. 122 (7): 3891–3903. https://doi.org/10.1002/2016JD026256.
Haxton, T., and J. Uber. 2010. “Flushing under source uncertainties.” In Proc., 12th Int. Conf., WDSA 2010 on Water Distribution Systems Analysis, 604–612. Reston, VA: ASCE.
Herrero, J., A. Valencia, and J. Dopazo. 2001. “A hierarchical unsupervised growing neural network for clustering gene expression patterns.” Bioinformatics 17 (2): 126–136. https://doi.org/10.1093/bioinformatics/17.2.126.
Jolly, M. D., A. D. Lothes, L. Sebastian Bryson, and L. Ormsbee. 2014. “Research database of water distribution system models.” J. Water Resour. Plann. Manage. 140 (4): 410–416. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000352.
Jung, D., Y. H. Choi, and J. H. Kim. 2016. “Optimal node grouping for water distribution system demand estimation.” Water 8 (4): 160. https://doi.org/10.3390/w8040160.
Kang, D., and K. Lansey. 2009. “Real-time demand estimation and confidence limit analysis for water distribution systems.” J. Hydraul. Eng. 135 (10): 825–837. https://doi.org/10.1061/(ASCE)HY.1943-7900.0000086.
Kohonen, T. 2013. “Essentials of the self-organizing map.” Neural Networks 37 (Jan): 52–65. https://doi.org/10.1016/j.neunet.2012.09.018.
Kohonen, T., E. Oja, O. Simula, A. Visa, and J. Kangas. 1996. “Engineering applications of the self-organizing map.” Proc. IEEE 84 (10): 1358–1384. https://doi.org/10.1109/5.537105.
Mallick, K. N., I. Ahmed, K. S. Tickle, and K. E. Lansey. 2002. “Determining pipe groupings for water distribution networks.” J. Water Resour. Plann. Manage. 128 (2): 130–139. https://doi.org/10.1061/(ASCE)0733-9496(2002)128:2(130).
Marchi, A., G. C. Dandy, D. L. Boccelli, and S. M. M. Rana. 2018. “Assessing the observability of demand pattern multipliers in water distribution systems using algebraic and numerical derivatives.” J. Water Resour. Plann. Manage. 144 (5): 04018014. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000909.
Marchi, A., A. R. Simpson, and M. F. Lambert. 2016. “Optimization of pump operation using rule-based controls in Epanet2: New Ettar toolkit and correction of energy computation.” J. Water Resour. Plann. Manage. 142 (7): 04016012. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000637.
May, R. J., H. R. Maier, and G. C. Dandy. 2010. “Data splitting for artificial neural networks using SOM-based stratified sampling.” Neural Networks 23 (2): 283–294. https://doi.org/10.1016/j.neunet.2009.11.009.
Oliker, N., and A. Ostfeld. 2014. “Minimum volume ellipsoid classification model for contamination event detection in water distribution systems.” Environ. Modell. Software 57 (Jul): 1–12. https://doi.org/10.1016/j.envsoft.2014.03.011.
Oliveira, P., S. Rana, T. Qin, H. Woo, J. Chen, and D. Boccelli. 2016. “Case study: Evaluation of a composite demand-hydraulic modeling framework.” In Proc., 18th Conf. on Water Distribution System Analysis. Bogota, Colombia: Universidad de los Andes.
Oliveira, P. J., J. L. Steffen, and P. Cheung. 2017. “Parameter estimation of seasonal Arima models for water demand forecasting using the harmony search algorithm.” Procedia Eng. 186 (1): 177–185. https://doi.org/10.1016/j.proeng.2017.03.225.
Oliveira, P. J. A., and D. L. Boccelli. 2018. “k-nearest neighbor for short term water demand forecasting.” In World Environmental and Water Resources Congress 2017, 501–510. Reston, VA: ASCE.
Ormsbee, L. E. 1989. “Implicit network calibration.” J. Water Resour. Plann. Manage. 115 (2): 243–257. https://doi.org/10.1061/(ASCE)0733-9496(1989)115:2(243).
Perelman, L. S., M. Allen, A. Preis, M. Iqbal, and A. J. Whittle. 2015. “Automated sub-zoning of water distribution systems.” Environ. Modell. Software 65 (Mar): 1–14. https://doi.org/10.1016/j.envsoft.2014.11.025.
Piller, O., S. Elhay, J. Deuerlein, and A. R. Simpson. 2017. “Local sensitivity of pressure-driven modeling and demand-driven modeling steady-state solutions to variations in parameters.” J. Water Resour. Plann. Manage. 143 (2): 04016074. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000729.
Preis, A., A. J. Whittle, A. Ostfeld, and L. Perelman. 2011. “Efficient hydraulic state estimation technique using reduced models of urban water networks.” J. Water Resour. Plann. Manage. 137 (4): 343–351. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000113.
Qin, T., and D. L. Boccelli. 2017. “Grouping water-demand nodes by similarity among flow paths in water-distribution systems.” J. Water Resour. Plann. Manage. 143 (8): 04017033. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000788.
Qin, T., and D. L. Boccelli. 2019. “Estimating distribution system water demands using Markov chain Monte Carlo.” J. Water Resour. Plann. Manage. 145 (7): 04019023. https://doi.org/10.1061/(ASCE)WR.1943-5452.0001077.
Rana, S. M. M., and D. L. Boccelli. 2016. “Contaminant spread forecasting and confirmatory sampling location identification in a water-distribution system.” J. Water Resour. Plann. Manage. 142 (12): 4016059. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000704.
Rana, S. M. M., D. L. Boccelli, A. Marchi, and G. C. Dandy. 2017. “Impact of measurement location and clustering on real-time demand estimation.” In World Environmental and Water Resources Congress 2017, 602–610. Reston, VA: ASCE.
Romano, M., and Z. Kapelan. 2014. “Adaptive water demand forecasting for near real-time management of smart water distribution systems.” Environ. Modell. Software 60 (Oct): 265–276. https://doi.org/10.1016/j.envsoft.2014.06.016.
Rossman, L. A. 2000. EPANET 2. Cincinnati: USEPA, Water Supply and Water Resources Division, National Risk Management Research Laboratory.
Sanz, G., and R. Pérez. 2015. “Sensitivity analysis for sampling design and demand calibration in water distribution networks using the singular value decomposition.” J. Water Resour. Plann. Manage. 141 (10): 04015020. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000535.
Storn, R., and K. Price. 1997. “Differential evolution—A simple and efficient heuristic for global optimization over continuous spaces.” J. Global Optim. 11 (4): 341–359. https://doi.org/10.1023/A:1008202821328.
Tucciarelli, T., A. Criminisi, and D. Termini. 1999. “Leak analysis in pipeline systems by means of optimal valve regulation.” J. Hydraul. Eng. 125 (3): 277–285. https://doi.org/10.1061/(ASCE)0733-9429(1999)125:3(277).
UKY (University of Kentucky). 2020. “Water distribution system operations.” Accessed August 28, 2020. http://www.uky.edu/WDST/database.html.
Ultsch, A., and H. P. Siemon. 1990. “Kohonen’s self organizing feature maps for exploratory data analysis.” In Proc., Int. Neural Networks Conf., 305–308. Paris: Kluwer Academic.
Ung, H., O. Piller, D. Gilbert, and I. Mortazavi. 2017. “Accurate and optimal sensor placement for source identification of water distribution networks.” J. Water Resour. Plann. Manage. 143 (8): 04017032. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000777.
Vesanto, J., and E. Alhoniemi. 2000. “Clustering of the self-organizing map.” IEEE Trans. Neural Networks 11 (3): 586–600. https://doi.org/10.1109/72.846731.
Vesanto, J., J. Himberg, E. Alhoniemi, and J. Parhankangas. 1999. “Self-organizing map in MATLAB: The Som toolbox.” In Vol. 99 of Proc., MATLAB DSP Conf., 16–17. Espoo, Finland: DSP.
Vrugt, J. A. 2016. “Markov chain Monte Carlo simulation using the dream software package: Theory, concepts, and MATLAB implementation.” Environ. Modell. Software 75 (Jan): 273–316. https://doi.org/10.1016/j.envsoft.2015.08.013.
Wanner, F., W. Jentner, T. Schreck, A. Stoffel, L. Sharalieva, and D. A. Keim. 2016. “Integrated visual analysis of patterns in time series and text data-workflow and application to financial data analysis.” Inf. Visualization 15 (1): 75–90. https://doi.org/10.1177/1473871615576925.
Yang, X., and D. L. Boccelli. 2014. “Bayesian approach for real-time probabilistic contamination source identification.” J. Water Resour. Plann. Manage. 140 (8): 04014019. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000381.
Yang, X., and D. L. Boccelli. 2016. “Model-based event detection for contaminant warning systems.” J. Water Resour. Plann. Manage. 142 (11): 04016048. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000689.
Yang, X., and D. L. Boccelli. 2017. “Integrated systemwide model-based event detection algorithm.” J. Water Resour. Plann. Manage. 143 (8): 04017047. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000801.

Information & Authors

Information

Published In

Go to Journal of Water Resources Planning and Management
Journal of Water Resources Planning and Management
Volume 146Issue 12December 2020

History

Received: Mar 15, 2019
Accepted: Jun 11, 2020
Published online: Sep 25, 2020
Published in print: Dec 1, 2020
Discussion open until: Feb 25, 2021

Permissions

Request permissions for this article.

Authors

Affiliations

Ph.D. Candidate, Environmetal Engineering Program, Dept. of Chemical and Environmental Engineering, Univ. of Cincinnati, Cincinnati, OH 45221 (corresponding author). ORCID: https://orcid.org/0000-0002-7360-6412. Email: [email protected]; [email protected]
Dominic L. Boccelli, A.M.ASCE
Professor, Dept. of Civil and Architectural Engineering and Mechanics, Univ. of Arizona, Tucson, AZ 85721.
Angela Marchi
Lecturer, School of Civil, Environmental and Mining Engineering, Univ. of Adelaide, Adelaide, SA 5005, Australia.
Graeme C. Dandy, M.ASCE
Emeritus Professor, School of Civil, Environmental and Mining Engineering, Univ. of Adelaide, Adelaide, SA 5005, Australia.

Metrics & Citations

Metrics

Citations

Download citation

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited by

View Options

Get Access

Access content

Please select your options to get access

Log in/Register Log in via your institution (Shibboleth)
ASCE Members: Please log in to see member pricing

Purchase

Save for later Information on ASCE Library Cards
ASCE Library Cards let you download journal articles, proceedings papers, and available book chapters across the entire ASCE Library platform. ASCE Library Cards remain active for 24 months or until all downloads are used. Note: This content will be debited as one download at time of checkout.

Terms of Use: ASCE Library Cards are for individual, personal use only. Reselling, republishing, or forwarding the materials to libraries or reading rooms is prohibited.
ASCE Library Card (5 downloads)
$105.00
Add to cart
ASCE Library Card (20 downloads)
$280.00
Add to cart
Buy Single Article
$35.00
Add to cart

Get Access

Access content

Please select your options to get access

Log in/Register Log in via your institution (Shibboleth)
ASCE Members: Please log in to see member pricing

Purchase

Save for later Information on ASCE Library Cards
ASCE Library Cards let you download journal articles, proceedings papers, and available book chapters across the entire ASCE Library platform. ASCE Library Cards remain active for 24 months or until all downloads are used. Note: This content will be debited as one download at time of checkout.

Terms of Use: ASCE Library Cards are for individual, personal use only. Reselling, republishing, or forwarding the materials to libraries or reading rooms is prohibited.
ASCE Library Card (5 downloads)
$105.00
Add to cart
ASCE Library Card (20 downloads)
$280.00
Add to cart
Buy Single Article
$35.00
Add to cart

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share with email

Email a colleague

Share