Dynamic Time Warping Clustering to Discover Socioeconomic Characteristics in Smart Water Meter Data
Publication: Journal of Water Resources Planning and Management
Volume 147, Issue 6
Abstract
Socioeconomic characteristics are influencing the temporal and spatial variability of water demand, which are the biggest source of uncertainties within water distribution system modeling. Improving current knowledge of these influences can be utilized to decrease demand uncertainties. This paper aims to link smart water meter data to socioeconomic user characteristics by applying a novel clustering algorithm that uses a dynamic time warping metric on daily demand patterns. The approach is tested on simulated and measured single-family home data sets. It is shown that the novel algorithm performs better compared with commonly used clustering methods, both in finding the right number of clusters as well as assigning patterns correctly. Additionally, the methodology can be used to identify outliers within clusters of demand patterns. Furthermore, this study investigates which socioeconomic characteristics (e.g., employment status and number of residents) are prevalent within single clusters and, consequently, can be linked to the shape of the cluster’s barycenters. In future, the proposed methods in combination with stochastic demand models can be used to fill data gaps in hydraulic models.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code generated or used during the study are available in a repository or online in accordance with funder data retention policies (https://github.com/steffelbauer/swm_sdtw).
Acknowledgments
This project has received funding from the European Union’s Horizon 2020 research and innovation program under the Marie Skłodowska-Curie Grant Agreement No. 707404. The opinions expressed in this document reflect only the authors’ view. The European Commission is not responsible for any use that may be made of the information it contains.
References
Aksela, K., and M. Aksela. 2011. “Demand estimation with automated meter reading in a distribution network.” J. Water Resour. Plann. Manage. 137 (5): 456–467. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000131.
Alvisi, S., M. Franchini, and A. Marinelli. 2007. “A short-term, pattern-based model for water-demand forecasting.” J. Hydroinf. 9 (1): 39–50. https://doi.org/10.2166/hydro.2006.016.
Arthur, D., and S. Vassilvitskii. 2007. “K-means++: The advantages of careful seeding.” In Proc., 18th Annual ACM-SIAM Symp. on Discrete Algorithms, SODA 2007, 1027–1035. Philadelphia: Society for Industrial and Applied Mathematics.
Blokker, E. J. M., J. H. G. Vreeburg, S. G. Buchberger, and J. C. van Dijk. 2008. “Importance of demand modelling in network water quality models: A review.” Drinking Water Eng. Sci. 1 (1): 27–38. https://doi.org/10.5194/dwes-1-27-2008.
Blokker, E. J. M., J. H. G. Vreeburg, and J. C. van Dijk. 2010. “Simulating residential water demand with a stochastic end-use model.” J. Water Resour. Plann. Manage. 136 (1): 19–26. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000002.
Boyle, T., D. Giurco, P. Mukheibir, A. Liu, C. Moy, S. White, and R. Stewart. 2013. “Intelligent metering for urban water: A review.” Water (Switzerland) 5 (3): 1052–1081.
Buchberger, S. G., J. Carter, Y. Lee, and T. G. Schade. 2003. Random demands, travel times, and water quality in deadends. Denver: American Water Works Association Research Foundation.
Candelieri, A. 2017. “Clustering and support vector regression for water demand forecasting and anomaly detection.” Water (Switzerland) 9 (3): 224.
Cardell-Oliver, R. 2013a. “Discovering water use activities for smart metering.” In Proc., 2013 IEEE 8th Int. Conf. on Intelligent Sensors, Sensor Networks and Information Processing, 171–176. New York: IEEE.
Cardell-Oliver, R. 2013b. “Water use signature patterns for analyzing household consumption using medium resolution meter data.” Water Resour. Res. 49 (12): 8589–8599. https://doi.org/10.1002/2013WR014458.
Cardell-Oliver, R., J. Wang, and H. Gigney. 2016. “Smart meter analytics to pinpoint opportunities for reducing household water use.” J. Water Resour. Plann. Manage. 142 (6): 04016007. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000634.
Cheifetz, N., Z. Noumir, A. Samé, A.-C. Sandraz, C. Féliers, and V. Heim. 2017. “Modeling and clustering water demand patterns from real-world smart meter data.” Drinking Water Eng. Sci. 10 (2): 75–82. https://doi.org/10.5194/dwes-10-75-2017.
Cominola, A., M. Giuliani, A. Castelletti, D. Rosenberg, and A. Abdallah. 2018. “Implications of data sampling resolution on water use simulation, end-use disaggregation, and demand management.” Environ. Modell. Software 102 (Apr): 199–212. https://doi.org/10.1016/j.envsoft.2017.11.022.
Cominola, A., M. Giuliani, D. Piga, A. Castelletti, and A. Rizzoli. 2015. “Benefits and challenges of using smart meters for advancing residential water demand modeling and management: A review.” Environ. Modell. Software 72 (Oct): 198–214. https://doi.org/10.1016/j.envsoft.2015.07.012.
Cominola, A., K. Nguyen, M. Giuliani, R. A. Stewart, H. R. Maier, and A. Castelletti. 2019. “Data mining to uncover heterogeneous water use behaviors from smart meter data.” Water Resour. Res. 55 (11): 9315–9333.
Cuturi, M., and M. Blondel. 2017. “Soft-DTW: A differentiable loss function for time-series.” In Vol. 70 of Proc., 34th Int. Conf. on Machine Learning, edited by D. Precup and Y. W. Teh, 894–903. Cambridge, UK: Proceedings of Machine Learning Research.
Díaz, S., and J. González. 2020. “Analytical stochastic microcomponent modeling approach to assess network spatial scale effects in water supply systems.” J. Water Resour. Plann. Manage. 146 (8): 04020065. https://doi.org/10.1061/(ASCE)WR.1943-5452.0001237.
Dürrenmatt, D. 2011. “Data mining and data-driven modeling approaches to support wastewater treatment plant operation.” Doctoral thesis, Institute of Environmental Engineering, ETH Zürich.
Dürrenmatt, D. J., D. D. Giudice, and J. Rieckermann. 2013. “Dynamic time warping improves sewer flow monitoring.” Water Res. 47 (11): 3803–3816. https://doi.org/10.1016/j.watres.2013.03.051.
Espinoza, M., C. Joye, R. Belmans, and B. De Moor. 2005. “Short-term load forecasting, profile identification, and customer segmentation: A methodology based on periodic time series.” IEEE Trans. Power Syst. 20 (3): 1622–1630. https://doi.org/10.1109/TPWRS.2005.852123.
Garcia, D., D. González Vidal, J. Quevedo, V. Puig, and J. Saludes. 2015. “Water demand estimation and outlier detection from smart meter data using classification and big data methods.” In Proc., 2nd New Developments in IT & Water Conf., 1–8. London: IWA Publishing.
Gurung, T. R., R. A. Stewart, A. K. Sharma, and C. D. Beal. 2014. “Smart meters for enhanced water supply network modelling and infrastructure planning.” Resour. Conserv. Recycl. 90 (Sep): 34–50. https://doi.org/10.1016/j.resconrec.2014.06.005.
Huang, P., N. Zhu, D. Hou, J. Chen, Y. Xiao, J. Yu, G. Zhang, and H. Zhang. 2018. “Real-time burst detection in district metering areas in water distribution system based on patterns of water demand with supervised learning.” Water (Switzerland) 10 (12): 1765.
Hutton, C. J., Z. Kapelan, L. Vamvakeridou-Lyroudia, and D. A. Savić. 2014. “Dealing with uncertainty in water distribution system models: A framework for real-time modeling and data assimilation.” J. Water Resour. Plann. Manage. 140 (2): 169–183. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000325.
Kwac, J., J. Flora, and R. Rajagopal. 2014. “Household energy consumption segmentation using hourly data.” IEEE Trans. Smart Grid 5 (1): 420–430. https://doi.org/10.1109/TSG.2013.2278477.
Lin, J., S. Williamson, K. Borne, and D. DeBarr. 2012. “Pattern recognition in time series.” Chap. 1 in Advances in machine learning and data mining for astronomy, 617–645. Boca Raton, FL: CRC Press.
Lloyd, S. P. 1982. “Least squares quantization in PCM.” IEEE Trans. Inf. Theory 28 (2): 129–137. https://doi.org/10.1109/TIT.1982.1056489.
McKenna, S. A., F. Fusco, and B. J. Eck. 2014. “Water demand pattern classification from smart meter data.” Procedia Eng. 70 (2014): 1121–1130. https://doi.org/10.1016/j.proeng.2014.02.124.
Monks, I., R. A. Stewart, O. Sahin, and R. Keller. 2019. “Revealing unreported benefits of digital water metering: Literature review and expert opinions.” Water 11 (4): 838. https://doi.org/10.3390/w11040838.
Mounce, S. R., W. R. Furnass, E. Goya, M. Hawkins, and J. B. Boxall. 2016. “Clustering and classification of aggregated smart meter data to better understand how demand patterns relate to customer type.” In Proc., 14th Int. Conf. of Computing and Control for the Water Industry—CCWI 2016, 1–9. London: IWA Publishing.
Nambi, S. N., E. Pournaras, and R. V. Prasad. 2016. “Temporal self-regulation of energy demand.” IEEE Trans. Ind. Inf. 12 (3): 1196–1205. https://doi.org/10.1109/TII.2016.2554519.
Nguyen, K. A., R. A. Stewart, and H. Zhang. 2014. “An autonomous and intelligent expert system for residential water end-use classification.” Expert Syst. Appl. 41 (2): 342–356. https://doi.org/10.1016/j.eswa.2013.07.049.
Nguyen, K. A., R. A. Stewart, H. Zhang, O. Sahin, and N. Siriwardene. 2018. “Re-engineering traditional urban water management practices with smart metering and informatics.” Environ. Modell. Software 101 (Mar): 256–267. https://doi.org/10.1016/j.envsoft.2017.12.015.
Nizar, A. H., and Z. Y. Dong. 2009. “Identification and detection of electricity customer behaviour irregularities.” In Proc., 2009 IEEE/PES Power Systems Conf. and Exposition. New York: IEEE.
Nocedal, J., and S. J. Wright. 2006. Numerical optimization. 2nd ed. New York: Springer.
Rousseeuw, P. 1987. “Silhouettes: A graphical aid to the interpretation and validation of cluster analysis.” J. Comput. Appl. Math. 20 (1): 53–65.
Sakoe, H., and S. Chiba. 1978. “Dynamic programming algorithm optimization for spoken word recognition.” IEEE Trans. Acoust. Speech Signal Process. 26 (1): 43–49. https://doi.org/10.1109/TASSP.1978.1163055.
Shafiee, M. E., A. Rasekh, L. Sela, and A. Preis. 2020. “Streaming smart meter data integration to enable dynamic demand assignment for real-time hydraulic simulation.” J. Water Resour. Plann. Manage. 146 (6): 06020008. https://doi.org/10.1061/(ASCE)WR.1943-5452.0001221.
Shumway, R. H., and D. S. Stoffer. 2010. Time series analysis and its applications: With R examples. 4th ed. Basel, Switzerland: Springer.
Stewart, R. A., et al. 2018. “Integrated intelligent water-energy metering systems and informatics: Visioning a digital multi-utility service provider.” Environ. Modell. Software 105 (Jul): 94–117. https://doi.org/10.1016/j.envsoft.2018.03.006.
Walski, T., D. Chase, D. Savic, W. Grayman, S. Backwith, and E. Koelle. 2003. Advanced water distribution modeling and management. 1st ed. Waterbury, CT: Haestead.
Wang, J., R. Cardell-Oliver, and W. Liu. 2015. “Efficient discovery of recurrent routine behaviours in smart meter time series by growing subsequences.” Vol. 9078 of Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), 522–533. New York: Springer.
Witten, I. H., E. Frank, and M. A. Hall. 2011. Data mining: Practical machine learning tools and techniques. 3rd ed. San Francisco: Morgan Kaufmann.
Yang, A., H. Zhang, R. A. Stewart, and K. Nguyen. 2018. “Enhancing residential water end use pattern recognition accuracy using self-organizing maps and -means clustering techniques: Autoflow v3.1.” Water (Switzerland) 10 (9): 1221. https://doi.org/10.3390/w10091221.
Zhou, X., W. Xu, K. Xin, H. Yan, and T. Tao. 2018. “Self-adaptive calibration of real-time demand and roughness of water distribution systems.” Water Resour. Res. 54 (8): 5536–5550. https://doi.org/10.1029/2017WR022147.
Information & Authors
Information
Published In
Journal of Water Resources Planning and Management
Volume 147 • Issue 6 • June 2021
Copyright
© 2021 American Society of Civil Engineers.
History
Received: Mar 6, 2020
Accepted: Nov 7, 2020
Published online: Mar 31, 2021
Published in print: Jun 1, 2021
Discussion open until: Aug 31, 2021
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.