Real-Time Water Quality Modeling with Ensemble Kalman Filter for State and Parameter Estimation in Water Distribution Networks
Publication: Journal of Water Resources Planning and Management
Volume 145, Issue 11
Abstract
This study presents a novel approach to real-time water quality state (chlorine concentration) and reaction parameter estimation in water distribution systems (WDSs) using ensemble Kalman filter (EnKF)–based data assimilation techniques. Two different types of EnKF-based methods are used in this study: noniterative restart-EnKF (NIR-EnKF) and iterative restart-EnKF (IR-EnKF). The use of these data assimilation frameworks for addressing key uncertainties in water quality models, such as uncertainty in the source or initial concentration of chlorine and uncertainty in the wall reaction parameter, is studied. The effect of ensemble size, number and location of measurement nodes, measurement error, and noise are also studied extensively in this work. The performance of the proposed methodology is tested on two different water networks: a brushy plains network and a large, citywide WDS, the Bangalore inflow network. The results of the simulation study show that both the NIR-EnKF and IR-EnKF methods are appropriate for dealing with uncertainty in source chlorine concentration, but the IR-EnKF method performs better than the NIR-EnKF method in the case of reaction parameter uncertainty.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, and code generated or used during the study are available from the corresponding author upon request.
References
Al-Jasser, A. O. 2007. “Chlorine decay in drinking-water transmission and distribution systems: Pipe service age effect.” Water Res. 41 (2): 387–396. https://doi.org/10.1016/j.watres.2006.08.032.
Anderson, J. L. 2007. “An adaptive covariance inflation error correction algorithm for ensemble filters.” Tellus A: Dyn. Meteorol. Oceanogr. 59 (2): 210–224. https://doi.org/10.1111/j.1600-0870.2006.00216.x.
Anderson, J. L., and S. L. Anderson. 1999. “A Monte Carlo implementation of the nonlinear filtering problem to produce ensemble assimilations and forecasts.” Mon. Weather Rev. 127 (12): 2741–2758. https://doi.org/10.1175/1520-0493(1999)127%3C2741:AMCIOT%3E2.0.CO;2.
Aral, M. M., J. Guan, and M. L. Maslia. 2009. “Optimal design of sensor placement in water distribution networks.” J. Water Resour. Plann. Manage. 136 (1): 5–18. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000001.
Baxter, C. W., S. J. Stanley, and Q. Zhang. 1999. “Development of a full-scale artificial neural network model for the removal of natural organic matter by enhanced coagulation.” J. Water Supply: Res. Tech.-Aqua 48 (4): 129–136. https://doi.org/10.2166/aqua.1999.0013.
Baxter, C. W., Q. Zhang, S. J. Stanley, R. Shariff, R. R. Tupas, and H. L. Stark. 2001. “Drinking water quality and treatment: The use of artificial neural networks.” Supplement, Can. J. Civ. Eng. 28 (S1): 26–35. https://doi.org/10.1139/l00-053.
Beides, H. M., and G. T. Heydt. 1991. “Dynamic state estimation of power system harmonics using Kalman filter methodology.” IEEE Trans. Power Delivery 6 (4): 1663–1670. https://doi.org/10.1109/61.97705.
Benkherouf, A., and A. Y. Allidina. 1988. “Leak detection and location in gas pipelines.” IEE Proc. D-Control Theory Appl. 135 (2): 142–148. https://doi.org/10.1049/ip-d.1988.0020.
Biswas, P., C. Lu, and R. M. Clark. 1993. “A model for chlorine concentration decay in pipes.” Water Res. 27 (12): 1715–1724. https://doi.org/10.1016/0043-1354(93)90108-T.
Blood, E. A., B. H. Krogh, and M. D. Ilic. 2008. “Electric power system static state estimation through Kalman filtering and load forecasting.” In Proc., Power and Energy Society General Meeting-Conversion and Delivery of Electrical Energy in the 21st Century, 1–6. New York: IEEE.
Boccelli, D. L., M. E. Tryby, J. G. Uber, L. A. Rossman, M. L. Zierolf, and M. M. Polycarpou. 1998. “Optimal scheduling of booster disinfection in water distribution systems.” J. Water Resour. Plann. Manage. 124 (2): 99–111. https://doi.org/10.1061/(ASCE)0733-9496(1998)124:2(99).
Bowden, G. J., J. B. Nixon, G. C. Dandy, H. R. Maier, and M. Holmes. 2006. “Forecasting chlorine residuals in a water distribution system using a general regression neural network.” Math. Comput. Modell. 44 (5–6): 469–484. https://doi.org/10.1016/j.mcm.2006.01.006.
Burgers, G., P. Jan van Leeuwen, and G. Evensen. 1998. “Analysis scheme in the ensemble Kalman filter.” Mon. Weather Rev. 126 (6): 1719–1724. https://doi.org/10.1175/1520-0493(1998)126%3C1719:ASITEK%3E2.0.CO;2.
Carton, J. A., and B. S. Giese. 2008. “A reanalysis of ocean climate using Simple Ocean Data Assimilation (SODA).” Mon. Weather Rev. 136 (8): 2999–3017. https://doi.org/10.1175/2007MWR1978.1.
Clark, R. M. 2015. “The USEPA’s distribution system water quality modelling program: A historical perspective.” Water Environ. J. 29 (3): 320–330. https://doi.org/10.1111/wej.12132.
Clark, R. M., W. M. Grayman, R. M. Males, and A. F. Hess. 1993. “Modeling contaminant propagation in drinking-water distribution systems.” J. Environ. Eng. 119 (2): 349–364. https://doi.org/10.1061/(ASCE)0733-9372(1993)119:2(349).
Clark, R. M., L. A. Rossman, and L. J. Wymer. 1995. “Modeling distribution system water quality: Regulatory implications.” J. Water Resour. Plann. Manage. 121 (6): 423–428. https://doi.org/10.1061/(ASCE)0733-9496(1995)121:6(423).
Doucet, A., N. J. Gordon, and V. Krishnamurthy. 2001. “Particle filters for state estimation of jump Markov linear systems.” IEEE Trans. Signal Process. 49 (3): 613–624. https://doi.org/10.1109/78.905890.
Drécourt, J. P., H. Madsen, and D. Rosbjerg. 2006. “Calibration framework for a Kalman filter applied to a groundwater model.” Adv. Water Resour. 29 (5): 719–734. https://doi.org/10.1016/j.advwatres.2005.07.007.
D’souza, C. D., and M. S. Mohan Kumar. 2010. “Comparison of ANN models for predicting water quality in distribution systems.” J. Am. Water Works Assn. 102 (7): 92. https://doi.org/10.1002/j.1551-8833.2010.tb10152.x.
Emara-Shabaik, H. E., Y. A. Khulief, and I. Hussaini. 2002. “A non-linear multiple-model state estimation scheme for pipeline leak detection and isolation.” Proc. Inst. Mech. Eng., Part I: J. Syst. Control Eng. 216 (6): 497–512. https://doi.org/10.1177/095965180221600605.
Evensen, G. 1994. “Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics.” J. Geophys. Res. C: Oceans 99 (C5): 10143–10162.https://doi.org/10.1029/94JC00572.
Evensen, G. 2003. “The ensemble Kalman filter: Theoretical formulation and practical implementation.” Ocean Dyn. 53 (4): 343–367. https://doi.org/10.1007/s10236-003-0036-9.
Fisher, I., G. Kastl, and A. Sathasivan. 2017. “New model of chlorine-wall reaction for simulating chlorine concentration in drinking water distribution systems.” Water Res. 125 (Nov): 427–437.https://doi.org/10.1016/j.watres.2017.08.066.
Geir, N., L. M. Johnsen, S. I. Aanonsen, and E. H. Vefring. 2003. “Reservoir monitoring and continuous model updating using ensemble Kalman filter.” In Proc., SPE Annual Technical Conf. and Exhibition. Richardson, TX: Society of Petroleum Engineers.
Gibbs, M. S., N. Morgan, H. R. Maier, G. C. Dandy, J. B. Nixon, and M. Holmes. 2006. “Investigation into the relationship between chlorine decay and water distribution parameters using data driven methods.” Math. Comput. Modell. 44 (5–6): 485–498. https://doi.org/10.1016/j.mcm.2006.01.007.
Grayman, W. M., R. M. Clark, and R. M. Males. 1988. “Modeling distribution-system water quality; dynamic approach.” J. Water Resour. Plann. Manage. 114 (3): 295–312. https://doi.org/10.1061/(ASCE)0733-9496(1988)114:3(295).
Gu, Y., and D. S. Oliver. 2007. “An iterative ensemble Kalman filter for multiphase fluid flow data assimilation.” SPE J. 12 (4): 438–446. https://doi.org/10.2118/108438-PA.
Hall, J., A. D. Zaffiro, R. B. Marx, P. C. Kefauver, E. R. Krishnan, R. C. Haught, and J. G. Herrmann. 2007. “On-line water quality parameters as indicators of distribution system contamination.” J. Am. Water Works Assn. 99 (1): 66–77. https://doi.org/10.1002/j.1551-8833.2007.tb07847.x.
Hallam, N. B., J. R. West, C. F. Forster, J. C. Powell, and I. Spencer. 2002. “The decay of chlorine associated with the pipe wall in water distribution systems.” Water Res. 36 (14): 3479–3488.https://doi.org/10.1016/S0043-1354(02)00056-8.
Hamill, T. M., S. L. Mullen, C. Snyder, D. P. Baumhefner, and Z. Toth. 2000. “Ensemble forecasting in the short to medium range: Report from a workshop.” Bull. Am. Meteorol. Soc. 81 (11): 2653–2664. https://doi.org/10.1175/1520-0477(2000)081%3C2653:EFITST%3E2.3.CO;2.
Hart, W. E., and R. Murray. 2010. “Review of sensor placement strategies for contamination warning systems in drinking water distribution systems.” J. Water Resour. Plann. Manage. 136 (6): 611–619. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000081.
Hendricks Franssen, H. J., P. Brunner, P. Makobo, and W. Kinzelbach. 2008. “Equally likely inverse solutions to a groundwater flow problem including pattern information from remote sensing images.” Water Resour. Res. 44 (1). https://doi.org/10.1029/2007WR006097.
Hendricks Franssen, H. J., and W. Kinzelbach. 2008. “Real-time groundwater flow modeling with the ensemble Kalman filter: Joint estimation of states and parameters and the filter inbreeding problem.” Water Resour. Res. 44 (9): 1–21. https://doi.org/10.1029/2007WR006505.
Houtekamer, P. L., and H. L. Mitchell. 1998. “Data assimilation using an ensemble Kalman filter technique.” Mon. Weather Rev. 126 (3): 796–811. https://doi.org/10.1175/1520-0493(1998)126%3C0796:DAUAEK%3E2.0.CO;2.
Hutton, C. J., Z. Kapelan, L. Vamvakeridou-Lyroudia, and D. A. Savić. 2012. “Dealing with uncertainty in water distribution system models: A framework for real-time modeling and data assimilation.” J. Water Resour. Plann. Manage. 140 (2): 169–183. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000325.
Jung, D., and K. Lansey. 2014. “Water distribution system burst detection using a nonlinear Kalman filter.” J. Water Resour. Plann. Manage. 141 (5): 04014070. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000464.
Kalman, R. E. 1960. “A new approach to linear filtering and prediction problems.” J. Basic Eng. 82 (1): 35–45. https://doi.org/10.1115/1.3662552.
Kalnay, E. 2003. Atmospheric modeling, data assimilation and predictability. Cambridge, UK: Cambridge University Press.
Kang, D., and K. Lansey. 2009. “Real-time demand estimation and confidence limit analysis for water distribution systems.” J. Hyd. Eng. 135 (10): 825–837. https://doi.org/10.1061/(ASCE)HY.1943-7900.0000086.
Liu, M., S. Zang, and D. Zhou. 2005. “Fast leak detection and location of gas pipelines based on an adaptive particle filter.” Int. J. Appl. Math. Comput. Sci. 15 (4): 541.
Liu, Y., et al. 2012. “Advancing data assimilation in operational hydrologic forecasting: Progresses, challenges, and emerging opportunities.” Hydrol. Earth Syst. Sci. 16 (10): 3863–3887. https://doi.org/10.5194/hess-16-3863-2012.
Lorenc, A. C. 2003. “The potential of the ensemble Kalman filter for NWP: A comparison with 4D-Var.” Q. J. R. Meteorol. Soc. 129 (595): 3183–3203. https://doi.org/10.1256/qj.02.132.
Maier, H. R., N. Morgan, and C. W. Chow. 2004. “Use of artificial neural networks for predicting optimal alum doses and treated water quality parameters.” Environ. Modell. Software 19 (5): 485–494. https://doi.org/10.1016/S1364-8152(03)00163-4.
Manohar, U., and M. S. M. Mohan Kumar. 2013. “Modeling equitable distribution of water: Dynamic inversion-based controller approach.” J. Water Resour. Plann. Manage. 140 (5): 607–619. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000368.
May, R. J., G. C. Dandy, H. R. Maier, and J. B. Nixon. 2008. “Application of partial mutual information variable selection to ANN forecasting of water quality in water distribution systems.” Environ. Modell. Software 23 (10–11): 1289–1299. https://doi.org/10.1016/j.envsoft.2008.03.008.
Milot, J., M. J. Rodriguez, and J. B. Sérodes. 2002. “Contribution of neural networks for modeling trihalomethanes occurrence in drinking water.” J. Water Resour. Plann. Manage. 128 (5): 370–376. https://doi.org/10.1061/(ASCE)0733-9496(2002)128:5(370).
Moradkhani, H., S. Sorooshian, H. V. Gupta, and P. R. Houser. 2005. “Dual state–parameter estimation of hydrological models using ensemble Kalman filter.” Adv. Water Resour. 28 (2): 135–147.https://doi.org/10.1016/j.advwatres.2004.09.002.
Munavalli, G. R. 2002. “Simulation and parameter estimation of water quality in water distribution system.” Ph.D. thesis, Dept. of Civil Engineering, Indian Institute of Science.
Munavalli, G. R., and M. S. M. Kumar. 2003. “Water quality parameter estimation in steady-state distribution system”. J. Water Resour. Plann. Manage. 129 (2): 124–134. https://doi.org/10.1061/(ASCE)0733-9496(2003)129:2(124).
Munavalli, G. R., and M. S. M. Kumar. 2004. “Modified Lagrangian method for modeling water quality in distribution systems.” Water Res. 38 (13): 2973–2988. https://doi.org/10.1016/j.watres.2004.04.007.
Munavalli, G. R., and M. S. M. Kumar. 2005. “Water quality parameter estimation in a distribution system under dynamic state.” Water Res. 38 (13): 2973–2988. https://doi.org/10.1016/j.watres.2004.04.007.
Naevdal, G., L. V. Johnsen, S. I. Aanonsen, and E. H. Vefring. 2003. “Reservoir monitoring and continuous model updating using ensemble Kalman filter.” In Proc., SPE Annual Technical Conf. and Exhibition. Richardson, TX: Society of Petroleum Engineers.
Nilsson, K. A., S. G. Buchberger, and R. M. Clark. 2005. “Simulating exposures to deliberate intrusions into water distribution systems.” J. Water Resour. Plann. Manage. 131 (3): 228–236. https://doi.org/10.1061/(ASCE)0733-9496(2005)131:3(228).
Okeya, I., Z. Kapelan, C. Hutton, and D. Naga. 2014. “Online modelling of water distribution system using data assimilation.” Procedia Eng. 70: 1261–1270. https://doi.org/10.1016/j.proeng.2014.02.139.
Ostfeld, A., J. G. Uber, E. Salomons, J. W. Berry, W. E. Hart, C. A. Phillips, and F. di Pierro. 2008. “The battle of the water sensor networks (BWSN): A design challenge for engineers and algorithms.” J. Water Resour. Plann. Manage. 134 (6): 556–568. https://doi.org/10.1061/(ASCE)0733-9496(2008)134:6(556).
Park, J. H., and A. Kaneko. 2000. “Assimilation of coastal acoustic tomography data into a barotropic ocean model.” Geophys. Res. Lett. 27 (20): 3373–3376. https://doi.org/10.1029/2000GL011600.
Pasha, M. F., and K. Lansey. 2010. “Effect of parameter uncertainty on water quality in distribution systems—Case study.” J. Hydroinf. 12 (1): 1–21. https://doi.org/10.2166/hydro.2010.053.
Pastres, R., S. Ciavatta, and C. Solidoro. 2003. “The extended Kalman filter (EKF) as a tool for the assimilation of high frequency water quality data.” Ecol. Modell. 170 (2–3): 227–235. https://doi.org/10.1016/S0304-3800(03)00230-8.
Polycarpou, M. M., J. G. Uber, Z. Wang, F. Shang, and M. Brdys. 2002. “Feedback control of water quality.” IEEE Control Syst. 22 (3): 68–87. https://doi.org/10.1109/MCS.2002.1004013.
Rodriguez, M. J., and J. B. Sérodes. 1998. “Assessing empirical linear and non-linear modelling of residual chlorine in urban drinking water systems.” Environ. Modell. Software 14 (1): 93–102.https://doi.org/10.1016/S1364-8152(98)00061-9.
Rossman, L. A. 2000. EPANET 2: User’s manual. Cincinnati: USEPA.
Rossman, L. A., R. M. Clark, and W. M. Grayman. 1994. “Modeling chlorine residuals in drinking-water distribution systems.” J. Environ. Eng. 120 (4): 803–820. https://doi.org/10.1061/(ASCE)0733-9372(1994)120:4(803).
Sérodes, J. B., M. J. Rodriguez, and A. Ponton. 2001. “Chlorcast©: A methodology for developing decision-making tools for chlorine disinfection control.” Environ. Modell. Software 16 (1): 53–62. https://doi.org/10.1016/S1364-8152(00)00064-5.
Simone, A., O. Giustolisi, and D. B. Laucelli. 2016. “A proposal of optimal sampling design using a modularity strategy.” Water Resour. Res. 52 (8): 6171–6185. https://doi.org/10.1002/2016WR018944.
Song, X., L. Shi, M. Ye, J. Yang, and I. M. Navon. 2014. “Numerical comparison of iterative ensemble Kalman filters for unsaturated flow inverse modeling.” Vadose Zone J. 13 (2). https://doi.org/10.2136/vzj2013.05.0083.
Soyupak, S., H. Kilic, I. E. Karadirek, and H. Muhammetoglu. 2011. “On the usage of artificial neural networks in chlorine control applications for water distribution networks with high quality water.” J. Water Supply Res. Tech.-Aqua 60 (1): 51–60. https://doi.org/10.2166/aqua.2011.086.
Suresh, M., U. Manohar, G.R Anjana, R. Stoleru, and M. S. M. Kumar. 2014. “A cyber-physical system for continuous monitoring of water distribution systems.” In Proc., IEEE 10th Int. Conf. on Wireless and Mobile Computing, Networking and Communications (WiMob), 570–577. New York: IEEE.
van Loon, M., P. J. Builtjes, and A. J. Segers. 2000. “Data assimilation of ozone in the atmospheric transport chemistry model LOTOS.” Environ. Modell. Software 15 (6–7): 603–609. https://doi.org/10.1016/S1364-8152(00)00048-7.
Vasconcelos, J. J., L. A. Rossman, W. M. Grayman, P. F. Boulos, and R. M. Clark. 1997. “Kinetics of chlorine decay.” J. Am. Water Works Assn. 89 (7): 54–65. https://doi.org/10.1002/j.1551-8833.1997.tb08259.x.
Whitaker, J. S., and T. M. Hamill. 2002. “Ensemble data assimilation without perturbed observations.” Mon. Weather Rev. 130 (7): 1913–1924. https://doi.org/10.1175/1520-0493(2002)130%3C1913:EDAWPO%3E2.0.CO;2.
Ye, G., and R. A. Fenner. 2011. “Kalman filtering of hydraulic measurements for burst detection in water distribution systems.” J. Pipeline Syst. Eng. Pract. 2 (1): 14–22. https://doi.org/10.1061/(ASCE)PS.1949-1204.0000070.
Ye, G., and R. A. Fenner. 2014. “Weighted least squares with expectation-maximization algorithm for burst detection in UK water distribution systems.” J. Water Resour. Plann. Manage. 140 (4): 417–424. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000344.
Information & Authors
Information
Published In
Journal of Water Resources Planning and Management
Volume 145 • Issue 11 • November 2019
Copyright
©2019 American Society of Civil Engineers.
History
Received: Jul 9, 2018
Accepted: Mar 21, 2019
Published online: Aug 29, 2019
Published in print: Nov 1, 2019
Discussion open until: Jan 29, 2020
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.