Multivariate Trajectory Clustering for False Positive Reduction in Online Event Detection
Publication: Journal of Water Resources Planning and Management
Volume 139, Issue 1
Abstract
Online monitoring of multivariate water quality data is becoming a practical means of improving distribution network management and meeting water security goals. Changes in water quality are often due to changes in the hydraulic operations of the network. These operational changes create patterns of water quality change that are similar, but not exactly the same, from one instance to the next. Classification of multivariate change patterns through trajectory clustering is introduced in this paper to create a pattern library from historical water quality data and as an online process with the goal of reducing false positive water quality event detections. Prior to event declaration, a short sequence of the preceding multivariate data is compared against the pattern library to assess its similarity to a previously observed pattern. A fuzzy clustering algorithm is utilized to assign multivariate pattern memberships for water quality patterns associated with water quality events in both the offline and online modes of operation. The utility of trajectory clustering for multivariate pattern recognition in time-series data is demonstrated with two example applications. The first example uses observed water quality with simulated patterns and events. The pattern matching reduces the number of false positive event detections by 91% relative to the case of not using the pattern matching. The same false positive event reduction is achieved when both patterns and separate water quality events are added and 100% event detection is achieved. The second example uses observed water quality data from a metropolitan distribution system in the United States. The pattern matching approach developed in this paper is able to reduce the false positive event detections by 68%.
Get full access to this article
View all available purchase options and get full access to this article.
Acknowledgments
This work was performed under interagency agreement DW89921928 with the U.S. EPA and under contract with the Singapore National Water Authority (PUB). Sandia National Laboratories is a multiprogram laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000. The authors appreciate the efforts of three anonymous reviewers that greatly improved this paper.
References
Allgeier, S., and Umberg, K. (2008). “Operational experience with water quality anomaly detection during the Cincinnati Water Security Initiative Pilot.” Water Quality Technology Conf., American Water Works Association, Denver.
Bezdek, J. C. (1981). Pattern recognition with fuzzy objective function algorithms, Plenum Press, New York.
Camargo, S. J., Robertson, A. W., Gaffney, S. J., Smyth, P., and Ghil, M. (2007). “Cluster analysis of typhoon tracks. Part I: General properties.” J. Clim., 20(14), 3635–3653.
Cochran, J. (2011). “The multivariate normal distribution.” 〈http://cab.latech.edu/~jcochran/QA610/the%20multivariate%20normal%20distribution.ppt〉 (Dec. 6, 2011).
Cook, J. B., Byrne, J. F., Daamen, R. C., and Roehl, E. A. (2006). “Distribution system monitoring research at Charleston Water System.” Proceedings of World Environmental and Water Resources Congress, American Society of Civil Engineers, Reston, VA.
Dunn, J. C. (1973). “A fuzzy relative of the ISODATA process and its use in detecting compact well-separated clusters.” J. Cybern., 3(3), 32–57.
Gaffney, S., and Smyth, P. (1999). “Trajectory clustering with mixtures of regression models.” Proc., Fifth ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, Chaudhuri, S., and Madigan, D., eds., ACM Press, New York, 63–72 (Aug. 15–18).
Grayman, W. M., Deininger, R. A., and Males, R. M. (2001). Design of early warning and predictive source water monitoring systems, American Water Works Association, Denver, Colorado, 328 pp.
Hasan, J., States, S., and Deininger, R. (2004). “Safeguarding the security of public water supplies using early warning systems: A brief review.” Journal of Contemporary Water Research and Education, 129(1), 27–33.
Hart, D. B., and McKenna, S. A. (2009). User’s manual CANARY 4.1, U.S. Environmental Protection Agency, Washington, DC.
Hartigan, J. A., and Wong, M. A. (1978). “Algorithm AS 136: A K-means clustering algorithm.” Appl. Stat., 28(1), 100–108.
Koch, M. W., and McKenna, S. A. (2011). “Distributed sensor fusion in water quality event detection.” J. Water Resour. Plann. and Manage., 137(1), 10–19.
Kroll, D., and King, K. (2006). “Laboratory and flow loop validation and testing of the operational effectiveness of an on-line security platform for the water distribution system.” Proceedings of World Environmental and Water Resources Congress, American Society of Civil Engineers, Reston, VA.
McKenna, S. A., Hart, D. B., Klise, K. A., Cruz, V. A., and Wilson, M. P. (2007). “Event detection from water quality time series.” Proc. of ASCE World Environmental and Water Resources Congress, ASCE, Reston, VA.
McKenna, S. A., Wilson, M., and Klise, K. A. (2008). “Detecting changes in water quality data.” J. Am. Water Works Assoc., 100(1), 74–85.
Murray, R. et al. (2010). “Water quality event detection systems for drinking water contamination warning systems: Development, testing, and application of CANARY.”, U.S. Environmental Protection Agency, National Homeland Security Research Center, Cincinnati.
O’Halloran, R., Yang, S., Tulloh, A., Koltun, P., and Toifl, M. (2006). “Sensor-based water parcel tracking.” Proceedings of World Environmental and Water Resources Congress, American Society of Civil Engineers, Reston, VA.
Pakhira, M. K., Bandyopadhyay, S., and Maulik, U. (2004). “Validity index for crisp and fuzzy clusters.” Pattern Recognition, 37(3), 487–501.
Rizak, S. N., and Hrudey, S. E. (2006). “Misinterpretation of drinking water quality monitoring data with implications for risk management.” Environ. Sci. Technol., 40(17), 5244–5250.
Roberson, J. A., and Morley, K. M. (2005). “Contamination Warning Systems for Water: An Approach for Providing Actionable Information to Decision Makers.” American Water Works Association, Denver, Colorado, 23 pp.
Vugrin, E. D., McKenna, S. A., and Hart, D. B. (2009). “Trajectory clustering approach for reducing water quality event false alarms.” Proceedings of World Environmental and Water Resources Congress, American Society of Civil Engineers, Reston, VA.
Xu, R., and Wunsch, D. C. (2009). Clustering, Wiley, Hoboken, NJ.
Yang, Y. J., Haught, R. C., and Goodrich, J. A. (2009). “Real-time contaminant detection and classification in a drinking water pipe using conventional water quality sensors: Techniques and experimental results.” J. Environ. Manage., 90(8), 2494–2506.
Information & Authors
Information
Published In
Journal of Water Resources Planning and Management
Volume 139 • Issue 1 • January 2013
Pages: 3 - 12
Copyright
© 2013 American Society of Civil Engineers.
History
Received: Jul 20, 2011
Accepted: Feb 28, 2012
Published online: Mar 3, 2012
Published in print: Jan 1, 2013
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.