Data-Driven Sewer Pipe Data Random Generation and Validation
Publication: Journal of Construction Engineering and Management
Volume 146, Issue 12
Abstract
Sewer pipe systems are of great importance to modern cities in various ways, making preventive maintenance a necessary activity to ensure an acceptable level of service at all times. In this respect, closed-circuit television (CCTV) inspection data for sewer pipe systems serve as the basis for preventive maintenance in the context of sewer pipe condition ratings, maintenance schedule planning, and other similar ideas. Defects (i.e., those classified as either cracks, fractures, roots, deposits, broken, or holes) and construction features (i.e., taps) are the targets of the CCTV inspection process, which is used to mark and record the defects and features in the inspection database for the purpose of developing maintenance strategies. In considering sewer pipe maintenance operations in practical terms, the following CCTV inspection data for sewer pipes are of particular interest to this research: length of the pipes, defect interval, and defect sequence for different types of defects (and taps). However, the data collection process using CCTV inspections is typically expensive and time-consuming from the perspective of the municipal department. In this context, an input modeling technique that aims to exploit the potential value of historical data is proposed by combining the Markov chain model with distribution fitting techniques and other relevant methods. The generated dataset goes through a rigorous validation process that includes statistical analysis and comparison, cluster analysis and comparison, and distance-based similarity comparison. The whole process proves that the randomly generated dataset is reasonable since it expresses similar characteristics to the original dataset in many aspects. Overall, the research proposes an input modeling process that could generate human-made sewer pipe inspection data that inherent the major characteristic of the real-life data. The generated data could benefit the real-life practice in various ways, especially in the context of data deficiency.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Data generated or analyzed during the study are available from the corresponding author by request. Information about the Journal’s data-sharing policy can be found here: http://ascelibrary.org/doi/10.1061/(ASCE)CO.1943-7862.0001263.
Acknowledgments
The authors would like to thank the anonymous reviewers for their constructive comments. We gratefully acknowledge the financial support of the Natural Sciences and Engineering Research Council of Canada (CRDPJ 503647-16). The author would also like to thank the EPCOR Drainage Services for their support and technical assistance.
References
Abourizk, S. M., S. A. Hague, and R. Ekyalimpa. 2016. Construction simulation: An introduction using SIMPHONY. Edmonton, AB: Univ. of Alberta.
Abourizk, S. M., D. W. Halpin, and J. R. Wilson. 1994. “Fitting beta distributions based on sample data.” J. Constr. Eng. Manage. 120 (2): 288–305. https://doi.org/10.1061/(ASCE)0733-9364(1994)120:2(288).
Agbulos, A., Y. Mohamed, M. Al-Hussein, S. Abourizk, and J. Roesch. 2006. “Application of lean concepts and simulation analysis to improve efficiency of drainage operations maintenance crews.” J. Constr. Eng. Manage. 132 (3): 291–299. https://doi.org/10.1061/(ASCE)0733-9364(2006)132:3(291).
AL-Alawi, M., A. Bouferguene, and Y. Mohamed. 2018. “Random generation of industrial pipelines’ data using Markov chain model.” Adv. Eng. Inf. 38 (Oct): 725–745. https://doi.org/10.1016/j.aei.2018.10.003.
Alanis, R., A. Ingolfsson, and B. Kolfal. 2013. “A Markov chain model for an EMS system with repositioning.” Prod. Oper. Manage. 22 (1): 216–231. https://doi.org/10.1111/j.1937-5956.2012.01362.x.
Ana, E. V., and W. Bauwens. 2010. “Modeling the structural deterioration of urban drainage pipes: The state-of-the-art in statistical methods.” Urban Water J. 7 (1): 47–59. https://doi.org/10.1080/15730620903447597.
Aspert, N., D. Santa-Cruz, and T. Ebrahimi. 2002. “MESH: Measuring errors between surfaces using the Hausdorff distance.” In Vol. 1 of Proc., 2002 IEEE Int. Conf. on Multimedia and Expo, ICME 2002, 705–708. New York: IEEE. https://doi.org/10.1109/ICME.2002.1035879.
Baik, H., H. Seok, D. Jeong, and D. M. Abraham. 2006. “Estimating transition probabilities in Markov chain-based deterioration models for management of wastewater systems.” J. Water Resour. Plann. Manage. 132 (1): 15–24. https://doi.org/10.1061/(ASCE)0733-9496(2006)132:1(15).
Balekelayi, N., and S. Tesfamariam. 2019. “Statistical inference of sewer pipe deterioration using Bayesian geoadditive regression model.” J. Infrastruct. Syst. 25 (3): 04019021. https://doi.org/10.1061/(ASCE)IS.1943-555X.0000500.
Chinyio, E. A., P. O. Olomalaiye, S. T. Kometa, and F. C. Harris. 1998. “A needs-based methodology construction clients and selecting contractors.” Constr. Manage. Econ. 16 (1): 91–98. https://doi.org/10.1080/014461998372628.
Dubuisson, M.-P., and A. K. Jain. 1994. A modified Hausdorff distance for object matching.” In Vol. 1 of Proc., 12th Int. Conf. on Pattern Recognition, 566–568. New York: IEEE. https://doi.org/10.1109/ICPR.1994.576361.
Fenner, R. A. 2000. “Approaches to sewer maintenance: A review.” Urban Water 2 (4): 343–356. https://doi.org/10.1016/S1462-0758(00)00065-0.
Haan, C. T., D. M. Allen, and J. O. Street. 1976. “A Markov chain model of daily rainfall.” Water Resour. Res. https://doi.org/10.1029/WR012i003p00443.
Holt, G. D. 1996. “Applying cluster analysis to construction contractor classification.” Build. Environ. 31 (6): 557–568. https://doi.org/10.1016/0360-1323(96)00028-5.
Jensen, H., C. A. Biggs, and E. Karunakaran. 2016. “The importance of sewer biofilms.” Wiley Interdiscip. Rev.: Water 3 (4): 487–494. https://doi.org/10.1002/wat2.1144.
Jesorsky, O., K. J. Kirchberg, and R. W. Frischholz. 2001. “Robust face detection using the Hausdorff distance.” In Vol. 2091 of Audio- and video-based biometric person authentication. AVBPA 2001. Lecture notes in computer science, edited by J. Bigun and F. Smeraldi. Berlin: Springer. https://doi.org/10.1007/3-540-45344-X_14.
Kurach, L., H. Zaman, and J. Chang. 2019. Condition assessment and rehabilitation programs at the city of Edmonton. Edmonton, AB: City of Edmonton.
Lecchini-Visintini, A., J. Lygeros, and J. M. MacIejowski. 2010. “Stochastic optimization on continuous domains with finite-time guarantees by Markov chain Monte Carlo methods.” IEEE Trans. Autom. Control 55 (12): 2858–2863. https://doi.org/10.1109/TAC.2010.2078170.
Liu, L., Y. K. Ho, and S. Yau. 2006. “Clustering DNA sequences by feature vectors.” Mol. Phylogenet. Evol. 41 (1): 64–69. https://doi.org/10.1016/j.ympev.2006.05.019.
Lu, M. 2002. “Enhancing project evaluation and review technique simulation through artificial neural network-based input modeling.” J. Constr. Eng. Manage. 128 (5): 438–445. https://doi.org/10.1061/(ASCE)0733-9364(2002)128:5(438).
Micevski, T., G. Kuczera, and P. Coombes. 2002. “Markov model for storm water pipe deterioration.” J. Infrastruct. Syst. 8 (2): 49–56. https://doi.org/10.1061/(ASCE)1076-0342(2002)8:2(49).
NASSCO (National Association of Sewer Service Companies). 2015. Pipeline assessment and certification program (PACP) (version 7). Frederick, MD: NASSCO.
Navab-Kashani, R., L. F. Gay, and A. Bayat. 2015. “Productivity improvement of sewer CCTV inspection through time study and route optimization.” J. Constr. Eng. Manage. 141 (6): 04015009. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000976.
Nutanong, S., E. H. Jacox, and H. Samet. 2011. “An incremental Hausdorff distance calculation algorithm.” Proc. VLDB Endowment 4 (8): 506–517. https://doi.org/10.14778/2002974.2002978.
Pelleg, D., and A. Moore. 2000. “X-means: Extending K-means with efficient estimation of the number of clusters.” In Proc., of 17th Int. Conf. on Machine Learning, 727–734. San Francisco: Morgan Kaufmann Publishers.
Robertson, N., and T. Perera. 2002. “Automated data collection for simulation?” Simul. Pract. Theory 9 (6–8): 349–364. https://doi.org/10.1016/S0928-4869(01)00055-6.
Shen, Y., W. Pedrycz, Y. Chen, X. Wang, A. Gacek, and S. Member. 2019. “Hyperplane division in fuzzy C-means: Clustering big data.” In Proc., IEEE Transactions on Fuzzy Systems. New York: IEEE. https://doi.org/10.1109/TFUZZ.2019.2947231.
Sim, D. G., O. K. Kwon, and R. H. Park. 1999. “Object matching algorithms using robust Hausdorff distance measures.” IEEE Trans. Image Process. 8 (3): 425–429. https://doi.org/10.1109/83.748897.
Tran, D. H., A. W. M. Ng, K. J. McManus, and S. Burn. 2008. “Prediction models for serviceability deterioration of stormwater pipes.” Struct. Infrastruct. Eng. 4 (4): 287–295. https://doi.org/10.1080/15732470600792236.
Trybula, W. J. 1994. “Building simulation models without data.” In Proc., IEEE Int. Conf. on Systems, Man and Cybernetics. New York: IEEE. https://doi.org/10.1109/ICSMC.1994.399838.
Wagstaff, K., C. Cardie, S. Rogers, and S. Schrödl. 2001. “Constrained K-means clustering with background knowledge.” In Proc., Int. Conf. on Machine Learning ICML, 577–584, San Francisco: Morgan Kaufmann Publishers.
Wirahadikusumah, R., D. Abraham, and T. Iseley. 2001. “Challenging issues in modeling deterioration of combined sewers.” J. Infrastruct. Syst. 7 (2): 77–84. https://doi.org/10.1061/(ASCE)1076-0342(2001)7:2(77).
WRc (Water Research Centre). 2013. Manual of sewer condition classification. 5th ed. WRc.
Wu, L., W. Ji, and S. M. AbouRizk. 2020. “Bayesian Inference with Markov chain Monte Carlo–based numerical approach for input model updating.” J. Comput. Civ. Eng. 34 (1): 04019043. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000862.
Yang, J., M. Gunaratne, J. J. Lu, and B. Dietrich. 2005. “Use of recurrent Markov chains for modeling the crack performance of flexible pavements.” J. Transp. Eng. 131 (11): 861–872. https://doi.org/10.1061/(ASCE)0733-947X(2005)131:11(861).
Yin, X., Y. Chen, A. Bouferguene, and M. Al-Hussein. 2020a. “Data-driven bi-level sewer pipe deterioration model: Design and analysis.” Autom. Constr. 116 (Aug): 103181. https://doi.org/10.1016/j.autcon.2020.103181.
Yin, X., Y. Chen, A. Bouferguene, H. Zaman, M. Al-Hussein, and L. Kurach. 2020b. “A deep learning-based framework for an automated defect detection system for sewer pipes.” Autom. Constr. 109 (Jan): 102967. https://doi.org/10.1016/j.autcon.2019.102967.
Yin, X., Y. Chen, A. Bouferguene, H. Zaman, M. Al-Hussein, R. Russell, and L. Kurach. 2019. “Standard closed-circuit television (CCTV) collection time extraction of sewer pipes with machine learning algorithm.” In Proc., 36th Int. Symp. on Automation and Robotics in Construction, ISARC 2019, 107–113. Edmonton, Canada: Univ. of Alberta. https://doi.org/10.22260/isarc2019/0015.
Zaman, H., A. Bouferguene, M. Al-Hussein, and C. Lorentz. 2017. “Improving the productivity of drainage operations activities through schedule optimization.” Urban Water J. 14 (3): 298–306. https://doi.org/10.1080/1573062X.2015.1112409.
Zouaoui, F., and J. R. Wilson. 2004. “Accounting for input-model and input-parameter uncertainties in simulation.” IIE Trans. 36 (11): 1135–1151. https://doi.org/10.1080/07408170490500708.
Information & Authors
Information
Published In
Journal of Construction Engineering and Management
Volume 146 • Issue 12 • December 2020
Copyright
© 2020 American Society of Civil Engineers.
History
Received: Feb 21, 2020
Accepted: Jun 30, 2020
Published online: Sep 20, 2020
Published in print: Dec 1, 2020
Discussion open until: Feb 20, 2021
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.