Covariate-Shift Generative Adversarial Network and Railway Track Image Analysis
Publication: Journal of Transportation Engineering, Part A: Systems
Volume 149, Issue 3
Abstract
The acceptance of railway systems as a frontier transportation infrastructure can be attributed to their reliability, safety, and support for green technology. With the recent advances in artificial intelligence and machine learning (AI/ML), the maintenance of railroad transportation systems has taken a different direction, especially in the analysis of railroad big data, leading to real-time processing and detection of railway problems. However, using limited track data may result in overfitting, hindering the accurate implementation of robust models. In this paper, the authors consider generative adversarial networks (GANs) with keen consideration for possible covariate shifts to improve track defect detection and decrease data imbalance. The results show that implementing covariate-shift GAN (COGAN) reduces image processing time and eliminates image biases.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
The authors wish to thank the US Department of Transportation, and the University Transportation Center Program, which enables a globally competitive research network that addresses the grand challenge of railway infrastructure. The authors also would like to thank the reviewers and the editor for their valuable and insightful reviews and comments.
References
Al-Douri, Y. K., P. Tretten, and R. Karim. 2016. “Improvement of railway performance: A study of Swedish railway infrastructure.” J. Mod. Transp. 24 (1): 22–37. https://doi.org/10.1007/s40534-015-0092-0.
Ali-Gombe, A., and E. Elyan. 2019. “MFC-GAN: Class-imbalanced dataset classification using multiple fake class generative adversarial network.” Neurocomputing 361 (Oct): 212–221. https://doi.org/10.1016/j.neucom.2019.06.043.
Balogun, I., M. Leadingham, D. Gulliot, and N. Attoh-Okine. 2022. “Deep learning approach towards squat isolation in a multi-embedded track geometry defects.” In Proc., IEEE Int. Conf. on Big Data (Big Data). New York: IEEE.
Bhattarai, B., S. Baek, R. Bodur, and T. Kyun Kim. 2020. “Sampling strategies for gan synthetic data.” In Proc., ICASSP, IEEE Int. Conf. on Acoustics, Speech and Signal Processing, 2303–2307. New York: IEEE.
Borji, A. 2019. “Pros and cons of GAN evaluation measures.” Comput. Vis. Image Understanding 179 (Jul): 41–65. https://doi.org/10.1016/j.cviu.2018.10.009.
Bourou, S., A. El Saer, T. H. Velivassaki, A. Voulkidis, and T. Zahariadis. 2021. “A review of tabular data synthesis using gans on an Ids dataset.” Information 12 (9): 375. https://doi.org/10.3390/info12090375.
Bressem, K. K., L. C. Adams, C. Erxleben, B. Hamm, S. M. Niehues, and J. L. Vahldiek. 2020. “Comparing different deep learning architectures for classification of chest radiographs.” Sci. Rep. 10 (1): 1–16. https://doi.org/10.1038/s41598-020-70479-z.
Cai, Z., Z. Xiong, H. Xu, P. Wang, W. Li, and Y. Pan. 2021. “Generative adversarial networks: A survey toward private and secure applications.” ACM Comput. Surv. 54 (6): 1–38. https://doi.org/10.1145/3459992.
Chandran, P., J. Asber, F. Thiery, J. Odelius, and M. Rantatalo. 2021. “An investigation of railway fastener detection using image processing and augmented deep learning.” Sustainability 13 (21): 12051. https://doi.org/10.3390/su132112051.
Chen, S., E. Dobriban, and J. H. Lee. 2020. “A group-theoretic framework for data augmentation.” J. Mach. Learn. Res. 21 (245): 1–71.
Choi, J., T. Kim, and C. Kim. 2019. “Self-ensembling with GAN-based data augmentation for domain adaptation in semantic segmentation.” In Proc., IEEE Int. Conf. on Computer Vision, 6829–6839. New York: IEEE.
Chooch, A. I. 2021. “Visual Ai railway inspections: Better detection of railroad defects and obstacles.” Accessed December 29, 2021. https://chooch.ai/computer-vision/visual-ai-railway-inspections-better-detection-of-railroad-defects-and-obstacles/.
Creswell, A., T. White, V. Dumoulin, K. Arulkumaran, B. Sengupta, and A. A. Bharath. 2018. “Generative adversarial networks: An overview.” IEEE Signal Process Mag. 35 (1): 53–65. https://doi.org/10.1109/MSP.2017.2765202.
Diana, G., S. Bruni, E. Di Gialleonardo, R. Corradi, and A. Facchinetti. 2016. “A study of the factors affecting flange-climb derailment in railway vehicles.” In Proc., 3rd Int. Conf. on Railway Technology: Research, Development and Maintenance, edited by J. Pombo. Stirlingshire, Scotland: Civil-Comp Press.
Ekberg, A., and E. Kabo. 2005. “Fatigue of railway wheels and rails under rolling contact and thermal loading-an overview.” Wear 258 (7–8): 1288–1300. https://doi.org/10.1016/j.wear.2004.03.039.
Enerators, M., G. Ultiple, and Q. Hoang. 2016. “Multi-generator generative adversarial.” Preprint, submitted August 8, 2017. https://arXiv.org/abs/1708.02556.
Fairclough, D. L. 2015. “Response shift in the presence of missing data.” Qual. Life Res. 24 (3): 565–566. https://doi.org/10.1007/s11136-015-0920-z.
Fedus, W., M. Rosca, B. Lakshminarayanan, A. M. Dai, S. Mohamed, and I. Goodfellow. 2017. “Many paths to equilibrium: GANs do not need to decreasea divergence at every step.” Preprint, submitted October 23, 2017. https://arXiv.org/abs/1710.08446.
Fekri, M. N., A. Mohon Ghosh, and K. Grolinger. 2019. “Generating energy data for machine learning with recurrent generative adversarial networks.” Energies 13 (1): 130. https://doi.org/10.3390/en13010130.
Feng, J. H., H. Yuan, Y. Q. Hu, J. Lin, S. W. Liu, and X. Luo. 2020. “Research on deep learning method for rail surface defect detection.” IET Electr. Syst. Transp. 10 (4): 436–442. https://doi.org/10.1049/iet-est.2020.0041.
Fernández, A., S. García, and F. Herrera. 2011. “Addressing the classification with imbalanced data: Open problems and new challenges on class distribution.” In Proc., Int. Conf. on Hybrid Artificial Intelligence Systems. Berlin: Springer.
Goodfellow, I., J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. 2020. “Generative adversarial networks.” Commun. ACM 63 (11): 139–144. https://doi.org/10.1145/3422622.
Gupta, S. K. 2021. “Reduction of covariate factors from silhouette image for robust gait recognition.” Multimedia Tools Appl. 80 (28–29): 36033–36058. https://doi.org/10.1007/s11042-021-10941-w.
Gurumurthy, S., R. Kiran Sarvadevabhatla, and R. Venkatesh Babu. 2017. “DeLiGAN: Generative adversarial networks for diverse and limited data.” In Proc., 30th IEEE Conf. on Computer Vision and Pattern Recognition, CVPR 2017, 4941–4949. New York: IEEE.
Kim, H., S. Lee, and S. Han. 2020. “Railroad surface defect segmentation using a modified fully convolutional network.” KSII Trans. Internet Inf. Syst. 14 (12): 4763–4775.
Lamprea-Pineda, A. C., D. P. Connolly, and M. F. M. Hussein. 2022. “Beams on elastic foundations—A review of railway applications and solutions.” Transp. Geotech. 33: 100696. https://doi.org/10.1016/j.trgeo.2021.100696.
Li, Z. 2009. “Squats on railway rails.” In Wheel-rail interface handbook, 409–436. Cambridge, UK: Woodhead.
Liu, J., S. Chen, G. Lederman, D. B. Kramer, H. Y. Noh, J. Bielak, J. H. Garrett, J. Kovačević, and M. Bergés. 2019. “Dynamic responses, GPS positions and environmental conditions of two light rail vehicles in Pittsburgh.” Sci. Data 6 (1): 1–11.
Liu, X., T. Li, R. Zhang, D. Wu, Y. Liu, and Z. Yang. 2021. “A GAN and feature selection-based oversampling technique for intrusion detection.” In Security and communication networks 2021. London: Hindawi.
Loey, M., F. Smarandache, and N. E. M. Khalifa. 2020. “Within the lack of chest COVID-19 X-ray dataset: A novel detection model based on GAN and deep transfer learning.” Symmetry 12 (4): 651. https://doi.org/10.3390/sym12040651.
Lyu, Y., et al. 2019. “A GAN-based anomaly detection method for isoelectric line in high-speed railway.” In Proc., I2MTC 2019-2019 IEEE Int. Instrumentation and Measurement Technology Conf. Piscataway, NJ: IEEE.
Majurski, M., P. Manescu, S. Padi, N. Schaub, N. Hotaling, C. Simon, and P. Bajcsy. 2019. “Cell image segmentation using generative adversarial networks, transfer learning, and augmentations.” In Proc., IEEE/CVF Conf. on Computer Vision and Pattern Recognition Workshops. New York: IEEE.
Maynard-Reid, M. 2021. “GAN training challenges: DCGAN for color images.” Accessed December 30, 2021. https://www.pyimagesearch.com/2021/12/13/gan-training-challenges-dcgan-for-color-images/.
Metz, L. 2017. “Compositional pattern producing GAN.” In Proc., Machine Learning for Creativity and Design—NIPS 2017 Workshop, 1–4. San Diego: Neural Information Processing Systems.
Olson, M. L., T. V. Nguyen, G. Dixit, N. Ratzlaff, W. K. Wong, and M. Kahng. 2021. “Contrastive identification of covariate shift in image data.” In Proc., IEEE Visualization Conf. New York: IEEE. https://doi.org/10.1109/VIS49827.2021.9623289.
Santurkar, S., L. Schmidt, and A. Madry. 2018. “A classification-based study of covariate shift in GAN distributions.” In Vol. 10 of Proc., 35th Int. Conf. on Machine Learning, ICML 2018, 7135–7144. Cambridge, MA: MIT Press.
Storkey, A. J., and M. Sugiyama. 2007. “Mixture regression for covariate shift.” In Advances in neural information processing systems, 1337–1344. Cambridge, MA: MIT Press.
Sugiyama, M., M. Krauledat, and K.-R. Müller. 2014. “Robust learning under uncertain test distributions.” J. Mach. Learn. Res. 8: 985–1005.
Sugiyama, M., S. Nakajima, H. Kashima, P. Buenau, and M. Kawanabe. 2007. “Direct importance estimation with model selection and its application to covariate shift adaptation.” In Proc., 20th Int. Conf. on Neural Information Processing Systems, 1433–1440. Tokyo: Institute of Statistical Mathematics.
Sugiyama, M., T. Suzuki, S. Nakajima, H. Kashima, P. von Bünau, and M. Kawanabe. 2008. “Direct importance estimation for covariate shift adaptation.” Ann. Inst. Stat. Math. 60 (4): 699–746. https://doi.org/10.1007/s10463-008-0197-x.
Sun, Y., P. Wang, J. Lu, J. Xu, P. Wang, S. Xie, Y. Li, J. Dai, B. Wang, and M. Gao. 2021. “Rail corrugation inspection by a self-contained triple-repellent electromagnetic energy harvesting system.” Appl. Energy 286 (Nov): 116512. https://doi.org/10.1016/j.apenergy.2021.116512.
Tibshirani, R. J., R. F. Barber, E. Candes, and A. Ramdas. 2019. “Conformal prediction under covariate shift.” Adv. Neural Inf. Process. Syst. 32.
Xiang, S., and H. Li. 2017. On the effects of batch and weight normalization in generative adversarial networks. New York: Cornell Univ.
Yang, Y., A. K. Kuchibhotla, and E. T. Tchetgen. 2022. “Doubly robust calibration of prediction sets under covariate shift.” Preprint, submitted March 3, 2022. https://arXiv.org/abs/2203.01761.
Yao, D., Q. Sun, J. Yang, H. Liu, and J. Zhang. 2020. “Railway fastener fault diagnosis based on generative adversarial network and residual network model.” In Shock and vibration 2020. London: Hindawi.
Zheng, D., L. Li, S. Zheng, X. Chai, S. Zhao, Q. Tong, J. Wang, and L. Guo. 2021. “A defect detection method for rail surface and fasteners based on deep convolutional neural network.” In Computational intelligence and neuroscience. London: Hindawi.
Zhu, M., P. Pan, W. Chen, and Y. Yang. 2019. “DM-GAN: Dynamic memory generative adversarial networks for text-to-image synthesis.” In Proc., IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 5795–5803. New York: IEEE.
Zhu, P., R. Abdal, J. Femiani, and P. Wonka. 2021. “Barbershop: GAN-based image compositing using segmentation masks.” ACM Trans. Graph. 41 (1): 1–13. https://doi.org/10.1145/3476828.
Information & Authors
Information
Published In
Journal of Transportation Engineering, Part A: Systems
Volume 149 • Issue 3 • March 2023
Copyright
© 2022 American Society of Civil Engineers.
History
Received: Feb 28, 2022
Accepted: Oct 17, 2022
Published online: Dec 30, 2022
Published in print: Mar 1, 2023
Discussion open until: May 30, 2023
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Jaya Kumari, Ramin Karim, Use Cases of Generative AI in Asset Management of Railways, International Congress and Workshop on Industrial AI and eMaintenance 2023, 10.1007/978-3-031-39619-9_2, (15-29), (2024).