Geographical Transferability of Pretrained K-Means Clustering–Artificial Neural Network Model for Disaggregation of Rainfall Data in an Indian Monsoon Climate
Publication: Journal of Hydrologic Engineering
Volume 28, Issue 12
Abstract
High temporal resolution rainfall data are among the most demanded resources for water resource engineers. In modern times, this need has only multiplied day by day due to the need for training large parameter-heavy models for the prediction of climatic features, analysis of extreme rainfall, etc. However, the availability of such high temporal resolution data is low, which can cause hindrances in research or development projects in several regions. It is therefore imperative to find newer and better models for the disaggregation of rainfall data from lower to higher temporal resolutions, such as a model that uses deep learning neural networks. The main issue with such a model is the requirement for historical rainfall data at different time scales for training, testing, and validating prior to use in practical scenarios, data that may not always be available for all regions necessary. In this paper, an attempt has been to test the accuracy and applicability of pretrained models for the purpose of disaggregating rainfall in other geographical locations, thus reducing the requirement for historical rainfall data for training and validation purposes. A large data set comprising rainfall data from 68 rain gauge stations across the Indian subcontinent has been used to test models pretrained using rainfall data from seven major stations in India (Bikaner, Chennai, Cherrapunji, Delhi, Kolkata, Mumbai, and Mangalore). The pretrained models are tested in their ability to conserve extreme rainfall characteristics by comparing intensity–duration–frequency (IDF) curves generated from observed and disaggregated rainfall, further which the errors in these IDF curves are used to generate heatmaps for the country using the inverse distance weighted interpolation method. At the end of this paper, a map is provided that covers the entire country of study, detailing that a pretrained model can be used for a certain region based on its accuracy of disaggregation and proximity to the city of pretraining data.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Hourly rainfall data used during the study were provided by a third party (India Meteorological Department, Pune). Direct requests for these materials may be made to the provider, as indicated in the acknowledgments. Python programs and machine learning models used for the study are available from the corresponding author upon reasonable request.
Acknowledgments
We acknowledge the research infrastructure provided by the Civil Engineering Department, Indian Institute of Engineering Science and Technology (IIEST), Shibpur. We also acknowledge the support rendered to us by India Meteorological Department, Pune, for supplying us with the necessary rainfall data, which are among the most important aspects of this study, along with the team at Tensorflow for providing the libraries needed for preparation of the model.
References
Bhattacharyya, D., and U. Saha. 2023. “Deep learning application for disaggregation of rainfall with emphasis on preservation of extreme rainfall characteristics for Indian monsoon conditions.” Stochastic Environ. Res. Risk Assess. 37 (3): 1021–1038. https://doi.org/10.1007/s00477-022-02331-x.
Burian, S. J., S. R. Durrans, S. Tomić, R. L. Pimmel, and C. N. Wai. 2000. “Rainfall disaggregation using artificial neural networks.” J. Hydrol. Eng. 5 (3): 299–307. https://doi.org/10.1061/(ASCE)1084-0699(2000)5:3(299).
Clevert, D.-A., T. Unterthiner, and S. Hochreiter. 2015. “Fast and accurate deep network learning by exponential linear units (ELUs).” Preprint, submitted January 23, 2023. http://arxiv.org/abs/1511.07289.
Cowpertwait, P. S. P. 2006. “A spatial–temporal point process model of rainfall for the Thames catchment, UK.” J. Hydrol. 330 (3–4): 586–595. https://doi.org/10.1016/j.jhydrol.2006.04.043.
Deka, P., and U. Saha. 2023. “Introduction of k-means clustering into random cascade model for disaggregation of rainfall from daily to 1-hour resolution with improved preservation of extreme rainfall.” J. Hydrol. 620 (Aug): 129478. https://doi.org/10.1016/j.jhydrol.2023.129478.
Diez-Sierra, J., S. Navas, and M. del Jesus. 2023. “NEOPRENE v1.0.1: A Python library for generating spatial rainfall based on the Neyman-Scott process.” Geosci. Model Dev. 16 (17): 5035–5048. https://doi.org/10.5194/gmd-16-5035-2023.
Econopouly, T. W., D. R. Davis, and D. A. Woolhiser. 1990. “Parameter transferability for a daily rainfall disaggregation model.” J. Hydrol. 118 (1–4): 209–228. https://doi.org/10.1016/0022-1694(90)90259-Z.
Gupta, V. K., and E. C. Waymire. 1993. “A statistical analysis of mesoscale rainfall as a random cascade.” J. Appl. Meteorol. 32 (2): 251–267. https://doi.org/10.1175/1520-0450(1993)032%3C0251:ASAOMR%3E2.0.CO;2.
Halff, A. H., H. M. Halff, and M. Azmoodeh. 1993. “Predicting runoff from rainfall using neural networks.” In Proc., Engineering Hydrology, 760–765. Reston, VA: ASCE.
Hershenhorn, J., and D. A. Woolhiser. 1987. “Disaggregation of daily rainfall.” J. Hydrol. 95 (3–4): 299–322. https://doi.org/10.1016/0022-1694(87)90008-4.
Islam, S., D. Entekhabi, R. L. Bras, and I. Rodriguez-Iturbe. 1990. “Parameter estimation and sensitivity analysis for the modified Bartlett-Lewis rectangular pulses model of rainfall.” J. Geophys. Res. 95 (D3): 2093. https://doi.org/10.1029/JD095iD03p02093.
Khaliq, M. N., and C. Cunnane. 1996. “Modelling point rainfall occurrences with the modified Bartlett-Lewis rectangular pulses model.” J. Hydrol. 180 (1–4): 109–138. https://doi.org/10.1016/0022-1694(95)02894-3.
Kingma, D. P., and J. Ba. 2014. “Adam: A method for stochastic optimization.” Preprint, submitted July 25, 2019. http://arxiv.org/abs/1412.6980.
McCulloch, W. S., and W. Pitts. 1943. “A logical calculus of the ideas immanent in nervous activity.” Bull. Math. Biophys. 5 (4): 115–133. https://doi.org/10.1007/BF02478259.
McQueen, J. B. 1967. “Some methods for classification and analysis of multivariate observations.” In Proc., of the 5th Berkeley Symp. on Mathematical Statistics and Probability, edited by L. M. L. Cam and J. Neyman. 281–297. Berkeley, CA: University of California Press.
Minns, A. W., and M. J. Hall. 1996. “Artificial neural networks as rainfall-runoff models.” Hydrol. Sci. J. 41 (3): 399–417. https://doi.org/10.1080/02626669609491511.
Misra, S., S. Sarkar, and P. Mitra. 2018. “Statistical downscaling of precipitation using long short-term memory recurrent neural networks.” Theor. Appl. Climatol. 134 (3–4): 1179–1196. https://doi.org/10.1007/s00704-017-2307-2.
Olsson, J. 1998. “Evaluation of a scaling cascade model for temporal rainfall disaggregation.” Hydrol. Earth Syst. Sci. 2 (1): 19–30. https://doi.org/10.5194/hess-2-19-1998.
Onof, C., R. E. Chandler, A. Kakou, P. Northrop, H. S. Wheater, and V. Isham. 2000. “Rainfall modelling using Poisson-cluster processes: A review of developments.” Stochastic Environ. Res. Risk Assess. 14 (6): 384–411. https://doi.org/10.1007/s004770000043.
Onof, C., and H. S. Wheater. 1993. “Modelling of British rainfall using a random parameter Bartlett-Lewis rectangular pulse model.” J. Hydrol. 149 (1–4): 67–95. https://doi.org/10.1016/0022-1694(93)90100-N.
Rodriguez-Iturbe, I., R. Cox, and V. Isham. 1987. “Some models for rainfall based on stochastic point processes.” Proc. R. Soc. London, Ser. A 410 (1839): 269–288. https://doi.org/10.1098/rspa.1987.0039.
Rosenblatt, F. 1958. “The perceptron: A probabilistic model for information storage and organization in the brain.” Psychol. Rev. 65 (6): 386–408. https://doi.org/10.1037/h0042519.
Rousseeuw, P. J. 1987. “Silhouettes: A graphical aid to the interpretation and validation of cluster analysis.” J. Comput. Appl. Math. 20 (Mar): 53–65. https://doi.org/10.1016/0377-0427(87)90125-7.
Rumelhart, D. E., G. E. Hinton, and R. J. Williams. 1986. “Learning representations by back-propagating errors.” Nature 323 (6088): 533–536. https://doi.org/10.1038/323533a0.
Scher, S., and S. Peßenteiner. 2021. “Technical note: Temporal disaggregation of spatial rainfall fields with generative adversarial networks.” Hydrol. Earth Syst. Sci. 25 (6): 3207–3225. https://doi.org/10.5194/hess-25-3207-2021.
Schertzer, D., and S. Lovejoy. 1987. “Physical modeling and analysis of rain and clouds by anisotropic scaling multiplicative processes.” J. Geophys. Res. 92 (D8): 9693. https://doi.org/10.1029/JD092iD08p09693.
Smith, J., and R. N. Eli. 1995. “Neural-network models of rainfall-runoff process.” J. Water Resour. Plann. Manage. 121 (6): 499–508. https://doi.org/10.1061/(ASCE)0733-9496(1995)121:6(499).
Thiessen, A. H. 1911. “Precipitation averages for large areas.” Mon. Weather Rev. 39 (7): 1082–1089. https://doi.org/10.1175/1520-0493(1911)39%3C1082b:PAFLA%3E2.0.CO;2.
Thirumalaiah, K., and M. C. Deo. 1998. “River stage forecasting using artificial neural networks.” J. Hydrol. Eng. 3 (1): 26–32. https://doi.org/10.1061/(ASCE)1084-0699(1998)3:1(26).
Tsintikidis, D., J. L. Haferman, E. N. Anagnostou, W. F. Krajewski, and T. F. Smith. 1997. “A neural network approach to estimating rainfall from spaceborne microwave data.” IEEE Trans. Geosci. Remote Sens. 35 (5): 1079–1093. https://doi.org/10.1109/36.628775.
Valencia, R. D., and J. C. Schakke Jr. 1973. “Disaggregation processes in stochastic hydrology.” Water Resour. Res. 9 (3): 580–585. https://doi.org/10.1029/WR009i003p00580.
Valiant, L. G. 1984. “A theory of the learnable.” Commun. ACM 27 (11): 1134–1142. https://doi.org/10.1145/1968.1972.
Vandenberghe, S., N. E. C. Verhoest, C. Onof, and B. De Baets. 2011. “A comparative copula-based bivariate frequency analysis of observed and simulated storm events: A case study on Bartlett-Lewis modeled rainfall.” Water Resour. Res. 47 (7): W07529. https://doi.org/10.1029/2009WR008388.
Wilks, D. S. 2010. “Use of stochastic weather generators for precipitation downscaling.” WIREs Clim. Change 1 (6): 898–907. https://doi.org/10.1002/wcc.85.
Wilks, D. S., and R. L. Wilby. 1999. “The weather generation game: A review of stochastic weather models.” Prog. Phys. Geogr.: Earth Environ. 23 (3): 329–357. https://doi.org/10.1177/030913339902300302.
Xiao, R., and V. Chandrasekar. 1997. “Development of a neural network based algorithm for rainfall estimation from radar observations.” IEEE Trans. Geosci. Remote Sens. 35 (1): 160–171. https://doi.org/10.1109/36.551944.
Zhang, H., X. Zhang, and B. Zhang. 2009. “System dynamics approach to urban water demand forecasting.” Transact. Tianjin Univ. 15 (1): 70–74. https://doi.org/10.1007/s12209-009-0014-5.
Information & Authors
Information
Published In
Journal of Hydrologic Engineering
Volume 28 • Issue 12 • December 2023
Copyright
© 2023 American Society of Civil Engineers.
History
Received: Apr 6, 2023
Accepted: Aug 7, 2023
Published online: Sep 29, 2023
Published in print: Dec 1, 2023
Discussion open until: Feb 29, 2024
ASCE Technical Topics:
- Artificial intelligence and machine learning
- Climates
- Computer programming
- Computing in civil engineering
- Education
- Engineering fundamentals
- Environmental engineering
- Hydrologic data
- Hydrologic engineering
- Hydrology
- Meteorology
- Model accuracy
- Models (by type)
- Neural networks
- Practice and Profession
- Precipitation
- Rain water
- Rainfall
- Training
- Water (by type)
- Water and water resources
- Water management
- Water policy
- Water resources
- Weather forecasting
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.