Evaluating Autoselection Methods Used for Choosing Solutions from Pareto-Optimal Set: Does Nondominance Persist from Calibration to Validation Phase?
Publication: Journal of Hydrologic Engineering
Volume 17, Issue 1
Abstract
The calibration of hydrological models using multiobjective algorithms generates several competitive solutions, usually referred to as a Pareto-optimal set. Pareto-optimal solutions are nondominated (i.e., accurate in different ways) and give users several decision scenarios and alternative trade-offs between model evaluation objectives. For decision-making purposes, a single solution is often chosen to represent the properties of the problem under consideration. From a practical standpoint, users are interested to know if selected solutions will continue to be nondominated when evaluated for future periods. In this paper demonstrates an evaluation framework to compare four autoselection methods commonly applied to select a solution from the Pareto-optimal set. The Pareto-optimal sets were generated by using the nondominated sorting genetic algorithm-II (NSGA-II) to calibrate the soil and water assessment tool (SWAT) for simulations of streamflow in the Fairchild Creek watershed in southern Ontario, Canada. The analysis was conducted for 15 calibration outputs in different periods, and each output was evaluated for another 15 different validation periods, resulting in a total of 225 evaluations for each autoselection method. Only a subset of nondominated solutions during the calibration phase remain equally accurate when evaluated at a future time. The results showed that a selection criteria based on a compromise between a representative pathway in parameter space and a dominant variability in objective space is important to finding solutions that remain nondominated for several validation periods. That is, the most suitable solutions are those that have commonalities in parameter space and whose responses at the watershed outlet are similar.
Get full access to this article
View all available purchase options and get full access to this article.
Acknowledgments
This work is supported by the Natural Sciences and Engineering Research Council of CanadaNSERC and the Canadian Foundation for Climate and Atmospheric Sciences. thank the anonymous reviewers for their comments and efforts, which have improved this paper.
References
Arnold, J. G., and Fohrer, N. (2005). “Swat2000: Current capabilities and research opportunities in applied watershed modelling.” Hydrol. Processes, 19(3), 563–572.
Arnold, J. G., Srinivasan, R., Muttiah, R. S., and Williams, J. R. (1998). “Large area hydrologic modeling and assessment part 1: Model development.” J. Am. Water Resour. Assoc., 34(1), 73–89.
Azzama, M., and Mousab, A. A. (2010). “Using genetic algorithm and TOPSIS technique for multiobjective reactive power compensation.” Electr. Power Syst. Res., 80(6), 675–681.
Bekele, E. G., and Nicklow, J. W. (2007). “Multi-objective automatic calibration of SWAT using NSGA-II.” J. Hydrol. (Amsterdam), 341(3–4), 165–176.
Beven, K. J. (2000). “Uniqueness of place and process representation in hydrological modeling.” Hydrol. Earth Syst. Sci., 4(2), 203–213.
Beven, K. (2001). Rainfall runoff modelling: The primer, Wiley, Chichester, UK.
Beven, K. (2006). “A manifesto for the equifinality thesis.” J. Hydrol. (Amsterdam), 320(1–2), 18–36.
Borah, D. K., and Bera, M. (2003a). “Swat model background and application reviews.” Proc., ASAE Annual Int. Meeting, American Society of Agricultural Engineers, St. Joseph, MI.
Borah, D. K., and Bera, M. (2003b). “Watershed-scale hydrologic and nonpoint source pollution models: Review of mathematical bases.” Trans. ASAE, 46(6), 1553–1566.
Coello Coello, C. A., Van Veldhuizen, D. A., and Lamont, G. B. (2002). Evolutionary algorithms for solving multi-objective problems, Kluwer Academic/Plenum, New York.
Confesor, R. B., and Whittaker, G. W. (2007). “Automatic calibration of hydrologic models with multi-objective evolutionary algorithm and Pareto optimization.” J. Am. Water Resour. Assoc., 43(4), 981–989.
Crispim, J. A., and de Sousa, J. P. (2009). “Partner selection in virtual enterprises: A multi-criteria decision support approach.” Int. J. Prod. Res., 47(17), 4791–4812.
Deb, K. (2001). Multi-objective optimization using evolutionary algorithms, Wiley, Chichester, NY.
Deb, K., Agrawal, S., Pratap, A., and Meyarivan, T. (2000). “A fast elitist nondominated sorting genetic algorithms for multi-objective optimization: NSGA-II.” Parallel problem solving from Nature VI (PPSN-VI), lecture notes in computer science, Vol. 1917, Springer, Paris, 849–858.
Deb, K., and Goel, T. (2001). “Controlled elitist non-dominated sorting genetic algorithms for better convergence.” Evolutionary multicriterion optimization, lecture notes in computer science, Vol. 1993/2001, Springer, Zurich, Switzerland, 67–81.
Deb, K., and Gupta, H. (2005). “Searching for robust Pareto-optimal solutions in multi-objective optimization.” Evolutionary multi-criterion optimization, Vol. 3410, Springer, Berlin, 150–164.
Deb, K., Pratap, A., Agrawal, S., and Meyarivan, T. (2002). “A fast and elitist multiobjective genetic algorithm: NSGA-II.” IEEE Trans. Evol. Comput., 6(2), 182–197.
Dumedah, G., Berg, A. A., Wineberg, M., and Collier, R. (2010). “Selecting model parameter sets from a trade-off surface generated from the nondominated sorting genetic algorithm-II.” Water Resour. Manage., 24(15), 4469–4489.
Efstratiadis, A., and Koutsoyiannis, D. (2010). “One decade of multi-objective calibration approaches in hydrological modelling: A review.” Hydrol. Sci. J., 55(1), 58–78.
Fenicia, F., Savenije, H. H. G., Matgen, P., and Pfister, L. (2008). “Understanding catchment behavior through stepwise model concept improvement.” Water Resour. Res., 44(1), W01402.
Ferreira, J. C., Fonseca, C. M., and Gaspar-Cunha, A. (2007). “Methodology to select solutions from the Pareto-optimal set: A comparative study.” Proc., 9th Annual Conf. on Genetic and Evolutionary Computation, GECCO, London, 789–796.
Gill, M. K., Kaheil, Y. H., Khalil, A., McKee, M., and Bastidas, L. (2006). “Multiobjective particle swarm optimization for parameter estimation in hydrology.” Water Resour. Res., 42(7), W07417.
Green, W. H., and Ampt, G. A. (1911). “Studies on soil physics, 1: The flow of air and water through soils.” J. Agric. Sci., 4(01), 1–24.
Grierson, D. E. (2008). “Pareto multi-criteria decision making.” Adv. Eng. Inform., 22(3), 371–384.
Gupta, H., Sorooshian, S., and Yapo, P. O. (1998). “Toward improved calibration of hydrologic models: Multiple and noncommensurable measures of information.” Water Resour. Res., 34(4), 751–763.
Khu, S. T., and Madsen, H. (2005). “Multiobjective calibration with Pareto preference ordering: An application to rainfall-runoff model calibration.” Water Resour. Res., 41(3), W03004.
Liu, Y., and Yang, W. (2007). “An interface of drainage division for modeling wetlands and riparian buffers in agricultural watersheds.” J. Spatial Hydrol., 7(1), 66–80.
Madsen, H. (2003). “Parameter estimation in distributed hydrological catchment modelling using automatic calibration with multiple objectives.” Adv. Water Resour., 26(2), 205–216.
Nazemi, A., Yao, X., and Chan, A. (2006). “Extracting a set of robust Pareto-optimal parameters for hydrologic models using NSGA-II and SCEM.” IEEE Congress on Evolutionary Computation, IEEE, New York, 1901–1908.
Neitsch, S. L., Arnold, J. G., Kiniry, J. R., and Williams, J. R. (2001). “Soil and water assessment tool: Theoretical documentation, version 2000.” Soil and Water Research Service, Temple, TX.
Shafii, M., and Smedt, F. D. (2009). “Multi-objective calibration of a distributed hydrological model (wetspa) using a genetic algorithm.” Hydrol. Earth Syst. Sci. Discuss., 6(1), 243–271.
Taboada, H., and Coit, D. (2006). “Data mining techniques to facilitate the analysis of the Pareto-optimal set for multiple objective problems.” Proc., Industrial Engineering Research Conf., Institute of Industrial Engineers, Norcross, GA.
Tang, Y., Reed, P., and Wagener, T. (2006). “How effective and efficient are multiobjective evolutionary algorithms at hydrologic model calibration?” Hydrol. Earth Syst. Sci., 10(2), 289–307.
Thorndike, R. L. (1953). “Who belong in the family?” Psychometrika, 18(4), 267–276.
Tolson, B. A., and Shoemaker, C. A. (2007). “Cannonsville reservoir watershed SWAT2000 model development, calibration and validation.” J. Hydrol. (Amsterdam), 337(1–2), 68–86.
Wang, Y., and He, Z. (2008). “Improved TOPSIS methods for multi-response optimization.” IEEE Symp. on Advanced Management of Information for Globalized Enterprises, IEEE, New York, 1–3.
Whling, T., Vrugt, J. A., and Barkle, G. F. (2008). “Comparison of three multiobjective optimization algorithms for inverse modeling of vadose zone hydraulic properties.” Soil Sci. Soc. Am. J., 72(2), 305–319.
Information & Authors
Information
Published In
Copyright
© 2012 American Society of Civil Engineers.
History
Received: Sep 21, 2010
Accepted: Feb 3, 2011
Published online: Dec 15, 2011
Published in print: Jan 1, 2012
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.