Abstract
Sensitivity analysis (SA) plays a vital role in hydrologic and water quality (H/WQ) model development and reliability assessment. This paper performed the first formal global SA (GSA) evaluation of the routing submodule of the Watershed Assessment Model (WAM), a spatially distributed high-dimensional H/WQ model, using a two-step GSA approach consisting of sequential application of the elementary effects (EE) parameter screening and the variance-based technique of Sobol’. The Taylor Creek Nubbin Slough basin (S-191 basin) in southcentral Florida is taken as a test case. Multioutput EE analysis successfully screened unimportant parameters, deeming the rest (27 of 297) likely influential. Sobol’ GSA with these 27 parameters showed that for flow and nutrient outputs, most of the model variability was attributed to only 12 parameters, fewer than 5% of the original parameter set. An approach to extract additional useful information about the output–parameter relationships, interactions, and monotonic/nonmonotonic behavior from the EE analysis with minimal additional calculations was explored. The proposed criteria for identifying parameter interactions from EE analysis showed good comparison with Sobol’ analysis results.
Get full access to this article
View all available purchase options and get full access to this article.
Acknowledgments
Financial assistance for this project was provided by the Center for Landscape Conservation and Ecology, Institute of Food and Agricultural Sciences, at the University of Florida; and Florida Sea Grant, with support from the National Oceanic and Atmospheric Administration (NOAA), Office of Sea Grant, US Department of Commerce, Award No. NA10OAR4170079. The authors acknowledge the High Performance Computing Center for their support. The statements, findings, conclusions, and recommendations are those of the research team and do not necessarily reflect the views of NOAA, the US Department of Commerce, or the US Government. Dr. Khare declares that most of the work presented in this article was done during author’s doctoral and postdoctoral research in the Department of Agricultural and Biological Engineering, University of Florida.
References
Ahmadi, M., J. C. Ascough, K. C. DeJonge, and M. Arabi. 2014. “Multisite-multivariable sensitivity analysis of distributed watershed models: Enhancing the perceptions from computationally frugal methods.” Ecol. Eng. 279: 54–67. https://doi.org/10.1016/j.ecolmodel.2014.02.013.
Benke, K. K., K. E. Lowell, and A. J. Hamilton. 2008. “Parameter uncertainty, sensitivity analysis and prediction error in a water-balance hydrological model.” Math. Comput. Modell. 47 (11–12): 1134–1149. https://doi.org/10.1016/j.mcm.2007.05.017.
Borah, D. K., and M. Bera. 2003. “Watershed-scale hydrologic and nonpoint-source pollution models: Review of mathematical bases.” Trans. ASAE 46 (6): 1553–1566. https://doi.org/10.13031/2013.15644.
Bottcher, A. B., and A. I. James. 2014. “Watershed assessment model (WAM): Florida applications.” In Proc., 21st Century Watershed Conf. St. Joseph, MI: American Society of Agricultural and Biological Engineers.
Bottcher, A. B., N. B. Pickering, and A. B. Cooper. 1998. “EAAMOD-FIELD: A flow and phosphorus model for high water tables.” In Proc., 7th Annual Drainage Symp., 599–606. St. Joseph, MI: ASAE.
Bottcher, A. B., B. J. Whiteley, A. I. James, and J. G. Hiscock. 2012. “Watershed Assessment Model (WAM): Model use, calibration and validation.” Tran. ASABE 55 (4): 1367–1383. https://doi.org/10.13031/2013.42248.
Byrne, M. J., and M. S. Wood. 2012. Concentrations and loads of nutrients in the tributaries of the Lake Okeechobee Watershed, south–central Florida, water years 2004–2008. Reston, VA: USGS.
Cacuci, D. G., and M. Ionescu-Bujor. 2004. “A comparative review of sensitivity and uncertainty analysis of large-scale systems—II: Statistical methods.” Nucl. Sci. Eng. 147 (3): 204–217. https://doi.org/10.13182/04-54CR.
Campolongo, F., J. Cariboni, and A. Saltelli. 2007. “An effective screening design for sensitivity analysis of large models.” Environ. Modell. Software 22 (10): 1509–1518. https://doi.org/10.1016/j.envsoft.2006.10.004.
Chebud, Y., G. M. Naja, and R. Rivero. 2011. “Phosphorus runoff assessment in a watershed.” J. Environ. Monit. 13 (1): 66–73. https://doi.org/10.1039/c0em00321b.
Chitale, J., Y. Khare, R. Muñoz-Carpena, G. Dulikravich, and C. Martinez. 2017. “An effective parameter screening strategy for high-dimensional models.” In Proc., ASME 2017 Int. Mechanical Engineering. New York: ASME.
Chow, V. T. 1959. Open-channel hydraulics, 680. New York: McGraw-Hill.
Cibin, R., K. P. Sudheer, and I. Chaubey. 2010. “Sensitivity and identifiability of stream flow generation parameters of SWAT model.” Hydrol. Processes 24 (9): 1133–1148. https://doi.org/10.1002/hyp.7568.
Daniel, E. B., J. V. Camp, E. J. LeBoeuf, J. R. Penrod, J. P. Dobbins, and M. D. Abkowitz. 2011. “Watershed modeling and its applications: A state of the art review.” Open Hydrol. J. 5 (1): 26–50. https://doi.org/10.2174/1874378101105010026.
DHI (Danish Hydraulic Institute). 2017. “AutoCal: Auto calibration tool: User guide.” Accessed April 15, 2018. http://manuals.mikepoweredbydhi.help/2017/General/AutoCal.pdf.
EC (European Commission). 2009. “Impact assessment guidelines: Technical report 92.” Accessed September 19, 2018. http://ec.europa.eu/smart-regulation/impact/commission_guidelines/docs/iag_2009_en.pdf.
FDOT (Florida Department of Transport). 2004. Drainage hankbook hydrology. Tallahassee, FL: FDOT.
Ferretti, F., A. Saltelli, and S. Tarantola. 2016. “Trends in sensitivity analysis in the last decade.” Sci. Total Environ. 568: 666–670. https://doi.org/10.1016/j.dcitotenv.2016.02.133.
Foglia, L., M. C. Hill, S. W. Mehl, and P. Burlando. 2009. “Sensitivity analysis, calibration, and testing of a distributed hydrological model using error-based weighting and one objective function.” Water Resour. Res. 45 (6): W06427. https://doi.org/10.1029/2008WR007255.
Foglia, L., M. C. Hill, S. W. Mehl, P. Perona, and P. Burlando. 2010. “Identifying important observations using cross-validation and computationally frugal sensitivity analysis methods.” Procedia Soc. Behav. Sci. 2 (6): 7650–7651. https://doi.org/10.1016/j.sbspro.2010.05.161.
Fonseca, A., D. P. Ames, P. Yang, C. Botelho, R. Boaventura, and V. Vilar. 2014. “Watershed model parameter estimation and uncertainty in data-limited environments.” Environ. Modell. Software 51: 84–93. https://doi.org/10.1016/j.envsoft.2013.09.023.
Francos, A., F. J. Elorza, F. Bouraoui, G. Bidoglio, and L. Galbiati. 2003. “Sensitivity analysis of distributed environmental simulation models: Understanding the model behavior in hydrological studies at the catchment scale.” Reliab. Eng. Syst. Safe 79 (2): 205–218. https://doi.org/10.1016/S0951-8320(02)00231-4.
Gan, Y., Q. Duan, W. Gong, C. Tong, Y. Sun, W. Chu, A. Ye, C. Miao, and Z. Di. 2014. “A comprehensive evaluation of various sensitivity analysis methods: A case study with a hydrologic model.” Environ. Modell. Software 51: 269–285. https://doi.org/10.1016/j.envsoft.2013.09.031.
Gao, L., and B. A. Bryan. 2016. “Incorporating deep uncertainty into the elementary effects method for robust global sensitivity analysis.” Ecol. Modell. 321: 1–9. https://doi.org/10.1016/j.ecolmodel.2015.10.016.
Graham, W. D., A. S. Donigian, R. Muñoz-Carpena, W. Skaggs, and A. Shirmohammadi. 2009. Peer review of the watershed assessment model. West Palm Beach, FL: South Florida Water Management District.
Guse, B., D. E. Reusser, and N. Fohrer. 2014. “How to improve the representation of hydrological processes in SWAT for a lowland catchment: Temporal analysis of parameter sensitivity and model performance.” Hydrol. Processes 28 (4): 2651–2670. https://doi.org/10.1002/hyp.9777.
Guzman, J. A., A. Shirmohammadi, A. M. Sadeghi, X. Wang, M. L. Chu, M. K. Jha, P. B. Parajuli, D. Harmel, Y. P. Khare, and J. E. Hernandez. 2015. “Uncertainty considerations in calibration and validation of hydrologic and water quality models.” Trans. ASABE 58 (6): 1745–1762. https://doi.org/10.13031/trans.58.10710.
Haan, C. T., D. E. Storm, T. Al-Issa, S. Prabhu, G. J. Sabbagh, and D. R. Edwards. 1998. “Effect of parameter distribution on uncertainty analysis of hydrologic models.” Trans. ASAE 41 (1): 65–70. https://doi.org/10.13031/2013.17158.
Helton, J. C., and F. J. Davis. 2002. “Illustration of sampling based methods for uncertainty and sensitivity analysis.” Risk Anal. 22 (3): 591–622. https://doi.org/10.1111/0272-4332.00041.
Hendricks, G., S. Shukla, C. Martinez, and G. Kiker. 2013. “Modified model for simulating hydrologic processes for plastic mulch production system.” J. Irrig. Drain. Eng. 139 (9): 738–746. https://doi.org/10.1061/(ASCE)IR.1943-4774.0000615.
Herman, J. D., J. B. Kollat, P. M. Reed, and T. Wagener. 2013. “Technical note: Method of Morris effectively reduces the computational demands of global sensitivity analysis for distributed watershed models.” Hydrol. Earth Syst. Sci. 17 (7): 2893–2903. https://doi.org/10.5194/hess-17-2893-2013.
Hill, M. C., D. Kavetski, M. Clark, M. Ye, M. Arabi, D. Lu, L. Foglia, and S. Mehl. 2016. “Practical use of computationally frugal model analysis methods.” Groundwater 54 (2): 159–170. https://doi.org/10.1111/gwat.12330.
Jacobson, B., and A. B. Bottcher. 1998. Unique routing algorithm for watershed assessment modeling. St. Joseph, MI: ASAE.
Kadlec, R. H., and S. D. Wallace. 2009. Treatment wetlands. 2nd ed. Boca Raton, FL: CRC.
Keener, V. W., K. T. Ingram, B. Jacobson, and J. W. Jones. 2007. “Effect of El Niño/Southern Oscillation on simulated phosphorus loading in South Florida.” Trans. ASABE 50 (6): 2081–2089. https://doi.org/10.13031/2013.24110.
Kelleher, C., B. McGlynn, and T. Wagener. 2017. “Characterizing and reducing equifinality by constraining a distributed catchment model with regional signatures, local observations, and process understanding.” Hydrol. Earth Syst. Sci. 21 (7): 3325–3352. https://doi.org/10.5194/hess-21-3325-2017.
Khare, Y. P. 2014. “Hydrologic and water quality model evaluation with global sensitivity analysis: Improvements and applications.” Ph.D. dissertation, Dept. of Agricultural and Biological Engineering, Univ. of Florida.
Khare, Y. P., R. Muñoz-Carpena, R. Rooney, and C. J. Martinez. 2015. “A multi-criteria trajectory-based parameter sampling strategy for the screening method of elementary effects.” Environ. Modell. Software 64: 230–239. https://doi.org/10.1016/j.envsoft.2014.11.013.
Kisekka, I., K. W. Migliaccio, R. Muñoz-Carpena, Y. P. Khare, and T. H. Boyer. 2013. “Sensitivity analysis and parameter estimation for an approximate analytical model of canal-aquifer interactions applied in the C-111 basin.” Trans. ASABE 56 (3): 977–992. https://doi.org/10.13031/trans.56.10037.
Kisekka, I., K. W. Migliaccio, R. Muñoz-Carpena, B. Schaffer, and Y. P. Khare. 2015. “Modeling soil moisture dynamics considering measurement uncertainty.” Hydrol. Processes 29 (5): 692–711. https://doi.org/10.1002/hyp.10173.
Knisel, W. G., ed. 1993. GLEAMS: Groundwater loading effects of agricultural management systems. Tifton, GA: Univ. of Georgia.
Liu, Y., I. Chaubey, L. C. Bowling, V. F. Bralts, and B. A. Engel. 2016. “Sensitivity and uncertainty of the L-THIA-LID 2.1 model.” Water Resour. Manage. 30 (13): 4927–4949. https://doi.org/10.1007/s11269-016-1462-z.
Manache, G., and C. S. Melching. 2008. “Identification of reliable regression and correlation based sensitivity measures for importance ranking of water quality model parameters.” Environ. Modell. Software 23 (5): 549–562. https://doi.org/10.1016/j.envsoft.2007.08.001.
McCormick, P., R. T. James, and J. Zhang. 2010. Chapter 10: Lake Okeechobee protection program: State of the lake and watershed. West Palm Beach, FL: South Florida Water Management District.
Migliaccio, K. W., and P. Srivastava. 2007. “Hydrologic components of watershed-scale models.” Trans. ASABE 50 (5): 1695–1703. https://doi.org/10.13031/2013.23955.
Moreau, P., V. Viaud, V. Parnaudeau, J. Salmon-Monviola, and P. Durand. 2013. “An approach for global sensitivity analysis of a complex environmental model to spatial inputs and parameters: A case study of an agro-hydrological model.” Environ. Modell. Software 47: 74–87. https://doi.org/10.1016/j.envsoft.2013.04.006.
Morris, M. D. 1991. “Factorial sampling plans for preliminary computational experiments.” Technometric 33 (2): 161–174. https://doi.org/10.1080/00401706.1991.10484804.
Muleta, M., and J. W. Nicklow. 2005. “Sensitivity and uncertainty analysis coupled with automatic calibration for a distributed watershed model.” J. Hydrol. 306 (1–4): 127–145. https://doi.org/10.1016/j.jhydrol.2004.09.005.
Muñoz-Carpena, R., Z. Zajac, and Y. M. Kuo. 2007. “Global sensitivity and uncertainty analyses of the water quality model VFSMOD-W.” Trans. ASABE 50 (5): 1719–1732.
Nash, J. E., and J. V. Sutcliffe. 1970. “River flow forecasting through conceptual models part I—A discussion of principles.” J. Hydrol. 10 (3): 282–290. https://doi.org/10.1016/0022-1694(70)90255-6.
Neitsch, S. L., J. G. Arnold, J. R. Kiniry, and J. R. Williams. 2011. Soil and water assessment tool theoretical documentation version 2009. College Station, TX: Texas Water Resources Institute.
Nossent, J., P. Elsen, and W. Bauwens. 2011. “Sobol’ sensitivity analysis of a complex environmental model.” Environ. Modell. Software 26 (12): 1515–1525. https://doi.org/10.1016/j.envsoft.2011.08.010.
Pianosi, F., K. Beven, J. Freer, J. W. Hall, J. Rougier, D. B. Stephenson, and T. Wagener. 2016. “Sensitivity analysis of environmental models: A systematic review with practical workflow.” Environ. Modell. Software 79: 214–232. https://doi.org/10.1016/j.envsoft.2016.02.008.
Razavi, S., and H. V. Gupta. 2015. “What do we mean by sensitivity analysis? The need for comprehensive characterization of ‘global’ sensitivity in Earth and environmental system models.” Water Resour. Res. 51 (5): 3070–3092. https://doi.org/10.1002/2014WR016527.
Ruano, M. V., J. JRibes, A. Seco, and J. Ferrer. 2012. “An improved sampling strategy based on trajectory design for application of the Morris method to systems with many input factors.” Environ. Modell. Software 37: 103–109. https://doi.org/10.1016/j.envsoft.2012.03.008.
Rutherford, J. C., R. B. Williamson, and A. B. Cooper. 1989. “Nitrogen, phosphorus, and oxygen dynamics in rivers.” In Inland waters of New Zealand, edited by A. B. Viner, 139–166. Wellington, New Zealand: DSIR Science Information Publishing Centre.
Saltelli, A. 2002. “Making best use of model valuations to compute sensitivity indices.” Comput. Phys. Commun. 145 (2): 280–297. https://doi.org/10.1016/S0010-4655(02)00280-1.
Saltelli, A., and P. Annoni. 2010. “How to avoid a perfunctory sensitivity analysis.” Environ. Modell. Software 25 (12): 1508–1517. https://doi.org/10.1016/j.envsoft.2010.04.012.
Saltelli, A., M. Ratto, T. Andres, F. Campolongo, J. Cariboni, D. Galtelli, M. Saisana, and S. Tarantola. 2008. Global sensitivity analysis: The primer. Chichester, UK: Wiley.
Saltelli, A., M. Ratto, S. Tarantola, and F. Campolongo. 2005. “Sensitivity analysis for chemical models.” Chem. Rev. 105 (7): 2811–2828. https://doi.org/10.1021/cr040659d.
SFWMD (South Florida Water Management District). 2010. Nutrient budget analysis for the Lake Okeechobee watershed. West Palm Beach, FL: SFWMD.
Shin, M.-J., J. H. A. Guillaume, B. F. W. Croke, and A. J. Jakeman. 2013. “Addressing ten questions about conceptual rainfall-runoff models with global sensitivity analysis in R.” J. Hydrol. 503: 135–152. https://doi.org/10.1016/j.jhydrol.2013.08.047.
Singh, V. P. 2012. Computer models of watershed hydrology. Highland Ranch, CO: Water Resources.
Singh, V. P., and D. K. Frevert. 2006. Watershed models. Boca Raton, FL: Taylor and Francis.
Sobol’, I. M. 1993. “Sensitivity analysis for non-linear mathematical models.” Math. Modell. Comput. Exp. 1 (4): 407–414.
Song, X., J. Zhang, C. Zhan, Y. Xuan, M. Ye, and C. Xu. 2015. “Global sensitivity analysis in hydrologic modeling: Review of concepts, methods, theoretical framework, and applications.” J. Hydrol. 523: 739–757. https://doi.org/10.1016/j.jhydrol.2015.02.013.
Srivastava, V., W. D. Graham, R. Muñoz-Carpena, and R. M. Maxwell. 2014. “Insights on geologic and vegetative control over hydrologic behavior of a large complex basin: Global sensitivity analysis of an integrated parallel hydrologic model.” J. Hydrol. 519: 2238–2257. https://doi.org/10.1016/j.jhydrol.2014.10.020.
Sudheer, K. P., G. Lakshmi, and I. Chaubey. 2011. “Application of a pseudo simulator to evaluate the sensitivity of parameters in complex watershed models.” Environ. Modell. Software 26 (2): 135–143. https://doi.org/10.1016/j.envsoft.2010.07.007.
Sun, X. Y., L. T. H. Newham, B. F. W. Croke, and J. P. Norton. 2012. “Three complementary methods for sensitivity analysis of a water quality model.” Environ. Modell. Software 37: 19–29. https://doi.org/10.1016/j.envsoft.2012.04.010.
SWET (Soil and Water Engineering Technology). 2009. WAM enhancement and application in the Lake Okeechobee Watershed. Gainesville, FL: SWET.
SWET (Soil and Water Engineering Technology). 2011. Watershed assessment model documentation and validation: Model Calibration and Validation Report. Gainesville, FL: SWET.
SWET (Soil and Water Engineering Technology). 2017. “Watershed assessment model: Model documentation.” Accessed August 27, 2017. http://www.swet.com/documentation/.
Tsai, L. Y., C. F. Chen, C. H. Fan, and J. Y. Lin. 2017. “Using the HSPF and SWMM models in a high pervious watershed and estimating their parameter sensitivity.” Water 9 (10): 780. https://doi.org/10.3390/w9100780.
UF HPC (High Performance Computing Facility of the University of Florida). 2014. “High Performance Computing Resources at the Research Computing Center, University of Florida.” Accessed August 29, 2014. http://researchcomputing.ufl.edu.
USEPA (US Environmental Protection Agency). 2009. “Guidance on the development, evaluation, and application of environmental models.” Accessed September 19, 2018. https://www.epa.gov/sites/production/files/2015-04/documents/cred_guidance_0309.pdf.
USEPA (US Environmental Protection Agency). 2012. “Nutrient water quality standards for the state of Florida, Athens, GA.” Accessed September 19, 2018. https://www.epa.gov/sites/production/files/documents/factsheet-fdep-approval-final-12-03-12_0.pdf.
USEPA (US Environmental Protection Agency). 2015. “Ecosystem research: Watershed and water quality modeling technical center, Athens, GA.” Accessed September 19, 2018. https://www.epa.gov/sites/production/files/2015-10/documents/toolbox-overview.pdf.
USGS (US Geological Survey). 2014. “National water data for the nation.” Accessed August 12, 2014. http://waterdata.usgs.gov/nwis.
van Griensven, A., T. Meixner, S. Grunwald, T. Bishop, M. Diluzio, and R. Srinivasan. 2006. “A global sensitivity analysis tool for the parameters of multi-variable catchment models.” J. Hydrol. 324 (1–4): 10–23. https://doi.org/10.1016/j.jhydrol.2005.09.008.
Vanrolleghem, P. A., G. Mannina, A. Cosenza, and M. B. Nuemann. 2015. “Global sensitivity analysis for urban water quality modelling: Terminology, convergence and comparison of different methods.” J. Hydrol. 522: 339–352. https://doi.org/10.1016/j.hydrol.2014.12.056.
Wainwright, H. M., S. Finsterle, Y. Jung, Q. Zhou, and J. T. Birkholzer. 2014. “Making sense of global sensitivity analyses.” Comput Geosci. 65: 84–94. https://doi.org/10.1016/j.cageo.2013.06.006.
Wallace, S. D., and R. L. Knight. 2006. Small scale constructed wetland treatment systems feasibility, design criteria, operation, and maintenance requirement. London: IWA Publishing.
Wang, X., S. R. Potter, J. R. Williams, J. D. Atwood, and T. Pitts. 2006. “Sensitivity analysis of APEX for national assessment.” Trans. ASABE 49 (3): 679–688. https://doi.org/10.13031/2013.20487.
Yang, J., Y. Liu, W. Yang, and Y. Chen. 2012. “Multi-objective sensitivity analysis of a fully distributed model WetSpa.” Water Resesour. Manage. 26 (1): 109–128. https://doi.org/10.1007/s11269-011-9908-9.
Yuan, Y., Y. P. Khare, X. Wang, P. Parajuli, I. Kisseka, and S. Finsterle. 2015. “Hydrologic and water quality models: Sensitivity analysis.” Trans. ASABE 58 (6): 1721–1744. https://doi.org/10.13031/trans.58.10611.
Zajac, Z., B. Revilla-Romeo, P. Salamon, P. Burek, F. A. Hipra, and H. Beck. 2017. “The impact of lake and reservoir parameterization on global streamflow simulation.” J. Hydrol. 548: 552–586. https://doi.org/10.1016/j.jhydrol.2017.03.022.
Zhan, C.-S., X.-M. Song, J. Xia, and C. Tong. 2013. “An efficient integrated approach for global sensitivity analysis of hydrological model parameters.” Environ. Modell. Software 41: 39–52. https://doi.org/10.1016/j.envsoft.2012.10.009.
Zhang, J., and S. I. Gornak. 1999. “Evaluation of field scale water quality models for the Lake Okeechobee regulatory program.” Trans. ASAE 15 (5): 441–447. https://doi.org/10.13031/2013.5802.
Zhang, J., J. G. Hiscock, A. B. Bottcher, B. Jacobson, and P. J. Bohlen. 2006. “Modeling phosphorus load reduction of agricultural water management practice on a beef cattle ranch.” In Proc., ASABE Annual Int. Meet: Paper Number 062010. St. Joseph, MI: American Society of Agricultural and Biological Engineers.
Zhang, J., and B. Sharfstein. 2013. Chapter 8: Lake Okeechobee Watershed protection program. West Palm Beach, FL: SFWMD.
Information & Authors
Information
Published In
Journal of Hydrologic Engineering
Volume 24 • Issue 1 • January 2019
Copyright
©2018 American Society of Civil Engineers.
History
Received: Dec 1, 2017
Accepted: Jul 12, 2018
Published online: Oct 29, 2018
Published in print: Jan 1, 2019
Discussion open until: Mar 29, 2019
ASCE Technical Topics:
- Analysis (by type)
- Basins
- Bodies of water (by type)
- Business management
- Energy engineering
- Engineering fundamentals
- Environmental engineering
- Flood routing
- Floods
- Hydrologic models
- Management methods
- Mathematics
- Models (by type)
- Parameters (statistics)
- Practice and Profession
- Quality control
- Sensitivity analysis
- Statistics
- Water and water resources
- Water management
- Water quality
- Water treatment
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.