Adjusting Error Calculation to Account for Temporal Mismatch in Evaluating Models

A new method is proposed to compute prediction errors so as to not penalize models for good predictions with a temporal error. Traditional goodness of fit measures are based on model error calculated by pair-wise comparison of observed and simulated values at the same time, (which the authors term isotemporal predictions). However, even good models could have a random temporal error. Relatively small time shifts in predictions could produce large individual errors, especially if there are rapid changes in the system. This could lead to misleadingly poor goodness-of-fit statistics. The authors propose a modification to the calculation of model error that pairs each data point with a prediction that is closest in a Euclidean sense, (which the authors term proxitemporal prediction), instead of the same time. A normalization criterion is proposed to make the prediction and time scales commensurate based on the slope of the model predictions. The method is tested using stochastic simulation for a sinusoidal model and applied to an uncalibrated Escherichia coli water quality model. The stochastic simulation showed that the ability of the new method to accurately capture the true goodness of fit depends on the true prediction error and the true temporal error.