Validation of models with constant bias: an applied approach

Validation of models with constant bias: an applied approach

ABSTRACT

Objective. This paper presents extensions to the statistical validation method based on the procedure of Freese when a model shows constant bias (CB) in its predictions and illustrate the method with data from a new mechanistic model that predict weight gain in cattle. Materials and methods. The extensions were the hypothesis tests and maximum anticipated error for the alternative approach, and the confidence interval for a quantile of the distribution of errors. Results. The model evaluated showed CB, once the CB is removed and with a confidence level of 95%, the magnitude of the error does not exceed 0.575 kg. Therefore, the validated model can be used to predict the daily weight gain of cattle, although it will require an adjustment in its structure based on the presence of CB to increase the accuracy of its forecasts. Conclusions. The confidence interval for the 1-α quantile of the distribution of errors after correcting the constant bias, allows determining the top limit for the magnitude of the error of prediction and use it to evaluate the evolution of the model in the forecasting of the system. The confidence interval approach to validate a model is more informative than the hypothesis tests for the same purpose.

1. Oberkampf WL, Trucano TG. Verification and validation in computational fluid dynamics. Prog Aerosp Sci 2002; 38(3):209-72. http://dx.doi.org/10.1016/S0376-0421(02)00005-2
3. Halachmi I, Edan Y, Moallem U, Maltz E. Predicting feed intake of the individual dairy cow. J Dairy Sci 2004; 87(7):2254-67. http://dx.doi.org/10.3168/jds.S0022-0302(04)70046-6
5. Barrales VL, Pe-a R, Fernández R. Model validation: an applied approach. Agric Tech (Chile) 2004; 64:66-73.
7. Mayer DG, Butler DG. Statistical Validation. Ecol Model 1993; 68(1-2):21-32. http://dx.doi.org/10.1016/0304-3800(93)90105-2
9. Tedeschi LO. Assessment of the adequacy of mathematical models. Agr Syst 2006; 89(2-3):225-47. http://dx.doi.org/10.1016/j.agsy.2005.11.004
11. Medina-Peralta S, Vargas-Villamil L, Navarro-Alberto J, Canul-Pech C, Peraza-Romero S. Comparación de medidas de desviación para validar modelos sin sesgo, sesgo constante o proporcional. Univ Cienc 2010; 26(3):255-63.
13. Freese F. Testing accuracy. For Sci 1960; 6:139-45.
15. Rennie JC, Wiant HVJ. Modification of Freese's chi-square test of accuracy. Note BLM Denver Colorado: USDI Bureau of Land Management; 1978.
17. Reynolds MR. Estimating the Error in Model Predictions. For Sci 1984; 30(2):454-69.
19. Medina PS. Validación de modelos mecanísticos basada en la prueba ji-cuadrada de Freese, su modificación y extensión. Montecillo, Mexico: Colegio de Postgraduados; 2006.
21. Casella G., Berger RL. Statistical Inference, Pacific Grove CA USA. USA: Duxbury Thompson Learning; 2002.