Wednesday, December 10
Mesa Laboratory, Directors Conference Room
Lecture 12:00pm (Bring your lunch)
Some Geometric Properties of the Mean Square Error,
Performance of Models and Their Averages
Given a collection of computational models that all estimate values of the same natural process, we compare the performance of the average of the collection to the individual member whose estimates are nearest a given set of observed data. Performance is the ability of a model, or average, to reproduce a set of observations of the process according to a mean square error criterion, which is the squared distance between a vector of model estimates of the process and a vector of corresponding observations normalized by the number of observations. We identify necessary and sufficient conditions for a single model to perform better than the average on a set of observations. We also give sharp bounds for the performance of the average, again on a given interval. We illustrate our results through an analysis of regional climate model data, and we conclude with a caution about relying too heavily on performance metrics in the absence of statistics.