Exchangeable Computer Models

Abstract

Analysts often use deterministic computer models to predict the behavior of com- plex physical systems when observational data are limited. However, inferences based partially or entirely on simulated data require adequate assessments of model uncer- tainty that can be hard to quantify. The deterministic nature of computer models limits the information we can extract from simulations to separate model signal from model error. In this paper we present a new approach to assess the uncertainty of computer models to which we refer as multi-deterministic. Evaluations from a multi- deterministic computer model can be considered to be a collection of deterministic simulators which share the same input and output space, do not present obvious the- oretical or computational advantages, and generate disparate predictions. To quantify the uncertainty of predictions from multi-deterministic models we use the construct of a latent model about which we learn from observed evaluations. We assume that outcomes from multi-deterministic models are sequences of second-order exchangeable functions (SOEF) and use Bayes linear methods to assess the latent model a posteriori. We demonstrate our methods using multi-deterministic results from a galaxy forma- tion model called Galform for which the system condition is the specification of dark matter over time and space.