-
Active Discrimination Learning for Gaussian Process Models
Authors:
Elham Yousefi,
Luc Pronzato,
Markus Hainy,
Werner G. Müller,
Henry P. Wynn
Abstract:
The paper covers the design and analysis of experiments to discriminate between two Gaussian process models, such as those widely used in computer experiments, kriging, sensor location and machine learning. Two frameworks are considered. First, we study sequential constructions, where successive design (observation) points are selected, either as additional points to an existing design or from the…
▽ More
The paper covers the design and analysis of experiments to discriminate between two Gaussian process models, such as those widely used in computer experiments, kriging, sensor location and machine learning. Two frameworks are considered. First, we study sequential constructions, where successive design (observation) points are selected, either as additional points to an existing design or from the beginning of observation. The selection relies on the maximisation of the difference between the symmetric Kullback Leibler divergences for the two models, which depends on the observations, or on the mean squared error of both models, which does not. Then, we consider static criteria, such as the familiar log-likelihood ratios and the Fréchet distance between the covariance functions of the two models. Other distance-based criteria, simpler to compute than previous ones, are also introduced, for which, considering the framework of approximate design, a necessary condition for the optimality of a design measure is provided. The paper includes a study of the mathematical links between different criteria and numerical illustrations are provided.
△ Less
Submitted 21 November, 2022;
originally announced November 2022.
-
A convex approach to optimum design of experiments with correlated observations
Authors:
Andrej Pázman,
Markus Hainy,
Werner G. Müller
Abstract:
Optimal design of experiments for correlated processes is an increasingly relevant and active research topic. Present methods have restricted possibilities to judge their quality. To fill this gap, we complement the virtual noise approach by a convex formulation leading to an equivalence theorem comparable to the uncorrelated case and to an algorithm giving an upper performance bound against which…
▽ More
Optimal design of experiments for correlated processes is an increasingly relevant and active research topic. Present methods have restricted possibilities to judge their quality. To fill this gap, we complement the virtual noise approach by a convex formulation leading to an equivalence theorem comparable to the uncorrelated case and to an algorithm giving an upper performance bound against which alternative design methods can be judged. Moreover, a method for generating exact designs follows naturally. We exclusively consider estimation problems on a finite design space with a fixed number of elements. A comparison on some classical examples from the literature as well as a real application is provided.
△ Less
Submitted 22 October, 2021; v1 submitted 4 March, 2021;
originally announced March 2021.
-
Sequential Experimental Design for Predator-Prey Functional Response Experiments
Authors:
Hayden Moffat,
Markus Hainy,
Nikos E. Papanikolaou,
Christopher Drovandi
Abstract:
Understanding functional response within a predator-prey dynamic is a cornerstone for many quantitative ecological studies. Over the past 60 years, the methodology for modelling functional response has gradually transitioned from the classic mechanistic models to more statistically oriented models. To obtain inferences on these statistical models, a substantial number of experiments need to be con…
▽ More
Understanding functional response within a predator-prey dynamic is a cornerstone for many quantitative ecological studies. Over the past 60 years, the methodology for modelling functional response has gradually transitioned from the classic mechanistic models to more statistically oriented models. To obtain inferences on these statistical models, a substantial number of experiments need to be conducted. The obvious disadvantages of collecting this volume of data include cost, time and the sacrificing of animals. Therefore, optimally designed experiments are useful as they may reduce the total number of experimental runs required to attain the same statistical results. In this paper, we develop the first sequential experimental design method for predator-prey functional response experiments. To make inferences on the parameters in each of the statistical models we consider, we use sequential Monte Carlo, which is computationally efficient and facilitates convenient estimation of important utility functions. It provides coverage of experimental goals including parameter estimation, model discrimination as well as a combination of these. The results of our simulation study illustrate that for predator-prey functional response experiments sequential design outperforms static design for our experimental goals. R code for implementing the methodology is available via https://github.com/haydenmoffat/sequential_design_for_predator_prey_experiments.
△ Less
Submitted 28 April, 2020; v1 submitted 3 July, 2019;
originally announced July 2019.
-
Optimal Bayesian design for model discrimination via classification
Authors:
Markus Hainy,
David J. Price,
Olivier Restif,
Christopher Drovandi
Abstract:
Performing optimal Bayesian design for discriminating between competing models is computationally intensive as it involves estimating posterior model probabilities for thousands of simulated datasets. This issue is compounded further when the likelihood functions for the rival models are computationally expensive. A new approach using supervised classification methods is developed to perform Bayes…
▽ More
Performing optimal Bayesian design for discriminating between competing models is computationally intensive as it involves estimating posterior model probabilities for thousands of simulated datasets. This issue is compounded further when the likelihood functions for the rival models are computationally expensive. A new approach using supervised classification methods is developed to perform Bayesian optimal model discrimination design. This approach requires considerably fewer simulations from the candidate models than previous approaches using approximate Bayesian computation. Further, it is easy to assess the performance of the optimal design through the misclassification error rate. The approach is particularly useful in the presence of models with intractable likelihoods but can also provide computational advantages when the likelihoods are manageable.
△ Less
Submitted 6 April, 2022; v1 submitted 14 September, 2018;
originally announced September 2018.
-
ABC model selection for spatial extremes models applied to South Australian maximum temperature data
Authors:
Xing Ju Lee,
Markus Hainy,
James P. McKeone,
Christopher C. Drovandi,
Anthony N. Pettitt
Abstract:
Max-stable processes are a common choice for modelling spatial extreme data as they arise naturally as the infinite-dimensional generalisation of multivariate extreme value theory. Statistical inference for such models is complicated by the intractability of the multivariate density function. Nonparametric, composite likelihood-based, and Bayesian approaches have been proposed to address this diff…
▽ More
Max-stable processes are a common choice for modelling spatial extreme data as they arise naturally as the infinite-dimensional generalisation of multivariate extreme value theory. Statistical inference for such models is complicated by the intractability of the multivariate density function. Nonparametric, composite likelihood-based, and Bayesian approaches have been proposed to address this difficulty. More recently, a simulation-based approach using approximate Bayesian computation (ABC) has been employed for estimating parameters of max-stable models. ABC algorithms rely on the evaluation of discrepancies between model simulations and the observed data rather than explicit evaluations of computationally expensive or intractable likelihood functions. The use of an ABC method to perform model selection for max-stable models is explored. Three max-stable models are regarded: the extremal-t model with either a Whittle-Matérn or a powered exponential covariance function, and the Brown-Resnick model with power variogram. In addition, the non-extremal Student-t copula model with a Whittle-Matérn or a powered exponential covariance function is also considered. The method is applied to annual maximum temperature data from 25 weather stations dispersed around South Australia.
△ Less
Submitted 9 August, 2018; v1 submitted 9 October, 2017;
originally announced October 2017.
-
Likelihood-free Simulation-based Optimal Design
Authors:
Markus Hainy,
Werner G. Müller,
Helga Wagner
Abstract:
Simulation-based optimal design techniques are a convenient tool for solving a particular class of optimal design problems. The goal is to find the optimal configuration of factor settings with respect to an expected utility criterion. This criterion depends on the specified probability model for the data and on the assumed prior distribution for the model parameters. We develop new simulation-bas…
▽ More
Simulation-based optimal design techniques are a convenient tool for solving a particular class of optimal design problems. The goal is to find the optimal configuration of factor settings with respect to an expected utility criterion. This criterion depends on the specified probability model for the data and on the assumed prior distribution for the model parameters. We develop new simulation-based optimal design methods which incorporate likelihood-free approaches and utilize them in novel applications.
Most simulation-based design strategies solve the intractable expected utility integral at a specific design point by using Monte Carlo simulations from the probability model. Optimizing the criterion over the design points is carried out in a separate step. Müller (1999) introduces an MCMC algorithm which simultaneously addresses the simulation as well as the optimization problem. In principle, the optimal design can be found by detecting the utility mode of the sampled design points. Several improvements have been suggested to facilitate this task for multidimensional design problems (see e.g. Amzal et al. 2006).
We aim to extend this simulation-based design methodology to design problems where the likelihood of the probability model is of an unknown analytical form but it is possible to simulate from the probability model. We further assume that prior observations are available. In such a setting it is seems natural to employ approximate Bayesian computation (ABC) techniques in order to be able to simulate from the conditional probability model. We provide a thorough review of adjacent literature and we investigate the benefits and the limitations of our design methodology for a particular paradigmatic example.
△ Less
Submitted 18 May, 2013;
originally announced May 2013.
