-
Understanding an Acquisition Function Family for Bayesian Optimization
Authors:
Jiajie Kong,
Tony Pourmohamad,
Herbert K. H. Lee
Abstract:
Bayesian optimization (BO) developed as an approach for the efficient optimization of expensive black-box functions without gradient information. A typical BO paper introduces a new approach and compares it to some alternatives on simulated and possibly real examples to show its efficacy. Yet on a different example, this new algorithm might not be as effective as the alternatives. This paper looks…
▽ More
Bayesian optimization (BO) developed as an approach for the efficient optimization of expensive black-box functions without gradient information. A typical BO paper introduces a new approach and compares it to some alternatives on simulated and possibly real examples to show its efficacy. Yet on a different example, this new algorithm might not be as effective as the alternatives. This paper looks at a broader family of approaches to explain the strengths and weaknesses of algorithms in the family, with guidance on what choices might work best on different classes of problems.
△ Less
Submitted 16 October, 2023;
originally announced October 2023.
-
Twisted carotenoids do not support efficient intramolecular singlet fission in the orange carotenoid protein
Authors:
George A. Sutherland,
James P. Pidgeon,
Harrison Ka Hin Lee,
Matthew S. Proctor,
Andrew Hitchcock,
Shuangqing Wang,
Dimitri Chekulaev,
Wing Chung Tsoi,
Matthew P. Johnson,
C. Neil Hunter,
Jenny Clark
Abstract:
Singlet exciton fission is the spin-allowed generation of two triplet electronic excited states from a singlet state. Intramolecular singlet fission has been suggested to occur on individual carotenoid molecules within protein complexes, provided the conjugated backbone is twisted out-of-plane. However, this hypothesis has only been forwarded in protein complexes containing multiple carotenoids an…
▽ More
Singlet exciton fission is the spin-allowed generation of two triplet electronic excited states from a singlet state. Intramolecular singlet fission has been suggested to occur on individual carotenoid molecules within protein complexes, provided the conjugated backbone is twisted out-of-plane. However, this hypothesis has only been forwarded in protein complexes containing multiple carotenoids and bacteriochlorophylls in close contact. To test the hypothesis on twisted carotenoids in a 'minimal' one-carotenoid system, we study the orange carotenoid protein (OCP). OCP exists in two forms: in its orange form (OCPo), the single bound carotenoid is twisted, whereas in its red form (OCPr), the carotenoid is planar. To enable room-temperature spectroscopy on canthaxanthin-binding OCPo and OCPr without laser-induced photoconversion, we trap them in trehalose glass. Using transient absorption spectroscopy, we show that there is no evidence of long-lived triplet generation through intramolecular singlet fission, despite the canthaxanthin twist in OCPo.
△ Less
Submitted 24 November, 2022;
originally announced November 2022.
-
Modeling an Augmented Lagrangian for Blackbox Constrained Optimization
Authors:
Robert B. Gramacy,
Genetha A. Gray,
Sebastien Le Digabel,
Herbert K. H. Lee,
Pritam Ranjan,
Garth Wells,
Stefan M. Wild
Abstract:
Constrained blackbox optimization is a difficult problem, with most approaches coming from the mathematical programming literature. The statistical literature is sparse, especially in addressing problems with nontrivial constraints. This situation is unfortunate because statistical methods have many attractive properties: global scope, handling noisy objectives, sensitivity analysis, and so forth.…
▽ More
Constrained blackbox optimization is a difficult problem, with most approaches coming from the mathematical programming literature. The statistical literature is sparse, especially in addressing problems with nontrivial constraints. This situation is unfortunate because statistical methods have many attractive properties: global scope, handling noisy objectives, sensitivity analysis, and so forth. To narrow that gap, we propose a combination of response surface modeling, expected improvement, and the augmented Lagrangian numerical optimization framework. This hybrid approach allows the statistical model to think globally and the augmented Lagrangian to act locally. We focus on problems where the constraints are the primary bottleneck, requiring expensive simulation to evaluate and substantial modeling effort to map out. In that context, our hybridization presents a simple yet effective solution that allows existing objective-oriented statistical approaches, like those based on Gaussian process surrogates and expected improvement heuristics, to be applied to the constrained setting with minor modification. This work is motivated by a challenging, real-data benchmark problem from hydrology where, even with a simple linear objective function, learning a nontrivial valid region complicates the search for a global minimum.
△ Less
Submitted 3 March, 2015; v1 submitted 19 March, 2014;
originally announced March 2014.
-
Cases for the nugget in modeling computer experiments
Authors:
Robert B. Gramacy,
Herbert K. H. Lee
Abstract:
Most surrogate models for computer experiments are interpolators, and the most common interpolator is a Gaussian process (GP) that deliberately omits a small-scale (measurement) error term called the nugget. The explanation is that computer experiments are, by definition, "deterministic", and so there is no measurement error. We think this is too narrow a focus for a computer experiment and a stat…
▽ More
Most surrogate models for computer experiments are interpolators, and the most common interpolator is a Gaussian process (GP) that deliberately omits a small-scale (measurement) error term called the nugget. The explanation is that computer experiments are, by definition, "deterministic", and so there is no measurement error. We think this is too narrow a focus for a computer experiment and a statistically inefficient way to model them. We show that estimating a (non-zero) nugget can lead to surrogate models with better statistical properties, such as predictive accuracy and coverage, in a variety of common situations.
△ Less
Submitted 21 November, 2010; v1 submitted 26 July, 2010;
originally announced July 2010.
-
Optimization Under Unknown Constraints
Authors:
Robert B. Gramacy,
Herbert K. H. Lee
Abstract:
Optimization of complex functions, such as the output of computer simulators, is a difficult task that has received much attention in the literature. A less studied problem is that of optimization under unknown constraints, i.e., when the simulator must be invoked both to determine the typical real-valued response and to determine if a constraint has been violated, either for physical or policy re…
▽ More
Optimization of complex functions, such as the output of computer simulators, is a difficult task that has received much attention in the literature. A less studied problem is that of optimization under unknown constraints, i.e., when the simulator must be invoked both to determine the typical real-valued response and to determine if a constraint has been violated, either for physical or policy reasons. We develop a statistical approach based on Gaussian processes and Bayesian learning to both approximate the unknown function and estimate the probability of meeting the constraints. A new integrated improvement criterion is proposed to recognize that responses from inputs that violate the constraint may still be informative about the function, and thus could potentially be useful in the optimization. The new criterion is illustrated on synthetic data, and on a motivating optimization problem from health care policy.
△ Less
Submitted 5 July, 2010; v1 submitted 22 April, 2010;
originally announced April 2010.
-
Adaptive design and analysis of supercomputer experiments
Authors:
Robert B. Gramacy,
Herbert K. H. Lee
Abstract:
Computer experiments are often performed to allow modeling of a response surface of a physical experiment that can be too costly or difficult to run except using a simulator. Running the experiment over a dense grid can be prohibitively expensive, yet running over a sparse design chosen in advance can result in obtaining insufficient information in parts of the space, particularly when the surfa…
▽ More
Computer experiments are often performed to allow modeling of a response surface of a physical experiment that can be too costly or difficult to run except using a simulator. Running the experiment over a dense grid can be prohibitively expensive, yet running over a sparse design chosen in advance can result in obtaining insufficient information in parts of the space, particularly when the surface calls for a nonstationary model. We propose an approach that automatically explores the space while simultaneously fitting the response surface, using predictive uncertainty to guide subsequent experimental runs. The newly developed Bayesian treed Gaussian process is used as the surrogate model, and a fully Bayesian approach allows explicit measures of uncertainty. We develop an adaptive sequential design framework to cope with an asynchronous, random, agent--based supercomputing environment, by using a hybrid approach that melds optimal strategies from the statistics literature with flexible strategies from the active learning literature. The merits of this approach are borne out in several examples, including the motivating computational fluid dynamics simulation of a rocket booster.
△ Less
Submitted 25 May, 2009; v1 submitted 28 May, 2008;
originally announced May 2008.
-
Gaussian Processes and Limiting Linear Models
Authors:
Robert B. Gramacy,
Herbert K. H. Lee
Abstract:
Gaussian processes retain the linear model either as a special case, or in the limit. We show how this relationship can be exploited when the data are at least partially linear. However from the perspective of the Bayesian posterior, the Gaussian processes which encode the linear model either have probability of nearly zero or are otherwise unattainable without the explicit construction of a pri…
▽ More
Gaussian processes retain the linear model either as a special case, or in the limit. We show how this relationship can be exploited when the data are at least partially linear. However from the perspective of the Bayesian posterior, the Gaussian processes which encode the linear model either have probability of nearly zero or are otherwise unattainable without the explicit construction of a prior with the limiting linear model in mind. We develop such a prior, and show that its practical benefits extend well beyond the computational and conceptual simplicity of the linear model. For example, linearity can be extracted on a per-dimension basis, or can be combined with treed partition models to yield a highly efficient nonstationary model. Our approach is demonstrated on synthetic and real datasets of varying linearity and dimensionality.
△ Less
Submitted 13 July, 2008; v1 submitted 29 April, 2008;
originally announced April 2008.
-
Bayesian treed Gaussian process models with an application to computer modeling
Authors:
Robert B. Gramacy,
Herbert K. H. Lee
Abstract:
Motivated by a computer experiment for the design of a rocket booster, this paper explores nonstationary modeling methodologies that couple stationary Gaussian processes with treed partitioning. Partitioning is a simple but effective method for dealing with nonstationarity. The methodological developments and statistical computing details which make this approach efficient are described in detai…
▽ More
Motivated by a computer experiment for the design of a rocket booster, this paper explores nonstationary modeling methodologies that couple stationary Gaussian processes with treed partitioning. Partitioning is a simple but effective method for dealing with nonstationarity. The methodological developments and statistical computing details which make this approach efficient are described in detail. In addition to providing an analysis of the rocket booster simulator, our approach is demonstrated to be effective in other arenas.
△ Less
Submitted 17 March, 2009; v1 submitted 24 October, 2007;
originally announced October 2007.
