Prof. Roman Garnett
Assistant Professor, Department of Computer Science and Engineering, Washington University in St. Louis
Bayesian optimization for automating model selection
Thursday, October 26, 2017, 3:00pm to 4:00pm | Room 5-314
We discuss a problem of enormous practical importance that is often ignored: the selection of appropriate prior mean/covariance functions for Gaussian process models. Despite the success of kernel-based nonparametric methods, kernel selection still requires considerable expertise, and is often described as a "black art." We present a sophisticated method for automatically searching for an appropriate kernel from an infinite space of potential choices. Previous efforts in this direction have focused on traversing a kernel grammar, only examining the data via computation of marginal likelihood. Our proposed search method is based on Bayesian optimization in model space, where we reason about model evidence as a function to be maximized. We explicitly reason about the data distribution and how it induces similarity between potential model choices in terms of the explanations they can offer for observed data. In this light, we construct a novel kernel between models to explain a given dataset (a "kernel kernel"). Our method is capable of finding a model that explains a given dataset well without any human assistance, often with fewer computations of model evidence than previous approaches, a claim we demonstrate empirically.
Roman Garnett is an assistant professor in the Computer Science and Engineering Department at Washington University in St. Louis. He was formally a postdoctoral researcher at the University of Bonn and Carnegie Mellon University, and an applied research mathematician at the National Security Agency (NSA). He received a Ph.D. in Machine Learning from the University of Oxford in 2010. His research interests include active learning (especially with atypical objectives such as active search), Bayesian optimization, and the automation of machine learning (AutoML).