skip to content

Timetable (UNQW04)

UQ for inverse problems in complex systems

Monday 9th April 2018 to Friday 13th April 2018

Monday 9th April 2018
09:00 to 09:50 Registration
09:50 to 10:00 Welcome from Christie Marr (INI Deputy Director)
10:00 to 11:00 Ilya Mandel (University of Birmingham)
Studying black holes with gravitational waves: Why GW astronomy needs you!
Following the first direct observation of gravitational waves from a pair of merging black holes in September 2015, we are now entering the era of gravitational-wave astronomy, where gravitational waves are increasingly being used as a tool to explore topics ranging from astronomy (stellar and binary evolution) to fundamental physics (tests of the general theory of relativity). Future progress depends on addressing several key problems in statistical inference on gravitational-wave observations, including (i) rapidly growing computational cost for future instruments; (ii) noise characterisation; (iii) model systematics; and (iv) model selection.
11:00 to 11:30 Morning Coffee
11:30 to 12:00 Paul Constantine (University of Colorado)
Subspace-based dimension reduction for forward and inverse uncertainty quantification
Many methods in uncertainty quantification suffer from the curse of dimensionality. I will discuss several approaches for identifying exploitable low-dimensional structure---e.g., active subspaces or likelihood-informed subspaces---that enable otherwise infeasible forward and inverse uncertainty quantification.

Related Links
12:00 to 13:30 Buffet Lunch at INI
13:30 to 14:30 Christoph Schwab (ETH Zürich)
Deterministic Multilevel Methods for Forward and Inverse UQ in PDEs
We present the numerical analysis of Quasi Monte-Carlo methods for high-dimensional integration applied to forward and inverse uncertainty quantification for elliptic and parabolic PDEs. Emphasis will be placed on the role of parametric holomorphy of data-to-solution maps. We present corresponding results on deterministic quadratures in Bayesian Inversion of parametric PDEs, and the related bound on posterior sparsity and (dimension-independent) QMC convergence rates. Particular attention will be placed on Higher-Order QMC, and on the interplay between the structure of the representation system of the distributed uncertain input data (KL, splines, wavelets,...) and the structure of QMC weights. We also review stable and efficient generation of interlaced polynomial lattice rules, and the numerical analysis of multilevel QMC Finite Element PDE discretizations with applications to forward and inverse computational uncertainty quantification. QMC convergence rates will be compared with those afforded by Smolyak quadrature. Joint work with Robert Gantner and Lukas Herrmann and Jakob Zech (SAM, ETH) and Josef Dick, Thong LeGia and Frances Kuo (Sydney). References: [1] R. N. Gantner and L. Herrmann and Ch. Schwab Quasi-Monte Carlo integration for affine-parametric, elliptic PDEs: local supports and product weights, SIAM J. Numer. Analysis, 56/1 (2018), pp. 111-135. [2] J. Dick and R. N. Gantner and Q. T. Le Gia and Ch. Schwab Multilevel higher-order quasi-Monte Carlo Bayesian estimation, Math. Mod. Meth. Appl. Sci., 27/5 (2017), pp. 953-995. [3] R. N. Gantner and M. D. Peters Higher Order Quasi-Monte Carlo for Bayesian Shape Inversion, accepted (2018) SINUM, SAM Report 2016-42. [4] J. Dick and Q. T. Le Gia and Ch. Schwab Higher order Quasi Monte Carlo integration for holomorphic, parametric operator equations, SIAM Journ. Uncertainty Quantification, 4/1 (2016), pp. 48-79 [5] J. Dick and F.Y. Kuo and Q.T. LeGia and Ch. Schwab Multi-level higher order QMC Galerkin discretization for affine parametric operator equations, SIAM J. Numer. Anal., 54/4 (2016), pp. 2541-2568
14:30 to 15:00 Richard Nickl (University of Cambridge)
Statistical guarantees for Bayesian uncertainty quantification in inverse problems
We discuss recent results in mathematical statistics that provide objective statistical guarantees for Bayesian algorithms in (possibly non-linear) noisy inverse problems. We focus in particular on the justification of Bayesian credible sets as proper frequentist confidence sets in the small noise limit via so-called `Bernstein - von Mises theorems', which provide Gaussian approximations to the posterior distribution, and introduce notions of such theorems in the infinite-dimensional settings relevant for inverse problems. We discuss in detail such a Bernstein-von Mises result for Bayesian inference on the unknown potential in the Schroedinger equation from an observation of the solution of that PDE corrupted by additive Gaussian white noise. See and also
15:00 to 15:30 Robert Scheichl (University of Bath)
Low-rank tensor approximation for sampling high dimensional distributions
High-dimensional distributions are notoriously difficult to sample from, particularly in the context of PDE-constrained inverse problems. In this talk, we will present general purpose samplers based on low-rank tensor surrogates in the tensor-train (TT) format, a methodology that has been exploited already for many years for scalable, high-dimensional function approximations in quantum chemistry. In the Bayesian context, the TT surrogate is built in a two stage process. First we build a surrogate of the entire PDE solution in the TT format, using a novel combination of alternating least squares and the TT cross algorithm. It exploits and preserves the block diagonal structure of the discretised operator in stochastic collocation schemes, requiring only independent PDE solutions at a few parameter values, thus allowing the use of existing high performance PDE solvers. In a second stage, we approximate the high-dimensional posterior density function also in TT format. Due to the particular structure of the TT surrogate, we can build an efficient conditional distribution method (or Rosenblatt transform) that only requires a sampling algorithm for one-dimensional conditionals. This conditional distribution method can also be used for other high-dimensional distributions, not necessarily coming from a PDE-constrained inverse problem. The overall computational cost and storage requirements of the sampler grow linearly with the dimension. For sufficiently smooth distributions, the ranks required for accurate TT approximations are moderate, leading to significant computational gains. We compare our new sampling method with established methods, such as the delayed rejection adaptive Metropolis (DRAM) algorithm, as well as with multilevel quasi-Monte Carlo ratio estimators. This is joint work with Sergey Dolgov (Bath), Colin Fox (Otago) and Karim Anaya-Izquierdo (Bath).
15:30 to 16:00 Afternoon Tea
16:00 to 17:00 Panel Discussion: Establishing UQ Challenge Problems & Supporting Infrastructure INI 1
17:00 to 18:00 Welcome Wine Reception at INI
Tuesday 10th April 2018
09:00 to 10:00 Youssef Marzouk (Massachusetts Institute of Technology)
Optimal Bayesian experimental design: focused objectives and observation selection strategies
I will discuss two complementary efforts in Bayesian optimal experimental design for inverse problems. The first focuses on evaluating an experimental design objective: we describe a new computational approach for ``focused'' optimal Bayesian experimental design with nonlinear models, with the goal of maximizing expected information gain in targeted subsets of model parameters. Our approach considers uncertainty in the full set of model parameters, but employs a design objective that can exploit learning trade-offs among different parameter subsets. We introduce a layered multiple importance sampling scheme that provides consistent estimates of expected information gain in this focused setting, with significant reductions in estimator bias and variance for a given computational effort. The second effort focuses on optimization of information theoretic design objectives---in particular, from the combinatorial perspective of observation selection. Given many potential experiments, one may wish to choose a most informative subset thereof. Even if the data have in principle been collected, practical constraints on storage, communication, and computational costs may limit the number of observations that one wishes to employ. We introduce methods for selecting near-optimal subsets of the data under cardinality constraints. Our methods exploit the structure of linear inverse problems in the Bayesian setting, and can be efficiently implemented using low-rank approximations and greedy strategies based on modular bounds. This is joint work with Chi Feng and Jayanth Jagalur-Mohan.
10:00 to 10:30 Claudia Schillings (Universität Mannheim)
On the Convergence of Laplace's Approximation and Its Implications for Bayesian Computation
Sampling methods for Bayesian inference show numerical instabilities in the case of concentrated posterior distributions. However, the concentration of the posterior is a highly desirable situation in practice, since it relates to informative or large data. In this talk, we will discuss convergence results of Laplace’s approximation and analyze the use of the approximation within sampling methods. This is joint work with Bjoern Sprungk (U Goettingen) and Philipp Wacker (FAU Erlangen).
10:30 to 11:00 Morning Coffee
11:00 to 11:30 Hugo Maruri-Aguilar (Queen Mary University of London)
Smooth metamodels
Smooth supersaturated polynomials (Bates et al., 2014) have been proposed for emulating computer experiments. These models are simple to interpret and have spline-like properties inherited from the Sobolev-type of smoothing which is at the core of the method. An implementation of these models is available in the R package ssm. This talk aims to describe the method as well as discuss designs that could be performed with the help of smooth models. To illustrate the methodology, we use data from the fan blade assembly. This is joint work with H Wynn, R Bates and P Curtis.
11:30 to 12:00 Michael Goldstein (Durham University)
Inverting the Pareto Boundary: Bayes linear decision support with a soft constraint
We consider problems of decision support based around computer simulators, where we must take into account a soft constraint on our decision choices. This leads to the problem of identifying and inverting the Pareto boundary for the decision. We show how Bayes linear methods may be used for this purpose and how the sensitivity of the decision choices may be quantified and explored. The approach is illustrated with a problem on wind farm construction. This is joint work with Hailiang Du.
12:00 to 13:30 Buffet Lunch at INI
13:30 to 14:30 Andrew Stuart (University of Warwick)
Large Graph Limits of Learning Algorithms
Many problems in machine learning require the classification of high dimensional data. One methodology to approach such problems is to construct a graph whose vertices are identified with data points, with edges weighted according to some measure of affinity between the data points. Algorithms such as spectral clustering, probit classification and the Bayesian level set method can all be applied in this setting. The goal of the talk is to describe these algorithms for classification, and analyze them in the limit of large data sets. Doing so leads to interesting problems in the calculus of variations, Bayesian inverse problems and in Monte Carlo Markov Chain, all of which will be highlighted in the talk. These limiting problems give insight into the structure of the classification problem, and algorithms for it.    

Collaboration with:  
Andrea Bertozzi (UCLA)
Michael Luo (UCLA)
Kostas Zygalakis (Edinburgh)  
Matt Dunlop (Caltech)
Dejan Slepcev (CMU)
Matt Thorpe (Cambridge)
(forthcoming paper)
14:30 to 15:00 Afternoon Tea
15:00 to 16:00 Tim Sullivan (Freie Universität Berlin); (Konrad-Zuse-Zentrum für Informationstechnik Berlin)
Bayesian probabilistic numerical methods
In this work, numerical computation - such as numerical solution of a PDE - is treated as a statistical inverse problem in its own right. The popular Bayesian approach to inversion is considered, wherein a posterior distribution is induced over the object of interest by conditioning a prior distribution on the same finite information that would be used in a classical numerical method. The main technical consideration is that the data in this context are non-random and thus the standard Bayes' theorem does not hold. General conditions will be presented under which such Bayesian probabilistic numerical methods are well-posed, and a sequential Monte-Carlo method will be shown to provide consistent estimation of the posterior. The paradigm is extended to computational ``pipelines'', through which a distributional quantification of numerical error can be propagated. A sufficient condition is presented for when such propagation can be endowed with a globally coherent Bayesian interpretation, based on a novel class of probabilistic graphical models designed to represent a computational work-flow. The concepts are illustrated through explicit numerical experiments involving both linear and non-linear PDE models. This is joint work with Jon Cockayne, Chris Oates, and Mark Girolami. Further details are available in the preprint arXiv:1702.03673.
16:00 to 17:00 Poster Session
Wednesday 11th April 2018
09:00 to 10:00 Angela Dean (Ohio State University)
Experimental Design for Prediction of Physical System Means Using Calibrated Computer Simulators
Computer experiments using deterministic simulators are often used to supplement physical system experiments. A common problem is that a computer simulator may provide biased output for the physical process due to the simplified physics or biology used in the mathematical model. However, when physical observations are available, it may be possible to use these data to align the simulator output to be close to the true mean response by constructing a bias-corrected predictor (a process called calibration). This talk looks at two aspects of experimental design for prediction of physical system means using a Bayesian calibrated predictor. First, the empirical prediction accuracy over the output space of several different types of combined physical and simulator designs is discussed. In particular, designs constructed using the integrated mean squared prediction error seem to perform well. Secondly, a sequential design methodology for optimizing a physical manufacturing process when there are multiple, competing product objectives is described. The goal is to identify a set of manufacturing conditions each of which leads to outputs on the Pareto Front of the product objectives, i.e. identify manufacturing conditions which cannot be modified to improve all the product objectives simultaneously. A sequential design methodology which maximizes the posterior expected minimax fitness function is used to add data from either the simulator or the manufacturing process. The method is illustrated with an example from an injection molding study. The presentation is based on joint work with Thomas Santner, Erin Leatherman, and Po-Hsu Allen Chen.
10:00 to 10:30 Peter Challenor (University of Exeter)
Experimental Design for Inverse Modelling: From Real to Virtual and Back
Inverse modelling requires both observations in the real world as well as runs of the computer model. As our inverse modelling method we look at history matching which uses waves of computer model runs to find areas of input space where the model is implausible and thus can be ruled out. However if we reduce the uncertainty on our observations we also rule out additional space. Given the relative costs of model runs and real world observations can we find a method of deciding which is best to do next? Using an example in c radiology we examine the interplay between taking real world observations and running additional computer experiments and explore some possible strategies. .
10:30 to 11:00 Morning Coffee
11:00 to 11:30 David Ginsbourger (Other); (Universität Bern)
Quantifying and reducing uncertainties on sets under Gaussian Process priors
Gaussian Process models have been used in a number of problems where an objective function f needs to be studied based on a drastically limited number of evaluations. Global optimization algorithms based on Gaussian Process models have been investigated for several decades, and have become quite popular notably in design of computer experiments. Also, further classes of problems involving the estimation of sets implicitly defined by f, e.g. sets of excursion above a given threshold, have inspired multiple research developments. In this talk, we will give an overview of recent results and challenges pertaining to the estimation of sets under Gaussian Process priors, with a particular interest for to the quantification and the sequential reduction of associated uncertainties. Based on a series of joint works primarily with Dario Azzimonti, François Bachoc, Julien Bect, Mickaël Binois, Clément Chevalier, Ilya Molchanov, Victor Picheny, Yann Richet and Emmanuel Vazquez.
11:30 to 12:00 Daniel Williamson (University of Exeter)
Parameter inference, model error and the goals of calibration
I have some data, a mathematical model describing a process in the real world that produced that data and I would like to learn something about the real world. We would typically formulate this as an inverse problem and apply our favourite techniques for solving it (e.g. Bayesian calibration or history matching), ultimately providing inference for those parameters in our mathematical model that are consistent with the data. Does this make sense? In this talk, I will use climate science as a lens through which we can look at how mathematical models are viewed and treated by the scientific community, and consider UQ approaches to inverse problems and how they might fit and ask whether it matters if they don't.
12:00 to 13:30 Buffet Lunch at INI
19:30 to 22:00 Formal Dinner Gonville & Caius College
Thursday 12th April 2018
09:00 to 10:00 Jeremy Oakley (University of Sheffield)
Bayesian calibration, history matching and model discrepancy
Bayesian calibration and history matching are both well established tools for solving inverse problems: finding model inputs to make model outputs match observed data as closely as possible. I will discuss and compare both, within the context of decision-making. I will discuss the sometimes contentious issue of model discrepancy: how and whether we might account for an imperfect or misspecified model within the inference procedure. I will also present some work on history matching of a high dimensional individual based HIV transmission model (joint work with I. Andrianakis, N. McCreesh, I. Vernon, T. J. McKinley, R. N. Nsubuga, M. Goldstein and R. J. White).
10:00 to 10:30 Masoumeh Dashti (University of Sussex)
Modes of posterior measure for Bayesian inverse problems with a class of non-Gaussian priors
We consider the inverse problem of recovering an unknown functional parameter from noisy and indirect observations. We adopt a Bayesian approach and, for a non-smooth, non-Gaussian and sparsity-promoting class of prior measures, show that maximum a posteriori (MAP) estimates are characterized by the minimizers of a generalized Onsager-Machlup functional of the posterior. We also discuss some posterior consistency results. This is based on joint works with S. Agapiou, M.Burger and T. Helin.
10:30 to 11:00 Morning Coffee
11:00 to 11:30 Nicholas Dexter (University of Tennessee); (Oak Ridge National Laboratory)
Joint-sparse recovery for high-dimensional parametric PDEs
Co-authors: Hoang Tran (Oak Ridge National Laboratory) & Clayton Webster (University of Tennessee & Oak Ridge National Laboratory) We present and analyze a novel sparse polynomial approximation method for the solution of PDEs with stochastic and parametric inputs. Our approach treats the parameterized problem as a problem of joint-sparse signal recovery, i.e., simultaneous reconstruction of a set of sparse signals, sharing a common sparsity pattern, from a countable, possibly infinite, set of measurements. In this setting, the support set of the signal is assumed to be unknown and the measurements may be corrupted by noise. We propose the solution of a linear inverse problem via convex sparse regularization for an approximation to the true signal. Our approach allows for global approximations of the solution over both physical and parametric domains. In addition, we show that the method enjoys the minimal sample complexity requirements common to compressed sensing-based approaches. We then perform extensive numerical experiments on several high-dimensional parameterized elliptic PDE models to demonstrate the recovery properties of the proposed approach.
11:30 to 12:00 Aretha Teckentrup (University of Edinburgh)
Deep Gaussian Process Priors for Bayesian Inverse Problems
Co-authors: Matt Dunlop (Caltech), Mark Girolami (Imperial College), Andrew Stuart (Caltech)

Deep Gaussian processes have received a great deal of attention in the last couple of years, due to their ability to model very complex behaviour. In this talk, we present a general framework for constructing deep Gaussian processes, and provide a mathematical argument for why the depth of the processes is in most cases finite. We also present some numerical experiments, where deep Gaussian processes have been employed as prior distributions in Bayesian inverse problems.

Related Links
12:00 to 13:30 Buffet Lunch at INI
13:30 to 14:30 Derek Bingham (Simon Fraser University)
Bayesian model calibration for generalized linear models: An application in radiation transport
Co-author: Mike Grosskopf (Los Alamos National Lab)

Model calibration uses outputs from a simulator and field data to build a predictive model for the physical system and to estimate unknown inputs. The conventional approach to model calibration assumes that the observations are continuous outcomes. In many applications this is not the case. The methodology proposed was motivated by an application in modeling photon counts at the Center for Exascale Radiation Transport. There, high performance computing is used for simulating the flow of neutrons through various materials. In this talk, new Bayesian methodology for computer model calibration to handle the count structure of our observed data allows closer fidelity to the experimental system and provides flexibility for identifying different forms of model discrepancy between the simulator and experiment.

14:30 to 15:00 Ian Vernon (University of Durham)
Multilevel Emulation and History Matching of EAGLE: an expensive hydrodynamical Galaxy formation simulation.
We discuss strategies for performing Bayesian uncertainty analyses for extremely expensive simulators. The EAGLE model is one of the most (arguably the most) complex hydrodynamical Galaxy formation simulations yet performed. It is however extremely expensive, currently taking approximately 5 million hours of CPU time, with order of magnitude increases in runtime planned. This makes a full uncertainty analysis involving the exploration of multiple input parameters along with several additional uncertainty assessments, seemingly impossible. We present a strategy for the resolution of this problem, which incorporates four versions of the EAGLE model, of varying speed and accuracy, within a specific multilevel emulation framework that facilitates the incorporation of detailed judgements regarding the uncertain links between the physically different model versions. We show how this approach naturally fits within the iterative history matching process, whereby regions of input parameter space are identified that may lead to acceptable matches between model output and the real universe, given all major sources of uncertainty. We will briefly discuss the detailed assessment of such uncertainties as observation error and structural model discrepancies and their various components, and emphasise that without such assessments any such analysis rapidly loses meaning.
15:00 to 15:30 Afternoon Tea
15:30 to 16:00 Matthew Pratola (Ohio State University)
A Comparison of Approximate Bayesian Computation and Stochastic Calibration for Spatio-Temporal Models of High-Frequency Rainfall Patterns
Modeling complex environmental phenomena such as rainfall patterns has proven challenging due to the difficulty in capturing heavy-tailed behavior, such as extreme weather, in a meaningful way. Recently, a novel approach to this task has taken the form of so-called stochastic weather generators, which use statistical formulations to emulate the distributional patterns of an environmental process. However, while sampling from such models is usually feasible, they typically do not possess closed-form likelihood functions, rendering the usual approaches to model fitting infeasible. Furthermore, some of these stochastic weather generators are now becoming so complex that even simulating from them can be computationally expensive. We propose and compare two approaches to fitting computationally expensive stochastic weather generators motivated by Approximate Bayesian Computation and Stochastic Simulator Calibration methodologies. The methods are then demonstrated by estimating important parameters of a recent stochastic weather generator model applied to rainfall data from the continental USA.
16:00 to 17:00 Poster Session
Friday 13th April 2018
09:00 to 10:00 Luc Pronzato (Université de Nice Sophia Antipolis); (CNRS (Centre national de la recherche scientifique))
Bayesian quadrature, energy minimization and kernel herding for space filling design
A standard objective in computer experiments is to predict the behaviour of an unknown function on a compact domain from a few evaluations inside the domain. When little is known about the function, space-filling design is advisable: typically, points of evaluation spread out across the available space are obtained by minimizing a geometrical (for instance, minimax-distance) or a discrepancy criterion measuring distance to uniformity. We shall make a survey of some recent results on energy functionals, and investigate connections between design for integration (quadrature design), construction of the (continuous) BLUE for the location model, and minimization of energy (kernel discrepancy) for signed measures. Integrally strictly positive definite kernels define strictly convex energy functionals, with an equivalence between the notions of potential and directional derivative for smooth kernels, showing the strong relation between discrepancy minimization and more traditional design of optimal experiments. In particular, kernel herding algorithms are special instances of vertex-direction methods used in optimal design, and can be applied to the construction of point sequences with suitable space-filling properties. The presentation is based on recent work with A.A. Zhigljavsky (Cardiff University).
10:00 to 10:30 Serge Guillas (University College London)
Computer model calibration with large nonstationary spatial outputs: application to the calibration of a climate model
Bayesian calibration of computer models tunes unknown input parameters by comparing outputs to observations. For model outputs distributed over space, this becomes computationally expensive due to the output size. To overcome this challenge, we employ a basis representations of the model outputs and observations: we match these decompositions to efficiently carry out the calibration. In a second step, we incorporate the nonstationary behavior, in terms of spatial variations of both variance and correlations, into the calibration. We insert two INLA-SPDE parameters into the calibration. A synthetic example and a climate model illustration highlight the benefits of our approach.
10:30 to 11:00 Morning Coffee
11:00 to 12:00 Final Panel Discussion INI 1
12:00 to 13:30 Buffet Lunch at INI
University of Cambridge Research Councils UK
    Clay Mathematics Institute London Mathematical Society NM Rothschild and Sons