Surrogate models for UQ in complex systems
Monday 5th February 2018 to Friday 9th February 2018
10:50 to 11:20  Registration & Morning Coffee  
11:20 to 11:30  Welcome from David Abrahams (INI Director)  
11:30 to 12:30 
Catherine Powell (University of Manchester) Adaptive Stochastic Galerkin Finite Element Approximation for Elliptic PDEs with Random Coefficients
Coauthor: Adam Crowder (University of Manchester)

INI 1  
12:30 to 13:30  Lunch @ Churchill College  
13:30 to 14:30 
Michael Goldstein (Durham University) Emulation for model discrepancy Careful assessment of model discrepancy is a crucial aspect of uncertainty quantification. We will discuss the different ways in which emulation may be used to support such assessment, illustrating with practical examples. 
INI 1  
14:30 to 15:30 
Ralph Smith (North Carolina State University) Active Subspace Techniques to Construct Surrogate Models for Complex Physical and Biological Models For many complex physical and biological models, the computational cost of highfidelity simulation codes precludes their direct use for Bayesian model calibration and uncertainty propagation. For example, the considered neutronics and nuclear thermal hydraulics codes can take hours to days for a single run. Furthermore, the models often have tens to thousands of inputscomprised of parameters, initial conditions, or boundary conditionsmany of which are unidentifiable in the sense that they cannot be uniquely determined using measured responses. In this presentation, we will discuss techniques to isolate influential inputs for subsequent surrogate model construction for Bayesian inference and uncertainty propagation. For input selection, we will discuss advantages and shortcomings of global sensitivity analysis to isolate influential inputs and the use of active subspace construction to determine lowdimensional input manifolds. We will also discuss the manner in which Bayesian calibration on active subspaces can be used to quantify uncertainties in physical parameters. These techniques will be illustrated for models arising in nuclear power plant design, quantuminformed material characterization, and HIV modeling and treatment. 
INI 1  
15:30 to 16:00  Afternoon Tea  
16:00 to 17:00 
Christoph Schwab (ETH Zürich) Domain Uncertainty Quantification
We address the numerical analysis of domain uncertainty in UQ for partial differential and integral equations.
For small amplitude shape variation, a first order, kth moment perturbation analysis and sparse tensor discretization
produces approximate kpoint correlations at near optimal order: work and memory scale loglinearly w.r. to N,
the number of degrees of freedom for approximating one instance of the nominal (meanfield) problem [1,3].
For large domain variations, the notion of shape holomorphy of the solution is introduced.
It implies (the `usual') sparsity and dimensionindependent convergence rates of gpc approximations
(e.g., anisotropic stochastic collocation, least squares, CS, ...) of parametric domaintosolution maps in forward UQ.
This property holds for a broad class of smooth elliptic and parabolic boundary value problems.
Shape holomorphy also implies sparsity of gpc expansions of certain posteriors in Bayesian inverse UQ [7], [>WS4].
We discuss consequences of gpc sparsity on some surrogate forward models, to be used e.g. in
optimization under domain uncertainty [8,9].
We also report on dimension independent convergence rates of Smolyak and higher order QuasiMonte Carlo integration [5,6,7].
Examples include the usual (anisotropic) diffusion problems, NavierStokes [2] and time harmonic Maxwell PDEs [4],
and forward UQ for fractional PDEs.
Joint work with
Jakob Zech (ETH), Albert Cohen (Univ. P. et M. Curie), Carlos JerezHanckes (PUC, Santiago, Chile).
Work supported in part by the Swiss National Science Foundation.
References:
[1] A. Chernov and Ch. Schwab:
First order kth moment finite element analysis of nonlinear operator equations with stochastic data,
Mathematics of Computation, 82 (2013), pp. 18591888.
[2] A. Cohen and Ch. Schwab and J. Zech:
Shape Holomorphy of the stationary NavierStokes Equations, accepted (2018),
SIAM J. Math. Analysis, SAM Report 201645.
[3] H. Harbrecht and R. Schneider and Ch. Schwab:
Sparse Second Moment Analysis for Elliptic Problems in Stochastic Domains,
Numerische Mathematik, 109/3 (2008), pp. 385414.
[4] C. JerezHanckes and Ch. Schwab and J. Zech:
Electromagnetic Wave Scattering by Random Surfaces: Shape Holomorphy,
Math. Mod. Meth. Appl. Sci., 27/12 (2017), pp. 22292259.
[5] J. Dick and Q. T. Le Gia and Ch. Schwab:
Higher order Quasi Monte Carlo integration for holomorphic, parametric operator equations.
SIAM Journ. Uncertainty Quantification, 4/1 (2016), pp. 4879.
[6] J. Zech and Ch. Schwab:
Convergence rates of high dimensional Smolyak quadrature.
In review, SAM Report 201727.
[7] J. Dick and R. N. Gantner and Q. T. Le Gia and Ch. Schwab:
Multilevel higherorder quasiMonte Carlo Bayesian estimation.
Math. Mod. Meth. Appl. Sci., 27/5 (2017), pp. 953995.
[8] P. Chen and Ch. Schwab:
Sparsegrid, reducedbasis Bayesian inversion: Nonaffineparametric nonlinear equations.
Journal of Computational Physics, 316 (2016), pp. 470503.
[9] Ch. Schwab and J. Zech:
Deep Learning in High Dimension. In review, SAM Report 201757.

INI 1  
17:00 to 18:00  Welcome Wine Reception at INI 
09:00 to 10:00 
Hoang Tran (Oak Ridge National Laboratory) Recovery conditions of compressed sensing approach to uncertainty quantification
Coauthor: Clayton Webster (UTK/ORNL).
This talk is concerned with the compressed sensing approach to reconstruction of highdimensional functions from limited amount of data. In this approach, the uniform bounds of the underlying global polynomial bases have often been relied on for the complexity analysis and algorithm development. We prove a new, improved recovery condition without using this uniform boundedness assumption, applicable to multidimensional Legendre approximations. Specifically, our sample complexity is established using the unbounded envelope of all polynomials, thus independent of polynomial subspaces. Some consequent, simple criteria for choosing good random sample sets will also be discussed.
In the second part, I will discuss the recovery guarantees of nonconvex optimizations. These minimizations are generally closer to l_0 penalty than l_1 norm, thus it is widely accepted (also demonstrated computationally in UQ) that they are able to enhance the sparsity and accuracy of the approximations. However, the theory proving that nonconvex penalties are as good as or better than l1 minimization in sparse reconstruction has not been available beyond a few specific cases. We aim to fill this gap by establishing new recovery guarantees through unified null space properties that encompass most of the currently proposed nonconvex functionals in the literature, verifying that they are truly superior to l_1.

INI 1  
10:00 to 11:00 
Maurizio Filippone (EURECOM) Random Feature Expansions for Deep Gaussian Processes
Drawing meaningful conclusions on the way complex real life phenomena work and being able to predict the behavior of systems of interest require developing accurate and highly interpretable mathematical models whose parameters need to be estimated from observations. In modern applications, however, we are often challenged with the lack of such models, and even when these are available they are too computational demanding to be suitable for standard parameter optimization/inference methods. While probabilistic models based on Deep Gaussian Processes (DGPs) offer attractive tools to tackle these challenges in a principled way and to allow for a sound quantification of uncertainty, carrying out inference for these models poses huge computational challenges that arguably hinder their wide adoption. In this talk, I will present our contribution to the development of practical and scalable inference for DGPs, which can exploit distributed and GPU computing. In particular, I will introduce a formulation of DGPs based on random features that we infer using stochastic variational inference. Through a series of experiments, I will illustrate how our proposal enables scalable deep probabilistic nonparametric modeling and significantly advances the stateoftheart on inference methods for DGPs.

INI 1  
11:00 to 11:30  Morning Coffee  
11:30 to 12:30 
Lorenzo Tamellini (Università degli Studi di Pavia) MultiIndex Stochastic Collocation (MISC) for Elliptic PDEs with random data
Coauthors: Joakim Beck (KAUST), AbdulLateef HajiAli (Oxford University), Fabio Nobile (EPFL), Raul Tempone (KAUST)

INI 1  
12:30 to 13:30  Lunch @ Churchill College  
13:30 to 14:30 
John Paul Gosling (University of Leeds) Modelling discontinuities in simulator output using Voronoi tessellations
Coauthors: Chris Pope (University of Leeds), Jill Johnson (University of Leeds), Stuart Barber (University of Leeds), Paul Blackwell (University of Sheffield)

INI 1  
14:30 to 15:30 
Guannan Zhang (Oak Ridge National Laboratory) A domaindecompositionbased model reduction method for convectiondiffusion equations with random coefficients
We focuses on linear steadystate convectiondiffusion equations with randomfield coefficients. Our particular interest to this effort are two types of partial differential equations (PDEs), i.e., diffusion equations with random diffusivities, and convectiondominated transport equations with random velocity fields. For each of them, we investigate two types of random fields, i.e., the colored noise and the discrete white noise. We developed a new domaindecompositionbased model reduction (DDMR) method, which can exploit the lowdimensional structure of local solutions from various perspectives. We divide the physical domain into a set of nonoverlapping subdomains, generate local random fields and establish the correlation structure among local fields. We generate a set of reduced bases for the PDE solution within subdomains and on interfaces, then define reduced local stiffness matrices by multiplying each reduced basis by the corresponding blocks of the local stiffness matrix. After that, we establish sparse approximations of the entries of the reduced local stiffness matrices in lowdimensional subspaces, which finishes the offline procedure. In the online phase, when a new realization of the global random field is generated, we map the global random variables to local random variables, evaluate the sparse approximations of the reduced local stiffness matrices, assemble the reduced global Schur complement matrix and solve the coefficients of the reduced bases on interfaces, and then assemble the reduced local Schur complement matrices and solve the coefficients of the reduced bases in the interior of the subdomains. The advantages and contributions of our method lie in the following three aspects. First, the DDMR method has the onlineoffline decomposition feature, i.e., the online computational cost is independent of the finite element mesh size. Second, the DDMR method can handle the PDEs of interest with nonaffine highdimensional random coefficients. The challenge caused by the nonaffine coefficients is resolved by approximating the entries of the reduced stiffness matrices. The highdimensionality is handled by the DD strategy. Third, the DDMR method can avoid building polynomial sparse approximations to local PDE solutions. This feature is useful in solving the convectiondominated PDE, whose solution has a sharp transition caused by the boundary condition. We demonstrate the performance of our method based on the diffusion equation and convectiondominated equation with colored noises and discrete white noises.

INI 1  
15:30 to 16:00  Afternoon Tea  
16:00 to 17:00  Poster Session 
09:00 to 10:00 
Martin Eigel (WeierstraßInstitut für Angewandte Analysis und Stochastik) Aspects of adaptive Galerkin FE for stochastic direct and inverse problems
Coauthors: Max Pfeffer (MPI MIS Leipzig), Manuel Marschall (WIAS Berlin), Reinhold Schneider (TU Berlin) The Stochastic Galerkin Finite Element Method (SGFEM) is a common approach to numerically solve random PDEs with the aim to obtain a functional representation of the stochastic solution. As with any spectral method, the curse of dimensionality renders the approach challenging when the randomness depends on a large or countable infinite set of parameters. This makes function space adaptation and model reduction strategies a necessity. We review adaptive SGFEM based on reliable a posteriori error estimators for affine and nonaffine parametric representations. Based on this, an adaptive explicit samplingfree Bayesian inversion in hierarchical tensor formats can be derived. As an outlook onto current research, a statistical learning viewpoint is presented, which connects concepts of UQ and machine learning from a Variational Monte Carlo perspective. 
INI 1  
10:00 to 11:00 
Elaine Spiller (None) Emulators for forecasting and UQ of natural hazards Geophysical hazards – landslides, tsunamis, volcanic avalanches, etc. – which lead to catastrophic inundation are rare yet devastating events for surrounding communities. The rarity of these events poses two significant challenges. First, there are limited data to inform aleatoric scenario models, how frequent, how big, where. Second, such hazards often follow heavytailed distributions resulting in a significant probability that a largerthanrecorded catastrophe might occur. To overcome this second challenge, we must rely on physical models of these hazards to “probe” the tail for these catastrophic events. We will present an emulatorbased strategy that allows great speedup in Monte Carlo simulations for creating probabilistic hazard forecast maps. This approach offers the flexibility to explore both the impacts of epistemic uncertainties on hazard forecasts and of nonstationary scenario modeling on short term forecasts. Collaborators: Jim Berger (Duke), Eliza Calder (Edinburgh), Abani Patra (Buffalo), Bruce Pitman (Buffalo), Regis Rutarindwa (Marquette), Robert Wolpert (Duke) 
INI 1  
11:00 to 11:30  Morning Coffee  
11:30 to 12:30  Panel comparisons: Challenor, Ginsbourger, Nobile, Teckentrup and Beck  INI 1  
12:30 to 13:30  Lunch @ Churchill College  
13:30 to 17:00  Free Afternoon  
19:30 to 22:00  Formal Dinner at Trinity College 
09:00 to 10:00 
Ben Adcock (Simon Fraser University) Polynomial approximation of highdimensional functions on irregular domains
Coauthor: Daan Huybrechs (KU Leuven) Smooth, multivariate functions defined on tensor domains can be approximated using orthonormal bases formed as tensor products of onedimensional orthogonal polynomials. On the other hand, constructing orthogonal polynomials in irregular domains is difficult and computationally intensive. Yet irregular domains arise in many applications, including uncertainty quantification, modelorder reduction, optimal control and numerical PDEs. In this talk I will introduce a framework for approximating smooth, multivariate functions on irregular domains, known as polynomial frame approximation. Importantly, this approach corresponds to approximation in a frame, rather than a basis; a fact which leads to several key differences, both theoretical and numerical in nature. However, this approach requires no orthogonalization or parametrization of the domain boundary, thus making it suitable for very general domains, including a priori unknown domains. I will discuss theoretical result s for the approximation error, stability and sample complexity of this approach, and show its suitability for highdimensional approximation through independence (or weak dependence) of the guarantees on the ambient dimension d. I will also present several numerical results, and highlight some open problems and challenges. 
INI 1  
10:00 to 11:00 
Christine Shoemaker (National University of Singapore) Deterministic RBF Surrogate Methods for Uncertainty Quantification, Global Optimization and Parallel HPC Applications
Coauthor: Antoine Espinet (Cornell University)

INI 1  
11:00 to 11:30  Morning Coffee  
11:30 to 12:30 
Aretha Teckentrup (University of Edinburgh) Surrogate models in Bayesian Inverse Problems
Coauthors: Andrew Stuart (Caltech) , Han Cheng Lie and Timm Sullivan (Free University Berlin)

INI 1  
12:30 to 13:30  Lunch @ Churchill College  
13:30 to 14:30 
David Ginsbourger (Other); (Universität Bern) Positive definite kernels for deterministic and stochastic approximations of (invariant) functions 
INI 1  
14:30 to 15:30 
Raul Fidel Tempone (King Abdullah University of Science and Technology (KAUST)) Uncertainty Quantification with MultiLevel and MultiIndex methods
We start by recalling the Monte Carlo and Multilevel Monte Carlo (MLMC) methods for computing statistics of the solution of a Partial Differential Equation with random data. Then, we present the MultiIndex Monte Carlo (MIMC) and MultiIndex Stochastic Collocation (MISC) methods. MIMC is both a stochastic version of the combination technique introduced by Zenger, Griebel and collaborators and an extension of the MLMC method first described by Heinrich and Giles. Instead of using firstorder differences as in MLMC, MIMC uses mixed differences to reduce the variance of the hierarchical differences dramatically, thus yielding improved convergence rates. MISC is a deterministic combination technique that also uses mixed differences to achieve better complexity than MIMC, provided enough regularity. During the presentation, we will showcase the behavior of the numerical methods in applications, some of them arising in the context of Regression based Surrogates and Optimal Experimental Design.
Coauthors: J. Beck, L. Espath (KAUST), A.L. HajiAli (Oxford), Q. Long (UT), F. Nobile (EPFL),
M. Scavino (UdelaR), L. Tamellini (IMATI), S. Wolfers (KAUST)
Webpages:
https://stochastic_numerics.kaust.edu.sa
https://sriuq.kaust.edu.sa

INI 1  
15:30 to 16:00  Afternoon Tea  
16:00 to 17:00 
Maria Adamou (University of Southampton) Bayesian optimal design for Gaussian process model
Coauthor: Dave Woods (University of Southampton)

INI 1 
09:00 to 10:00 
Olivier Roustant (Mines SaintÉtienne) Group covariance functions for Gaussian process metamodels with categorical inputs Coauthors : E. Padonou (Mines SaintEtienne), Y. Deville (AlpeStat), A. Clément (CEA), G. Perrin (CEA), J. Giorla (CEA) and H. Wynn (LSE). 
INI 1  
10:00 to 11:00 
Daniel Williamson (University of Exeter) Nonstationary Gaussian process emulators with covariance mixtures
Routine diagnostic checking of stationary Gaussian processes fitted to the output of complex computer codes often reveals nonstationary behaviour. There have been a number of approaches, both traditional and more recent, to modelling or accounting for this nonstationarity via the fitted process. These have included the fitting of complex mean functions to attempt to leave a stationary residual process (an idea that is often very difficult to get right in practice), using regression trees or other techniques to partition the input space into regions where different stationary processes are fitted (leading to arbitrary discontinuities in the modelling of the overall process), and other approaches which can be considered to live in one of these camps. In this work we allow the fitted process to be continuous by modelling the covariance kernel as a finite mixture of stationary covariance kernels and allowing the mixture weights to vary appropriately across parameter space. We introduce our method and compare its performance with the leading approaches in the literature for a variety of standard test functions and the cloud parameterisation of the French climate model. This is work led by my finalyear PhD student, Victoria Volodina.

INI 1  
11:00 to 11:30  Morning Coffee  
11:30 to 12:30 
Oliver Ernst (Technische Universität Chemnitz) HighDimensional Collocation for Lognormal Diffusion Problems
Coauthors: Björn Sprungk (Universität Mannheim), Lorenzo Tamellini (IMATICNR Pavia)
Many UQ models consist of random differential equations in which one or more data components are uncertain and modeled as random variables. When the latter take values in a separable function space, their representation typically requires a large or countably infinite number of random coordinates. Numerical approximation methods for such functions of an infinite number of parameters based on best Nterm approximation have recently been proposed and shown to converge at an algebraic rate. Collocation methods have a number of computational advantages over best Nterm approximation, and we show how ideas successful there can be used to show a similar convergence rate for sparse collocation of Hilbertspacevalued functions depending on countably many Gaussian random variables.
Such functions appear as solutions of elliptic PDEs with a lognormal diffusion coefficient. We outline a general L2convergence theory based on previous work by Bachmayr et al. and Chen and establish an algebraic convergence rate for sufficiently smooth functions assuming a mild growth bound for the univariate hierarchical surpluses of the interpolation scheme applied to Hermite polynomials. We verify specifically for GaussHermite nodes that this assumption holds and also show algebraic convergence with respect to the resulting number of sparse grid points for this case. Numerical experiments illustrate the dimensionindependent convergence rate.

INI 1  
12:30 to 13:30  Lunch @ Churchill College  
13:30 to 14:30 
Robert Gramacy (Virginia Polytechnic Institute and State University) Replication or exploration? Sequential design for stochastic simulation experiments We investigate the merits of replication, and provide methods that search for optimal designs (including replicates), in the context of noisy computer simulation experiments. We first show that replication offers the potential to be beneficial from both design and computational perspectives, in the context of Gaussian process surrogate modeling. We then develop a lookahead based sequential design scheme that can determine if a new run should be at an existing input location (i.e., replicate) or at a new one (explore). When paired with a newly developed heteroskedastic Gaussian process model, our dynamic design scheme facilitates learning of signal and noise relationships which can vary throughout the input space. We show that it does so efficiently, on both computational and statistical grounds. In addition to illustrative synthetic examples, we demonstrate performance on two challenging realdata simulation experiments, from inventory management and epidemiology. 
INI 1  
14:30 to 15:30  Future directions panel  INI 1 