TY - JOUR

T1 - Multi-index Sequential Monte Carlo Ratio Estimators for Bayesian Inverse problems

AU - Jasra, Ajay

AU - Law, Kody J.H.

AU - Walton, Neil

AU - Yang, Shangda

N1 - KAUST Repository Item: Exported on 2023-05-22
Acknowledgements: AJ was supported by KAUST baseline funding.

PY - 2023/5/8

Y1 - 2023/5/8

N2 - We consider the problem of estimating expectations with respect to a target distribution with an unknown normalising constant, and where even the un-normalised target needs to be approximated at finite resolution. This setting is ubiquitous across science and engineering applications, for example in the context of Bayesian inference where a physics-based model governed by an intractable partial differential equation (PDE) appears in the likelihood. A multi-index sequential Monte Carlo (MISMC) method is used to construct ratio estimators which provably enjoy the complexity improvements of multi-index Monte Carlo (MIMC) as well as the efficiency of sequential Monte Carlo (SMC) for inference. In particular, the proposed method provably achieves the canonical complexity of MSE - 1 , while single-level methods require MSE -ξ for ξ> 1 . This is illustrated on examples of Bayesian inverse problems with an elliptic PDE forward model in 1 and 2 spatial dimensions, where ξ= 5 / 4 and ξ= 3 / 2 , respectively. It is also illustrated on more challenging log-Gaussian process models, where single-level complexity is approximately ξ= 9 / 4 and multilevel Monte Carlo (or MIMC with an inappropriate index set) gives ξ= 5 / 4 + ω , for any ω> 0 , whereas our method is again canonical. We also provide novel theoretical verification of the product-form convergence results which MIMC requires for Gaussian processes built in spaces of mixed regularity defined in the spectral domain, which facilitates acceleration with fast Fourier transform methods via a cumulant embedding strategy, and may be of independent interest in the context of spatial statistics and machine learning.

AB - We consider the problem of estimating expectations with respect to a target distribution with an unknown normalising constant, and where even the un-normalised target needs to be approximated at finite resolution. This setting is ubiquitous across science and engineering applications, for example in the context of Bayesian inference where a physics-based model governed by an intractable partial differential equation (PDE) appears in the likelihood. A multi-index sequential Monte Carlo (MISMC) method is used to construct ratio estimators which provably enjoy the complexity improvements of multi-index Monte Carlo (MIMC) as well as the efficiency of sequential Monte Carlo (SMC) for inference. In particular, the proposed method provably achieves the canonical complexity of MSE - 1 , while single-level methods require MSE -ξ for ξ> 1 . This is illustrated on examples of Bayesian inverse problems with an elliptic PDE forward model in 1 and 2 spatial dimensions, where ξ= 5 / 4 and ξ= 3 / 2 , respectively. It is also illustrated on more challenging log-Gaussian process models, where single-level complexity is approximately ξ= 9 / 4 and multilevel Monte Carlo (or MIMC with an inappropriate index set) gives ξ= 5 / 4 + ω , for any ω> 0 , whereas our method is again canonical. We also provide novel theoretical verification of the product-form convergence results which MIMC requires for Gaussian processes built in spaces of mixed regularity defined in the spectral domain, which facilitates acceleration with fast Fourier transform methods via a cumulant embedding strategy, and may be of independent interest in the context of spatial statistics and machine learning.

UR - http://hdl.handle.net/10754/677866

UR - https://link.springer.com/10.1007/s10208-023-09612-z

UR - http://www.scopus.com/inward/record.url?scp=85158107572&partnerID=8YFLogxK

U2 - 10.1007/s10208-023-09612-z

DO - 10.1007/s10208-023-09612-z

M3 - Article

SN - 1615-3383

JO - Foundations of Computational Mathematics

JF - Foundations of Computational Mathematics

ER -