The Annealed Leap-Point MCMC Sampler (ALPS) for multi-modal posterior distributions

Abstract

Multi-modal distributions pose major challenges for the usual algorithms that are employed in statistical inference. These problems are exacerbated in high-dimensional settings, where techniques such as Markov Chain Monte Carlo (MCMC) and Expectation Maximisation (EM) typically rely upon localised update mechanisms: such localised algorithms can effectively become trapped in one of the local modes, leading to biased inference and underestimation of uncertainty.

In this talk, I will introduce the Annealed Leap-Point Sampler (ALPS), an MCMC algorithm that augments the state space of the target distribution with a sequence of modified, annealed (cooled) distributions. The temperature of the coldest state is chosen such that the corresponding annealed target density of each individual mode can be closely fitted by a Laplace approximation. As a result, independent MCMC proposals based on a mixture of Gaussians can jump between modes even in high-dimensional problems. The ability of this method to “mode hop” at the super-cold state is then filtered through to the target state by swapping information between neighbours, in a similar manner to parallel tempering. ALPS also incorporates the best aspects of current gold-standard approaches to multi-modal sampling in high-dimensional contexts.

We have implemented ALPS as an open-source R package. Our method is demonstrated using examples of multi-modal distributions that arise in econometrics and chemistry. These applications include a seemingly-unrelated regression (SUR) model of longitudinal data from U.S. manufacturing firms, as well as a model of line shape and broadening for curve fitting in spectroscopy.

This is joint work with Nick Tawn and Gareth Roberts at the University of Warwick:

Tawn, Moores & Roberts (2021) arXiv preprint https://arxiv.org/abs/2112.12908