State Space Emulation and Annealed Sequential Monte Carlo for High Dimensional Optimization

Abstract

Many high dimensional optimization problems can be reformulated into a problem of finding theoptimal state path under an equivalent state space model setting. In this article, we present a general emulation strategy for developing a state space model whose likelihood function (or posterior distribution) shares the same general landscape as the objective function of the original optimization problem. Then the solution of the optimization problem is the same as the optimal state path that maximizes the likelihood function or the posterior distribution under the emulated system. To find such an optimal path, we adapt a simulated annealing approach by inserting a temperature control into the emulated dynamic system and propose a novel annealed Sequential Monte Carlo (SMC) method that effectively generating Monte Carlo sample paths utilizing samples obtained on the high temperature scale. Compared to the vanilla simulated annealing implementation, annealed SMC is an iterative algorithm for state space model optimization that directly generates state paths from the equilibrium distributions with a decreasing sequence of temperatures through sequential importance sampling which does not require burn-in or mixing iterations to ensure quasi-equilibrium condition. Several applications of state space model emulation and the corresponding annealed SMC results are demonstrated.