Conservative Hamiltonian Monte Carlo

We introduce a new class of Hamiltonian Monte Carlo (HMC) algorithm called Conservative Hamiltonian Monte Carlo (CHMC), where energy-preserving integrators, derived from the Discrete Multiplier Method, are used instead of symplectic integrators. Due to the volume being no longer preserved under such a proposal map, a correction involving the determinant of the Jacobian of the proposal map is introduced within the acceptance probability of HMC. For a $p$-th order accurate energy-preserving integrator using a time step size $\tau$, we show that CHMC satisfies stationarity without detailed balance. Moreover, we show that CHMC satisfies approximate stationarity with an error of $\mathcal{O}(\tau^{(m+1)p})$ if the determinant of the Jacobian is truncated to its first $m+1$ terms of its Taylor polynomial in $\tau^p$. We also establish a lower bound on the acceptance probability of CHMC which depends only on the desired tolerance $\delta$ for the energy error and approximate determinant. In particular, a cost-effective and gradient-free version of CHMC is obtained by approximating the determinant of the Jacobian as unity, leading to an $\mathcal{O}(\tau^p)$ error to the stationary distribution and a lower bound on the acceptance probability depending only on $\delta$. Furthermore, numerical experiments show increased performance in acceptance probability and convergence to the stationary distribution for the Gradient-free CHMC over HMC in high dimensional problems.Comment: 24 pages, 18 figure