Reinforcement Learning Approach for Solving the Optimal Stopping Problem

The optimal stopping problem is concerned with finding an optimal policy to stop a stochastic process to maximize the expected return. This problem is critical in stochastic control and can be found in many different fields, such as operations research and finance. In this thesis, we model the optimal stopping problem as a Markov decision process and propose to solve it via a model-free value-based reinforcement learning approach, named delta-V. We consider two types of optimal stopping problems: the standard optimal stopping and the regenerative optimal stopping, which differ in their transition dynamics once the stopping action is executed. Then, we incorporate the unique structural properties of the optimal stopping problem into our algorithm design. We test our proposed approach in three cases and compare with three existing reinforcement learning algorithms. The results show that our delta-V method is able to outperform benchmark algorithms in all three cases.M.A.S