Stochastic singular control problems with state constraint

Singular control is an important and challenging class of problems in stochastic control theory. Such control problems can rarely be solved explicitly and thus numerical approximation schemes are necessary. In this work we develop approximation schemes for singular control problems with state constraints. The first problem we consider arises in problems of optimal consumption and investment under transaction costs. We use Markov chain approximations to develop a convergent numerical scheme. Proof of convergence uses techniques from the theory of weak convergence. Specific features that make the analysis nontrivial include unboundedness of state and control spaces and cost function; degeneracies in the dynamics; and presence of both singular and absolutely continuous controls. We present a computational algorithm and the results of a numerical study. Numerical schemes for singular control problems can be computationally quite intensive, and thus it is of great interest to develop less expensive schemes that exploit specific features of the underlying dynamics. To this end we investigate connections between singular control and optimal stopping problems. A key technical step in establishing such connections is proving existence of an optimal singular control. We prove such a result for a general class of singular control problems with linear dynamics and state constrained to be in a polyhedral cone. A particular example of this class of models are the so-called Brownian control problems (BCPs) and thus existence of optimal controls for BCPs follows as a consequence. Armed with this existence result, we consider a two-dimensional singular control problem that arises from queueing networks. We prove rigorously an equivalence of this iii problem with an optimal stopping problem. We exploit this connection in developing simple computational schemes for the singular control problem, and we investigate performance of the schemes in a numerical study