Simulation-based Methods for Stochastic Control and Global Optimization

Ideas of stochastic control have found applications in a variety of areas. A subclass of the problems with parameterized policies (including some stochastic impulse control problems) has received significant attention recently because of emerging applications in the areas of engineering, management, and mathematical finance. However, explicit solutions for this type of stochastic control problems only exist for some special cases, and effective numerical methods are relatively rare. Deriving efficient stochastic derivative estimators for payoff functions with discontinuities arising in many problems of practical interest is very challenging. Global optimization problems are extremely hard to solve due to the typical multimodal properties of objective functions. With the increasing availability of computing power and memory, there is a rapid development in the merging of simulation and optimization techniques. Developing new and efficient simulation-based optimization algorithms for solving stochastic control and global optimization problems is the primary goal of this thesis. First we develop a new simulation-based optimization algorithm to solve a stochastic control problem with a parameterized policy that arises in the setting of dynamic pricing and inventory control. We consider a joint dynamic pricing and inventory control problem with continuous stochastic demand and model the problem as a stochastic control problem. An explicit solution is given when a special demand model is considered. For general demand models with a parameterized policy, we develop a new simulation-based method to solve this stochastic control problem. We prove the convergence of the algorithm and show the effectiveness of the algorithm by numerical experiments. In the second part of this thesis, we focus on the problem of estimating the derivatives for a class of discontinuous payoff functions, for which existing methods are either not valid or not efficient. We derive a new unbiased stochastic derivative estimator for performance functions containing indicator functions. One important feature of this new estimator is that it can be computed from a single sample path or simulation, whereas existing estimators in the literature require additional simulations. Finally we propose a new framework for solving global optimization problems by establishing a connection with evolutionary games, and show that a particular equilibrium set of the evolutionary game is asymptotically stable. Based on this connection, we propose a Model-based Evolutionary Optimization (MEO) algorithm, which uses probabilistic models to generate new candidate solutions and uses dynamics from evolutionary game theory to govern the evolution of the probabilistic models. MEO gives new insight into the mechanism of model updating in model-based global optimization algorithms from the perspective of evolutionary game theory. Furthermore, it opens the door to developing new algorithms by using various learning algorithms and analysis techniques from evolutionary game theory