Stability analysis and optimal control of large-scale stochastic systems

In the past years, large-scale stochastic networks have been an intense subject of study due to their use in modelling a variety of systems including telecommunications, service and data centers, patient flows, etc. The optimal control of such systems has found numerous applications such as, but not limited to, finance and cognitive neuroscience. This thesis focuses on the stability analysis and optimal control of stochastic systems. In particular, we study: (1) the ergodic properties of multiclass multi-pool networks in the Halfin-Whitt regime; and (2) the optimal control of stochastic networks assuming a structural property relating the running cost to the solution of the Hamilton-Jacobi-Bellman (HJB) equation. In the first part of this thesis, we introduce a "system-wide safety staffing" (SWSS) parameter for multiclass multi-pool networks in the Halfin-Whitt regime which have any tree topology. This parameter can be regarded as the optimal reallocation of the capacity fluctuations (positive or negative) when each server pool employs a square-root staffing rule. First, we provide an explicit form of the SWSS as a function of the system parameters, which is derived using a graph theoretic approach based on Gaussian elimination. In addition, we give an equivalent characterization of the SWSS parameter via the drift parameters of the limiting diffusion. Then, we show that if the SWSS parameter is negative, the limiting diffusion and the diffusion-scaled queueing processes are transient under any Markov control, and cannot have a stationary distribution when this parameter is zero. If it is positive, we show that the diffusion-scaled queueing processes are stabilizable, that is, there exists a scheduling policy under which the stationary distributions of the controlled processes are tight over the size of the network. Finally, we show that there exists a control under which the limiting controlled diffusion is exponentially ergodic. Thus, we have identified a necessary and sufficient condition for the stabilizability of such networks in the Halfin-Whitt regime. In the second part of this thesis, we examine two problems related to the general topic of optimal control of stochastic systems. In the first problem, we consider a linear system with Gaussian noise observed by multiple sensors which transmit measurements over a dynamic lossy network. We assume that the system is stabilizable, that is, there exists a control such that all states variables are bounded during system's behavior. First, we characterize the stationary optimal sensor scheduling policy for the finite horizon, discounted, and long-term average cost problems. Then, we show that there exists a structural property relating the running cost to the value function which is the solution of the average cost problem. In addition, we show that the value iteration algorithm converges to this solution. Further, we show that the suboptimal policies provided by the rolling horizon truncation of the value iteration also guarantee stability and provide near-optimal average cost. Lastly, we provide qualitative characterizations of the multidimensional set of measurement loss rates for which the system is stabilizable for a static network, thus extending earlier results on intermittent observations. In the second problem and motivated by the results from the previous problem, a multiplicative relative value iteration algorithm (RVI) for infinite-horizon risk-sensitive control of controlled diffusions in [doublestruck R][superscript d] is studied. We assume that the running cost is near-monotone and that it is related to the solution of the multiplicative HJB equation through a structural assumption. We show that this structural assumption implies the existence of a control under which the ground state diffusion is exponentially ergodic. In addition, we show that the multiplicative RVI algorithm converges globally to the solution of the multiplicative dynamic programming equation starting from any positive initial condition; thus extending upon the results in the literature.Electrical and Computer Engineerin