Decentralized Control of Stochastic Dynamic Systems with Applications to Resource Allocation and Portfolio Management

Modern engineering and social systems are often too complex to be managed by a centralized agent. Instead, such systems are commonly structured with multiple decentralized agents each responsible for managing a subset of the system, but the resulting system performance depends on the aggregate of the decisions made by decentralized agents. Local agents' decision makings often exhibit selfish behavior as they seek to optimize their own objectives under their localized models, which if left uncoordinated can lead to substantial loss of efficiency compared with the system that can be optimized by a single (hypothetical) centralized agent. In this dissertation, we seek to study the fundamental issues of how to efficiently manage large-scale and multi-agent stochastic dynamic systems, especially on how to device efficient coordination mechanisms that would optimize system performance under various constraints that are unique to decentralized systems.In the first part of this dissertation we study decentralized control of a general class of stochastic dynamic resource allocation problems that have many applications. We consider a stochastic system in which multiple decentralized agents allocate shared system resources in response to customer requests that arrive stochastically over time. Each agent is responsible for a subset of the allocation decisions which it makes according to a dynamic allocation policy obtained by maximizing his own expected profit subject to a potentially mis-specified model of the way in which shared resources are consumed by other agents. We introduce the notion of a transfer contract which specifies how agents compensate one another whenever resources are consumed and establish the existence of contracts under which the decentralized system has no efficiency loss relative to centralized optimality. We also show that this property is insensitive to mis-specification by each agent of the dynamics of resource consumption by others in the system. An explicit characterization of the optimal transfer contract and an iterative decentralized algorithm for computing it is also provided. In the language of duality, contracts are analogous to shadow prices and the iterative algorithm has the favor of a dual update method, but strong duality and convergence of the iterative algorithm to the set of optimal contracts are guaranteed without assumptions of convexity.In the second part of this dissertation we study a class of related decentralized control problems but specialize to portfolio and risk management. Many financial institutions typically trade in multiple correlated markets. While centralized portfolio optimization over all trading decisions is ideal, it is generally not possible due to the complexity of each market, and firms typically adopt a decentralized setup in which trading in each market the responsibility of a particular desk. Decentralized portfolio optimization, however, is complicated by the fact that different agents are commonly only well informed about their own investment universe (proprietary research and forecasts, etc) and prefer to keep this private, and have their own incentives which they optimize on the basis of their limited models. It is well known, however, that the aggregate performance of such a system can be extremely inefficient due to the loss of diversification. In this dissertation, we formulate a multi-agent dynamic portfolio choice problem and study how to improve its efficiency. We show that an internal system of swap contracts, which define internal cash transfers between agents, can be used to facilitate risk sharing and induce agents to choose portfolios that as a collection are optimal for the firm. Conceptually using swap contracts is similar to performance benchmarking that is often employed in the finance literature for decentralized portfolio management, but our new approach offers a significant advantage in that the swap contracts can be constructed in decentralized manner without requiring an all-knowing central agent. We provide an explicit characterization of the optimal swap contracts and an iterative algorithm for computing them that can be implemented without compromising proprietary agent level data.Throughout this dissertation, we also discuss various important issues surrounding decentralized control of stochastic dynamic systems, including but not limited to approximation methods, performance attribution, sensitivity analysis, and fairness issues, etc