Reinforcement learning in the continuous double auction and the trading agents competition

In this thesis we investigate if reinforcement learning (RL) techniques can be successfully used to build automated trading agents.We present two case studies in which we develop RL agents for participating in auctions. The first case study focuses on the continuous double auction (CDA), a market mechanism used in many electronic trading venues. There are currently several automated bidding strategies for participating in the CDA, geared both toward personal profit and toward increasing the efficiency of the entire market. We use model-free reinforcement learning to construct a bidding strategy for the CDA and empirically evaluates its performance against other well-known automated strategies. The second case study deals with the larger but related problem of interdependent electronic auctions. We describe an RL agent for the trading agent competition (TAC), and analyze its performance. This competition features multiple dependent auctions, and hence provides a much harder test-bed. The empirical results did not show the same success for the RL strategy as in the CDA environment. We attribute this problem to the difficulty of dealing with dependent auctions, in which the optimal strategy in one auction depends on the state of the other auctions as well