Hierarchical Reinforcement Learning

A hierarchical representation of the input-output transition function in a learning system is suggested. The choice of either representing the knowledge in a learning system as a discrete set of input-output pairs or as a continuous input-output transition function is discussed. The conclusion that both representations could be efficient, but at different levels is made. The difference between strategies and actions is defined. An algorithm for using adaptive critic methods in a two-level reinforcement learning system is presented. Two problems that are faced, the hierarchical credit assignment problem and the equalized state problem are described. Simulations of a one dimensional hierarchical reinforcement learning system is presented. 1 Introduction 1.1 Reinforcement Learning Learning systems can be classified according to how the system is trained. Often the two classes unsupervised and supervised learning are suggested. Sometimes reinforcement learning is considered as a separat..