Applicability of reinforcement learning

We describe our experiences in trying to imple-ment a hierarchical reinforcement learning system, and follow with conclusions that we have drawn from the difficulties that we encountered. We present our objectives before we started, the prob-lems we encountered along the way, the solutions we devised for some of these problems, and our conclusions afterward about the class of problems for which reinforcement learning may be suitable. Our experience has made it clearer to us when and when not to select a reinforcement learning method