Learning to control dynamic systems with automated quantization

Learning to control dynamic systems with unknown models is a challenging research problem. However, most previous work that learns qualitative control rules does not construct qualitative states; a proper partition of continuous-state variables has to be designed by human users and given to the learning programs. We design a new learning method that learns appropriate qualitative state representation and the control rules simultaneously. Our method can aggressively partition the continuous-state variables into finer, discrete ranges until control rules based on these ranges are learned. As a case study, we apply our method to the benchmark control problem of cart-pole balancing (also known as the inverted pendulum). Experimental results show that our method not only derives different partitions for the cart-pole systems with different parameters but also learns to control the systems for an extended period of time from random initial positions. Key Words: adaptive control; reinforcement learning; automatic quantization