Self-Practice Imitation Learning from Weak Policy

Abstract. Imitation learning is an effective strategy to reinforcement learning, which avoids the delayed reward problem by learning from mentor-demonstrated trajectories. A limitation for imitation learning is that collecting sufficient qualified demonstrations is quite expensive. In this work, we study how an agent can automatically improve its perfor-mance from a weak policy, by automatically acquiring more demonstra-tions for learning. We propose the LEWE framework to sample tasks for the weak policy to execute, and then learn from the successful trajecto-ries to achieve an improvement. As the sampling strategy is the key to the efficiency of LEWE, we further propose to incorporate active learning for the sampling strategy for LEWE. Experiments in a spatial positioning task show that LEWE with active learning can effectively and efficiently improve the weak policy and achieves a better performance than the comparing sampling approaches