Noisy reinforcements in Reinforcement Learning: some case studies based on Gridworlds

Abstract: Reinforcement Learning (RL) has as its main objective to maximize the rewards of an objective func-tion. This is achieved by an agent which carries out a series of actions to modify the state of the environment. The reinforcements are the cornerstone of the RL. In this work, a modification of the classic scheme of RL is proposed. Our proposal is based on applying a reinforcement with uncertainty; namely, it adds a random signal to the reinforcement. This uncertainty makes the agent incapable to learn. In this work, we consider this variation in the reinforcements as noise added to the reinforcement; therefore, the proposal is to filter the reinforcement signal in order to remove the noise. In particular, a moving average filter is used; this is one of the most simple filters used in Digital Signal Processing. The proposed variation is tested in a classical RL problem, namely, Gridworlds with different characteristics regarding size, structure and noise level. Results show that our proposed approach finds the optimal solution under some conditions in which the classical procedure cannot find it