Extending Avalanche for Continual Reinforcement Learning: Design, Implementation, Experiments and Continual-Habitat-Lab.

Continual Reinforcement Learning (CRL) combines the non-stationarity assumption of the stream of tasks of continual learning with the agent-environment setting of reinforcement learning. While still in its early stages, CRL has seen a rising interest in publications in recent years. To support this growth, we focus on benchmarks and tools: we extend Avalanche, the staple framework for Continual Learning, to support Reinforcement Learning (AvalancheRL) in order to seamlessly train agents on a continuous stream of tasks, and we introduce Continual Habitat Lab, a high-level library enabling the usage of the photorealistic simulator Habitat-Sim for CRL. We then go through the design of both components and of the technologies on which they're based, while motivating the fundamental choices behind architecture and implementation of RL algorithms. Finally, we show the functionalities of the framework in learning multiple games with experiments on continual control and the Atari suite, as well as demonstrating the integration of Continual Habitat Lab into AvalancheRL as a usable environment