Programmable reinforcement learning agents

We present an expressive agent design language for reinforcement learn-ing that allows the user to constrain the policies considered by the learn-ing process.The language includes standard features such as parameter-ized subroutines, temporary interrupts, aborts, and memory variables, but also allows for unspecified choices in the agent program. For learning that which isn't specified, we present provably convergent learning algo-rithms. We demonstrate by example that agent programs written in the language are concise as well as modular. This facilitates state abstraction and the transferability of learned skills.