FAST(ER) DATA GENERATION FOR OFFLINE RL AND FPS ENVIRONMENTS FOR DECISION TRANSFORMERS

Reinforcement learning algorithms have traditionally been implemented with the goalof maximizing a reward signal. By contrast, Decision Transformer (DT) uses a transformermodel to predict the next action in a sequence. The transformer model is trained on datasetsconsisting of state, action, return trajectories. The original DT paper examined a smallnumber of environments, five from the Atari domain, and three from continuous control,and one that examined credit assignment. While this gives an idea of what the decisiontransformer can do, the variety of environments in the Atari domain are limited. In thiswork, we propose an extension of the environments that decision transformer can be trainedon by adding support for the VizDoom environment. We also developed a faster method foroffline RL dataset generation, using Sample Factory, a library focused on high throughput,to generate a dataset comparable in quality to existing methods using significantly less time.