Rewarding Air Combat Behavior in Training Simulations 1

Air combat training simulations require high quality virtual agents for optimal training. Reinforcement learning techniques offer the ability to let these virtual agents learn good behavior by rewarding them for their performances. In air combat, the (human or virtual) pilot’s missiles should hit the opponent, while the pilots should avoid being hit by their opponent’s missiles. However, missile hits depend on chance factors called the probability-of-kill (Pk). This makes learning good air combat behavior quite difficult, as low Pk missiles may be rewarded and high Pk missiles may be punished. We propose a method of incorporating the Pk into the learning process. Simulations show that rewarding virtual agents via reinforcement learning based on the Pk of their missiles leads to a 10%-19 % increase in performance under various conditions.