A reinforcement learning path planning approach for range-only underwater target localization with autonomous vehicles

Underwater target localization using range-only and single-beacon (ROSB) techniques with autonomous vehicles has been used recently to improve the limitations of more complex methods, such as long baseline and ultra-short baseline systems. Nonetheless, in ROSB target localization methods, the trajectory of the tracking vehicle near the localized target plays an important role in obtaining the best accuracy of the predicted target position. Here, we investigate a Reinforcement Learning (RL) approach to find the optimal path that an autonomous vehicle should follow in order to increase and optimize the overall accuracy of the predicted target localization, while reducing time and power consumption. To accomplish this objective, different experimental tests have been designed using state-of-the-art deep RL algorithms. Our study also compares the results obtained with the analytical Fisher information matrix approach used in previous studies. The results revealed that the policy learned by the RL agent outperforms trajectories based on these analytical solutions, e.g. the median predicted error at the beginning of the target’s localisation is 17% less. These findings suggest that using deep RL for localizing acoustic targets could be successfully applied to in-water applications that include tracking of acoustically tagged marine animals by autonomous underwater vehicles. This is envisioned as a first necessary step to validate the use of RL to tackle such problems, which could be used later on in a more complex scenarios.This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 893089. This work also received financial support from the Spanish Ministerio de Economía y Competitividad (SASES: RTI2018-095112-B-I00; BITER-ECO: PID2020-114732RB C31). This work acknowledges the ‘Severo Ochoa Centre of Excellence’ accreditation (CEX2019-000928-S), and from the Generalitat de Catalunya ”Sistemas de Adquisicion Remota de datos y Tratamiento de la Informacion en el Medio Marino (SARTI-MAR)” 2017 SGR 376. We gratefully acknowledge the David and Lucile Packard Foundation.Peer ReviewedPostprint (author's final draft