A reinforcement learning path planning approach for range-only underwater target localization with autonomous vehicles

18th International Conference on Automation Science and Engineering (CASE), 20-24 August 2022.-- 8 pages, 7 figures, 1 tableUnderwater target localization using range-only and single-beacon (ROSB) techniques with autonomous vehicles has been used recently to improve the limitations of more complex methods, such as long baseline and ultra-short baseline systems. Nonetheless, in ROSB target localization methods, the trajectory of the tracking vehicle near the localized target plays an important role in obtaining the best accuracy of the predicted target position. Here, we investigate a Reinforcement Learning (RL) approach to find the optimal path that an autonomous vehicle should follow in order to increase and optimize the overall accuracy of the predicted target localization, while reducing time and power consumption. To accomplish this objective, different experimental tests have been designed using state-of-the-art deep RL algorithms. Our study also compares the results obtained with the analytical Fisher information matrix approach used in previous studies. The results revealed that the policy learned by the RL agent outperforms trajectories based on these analytical solutions, e.g. the median predicted error at the beginning of the target’s localisation is 17% less. These findings suggest that using deep RL for localizing acoustic targets could be successfully applied to in-water applications that include tracking of acoustically tagged marine animals by autonomous underwater vehicles. This is envisioned as a first necessary step to validate the use of RL to tackle such problems, which could be used later on in a more complex scenariosThis project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 893089. This work also received financial support from the Spanish Ministerio de Econom´ıa y Competitividad (SASES: RTI2018-095112-B-I00; BITER-ECO: PID2020-114732RBC31). This work acknowledges the ‘Severo Ochoa Centre of Excellence’ accreditation (CEX2019-000928-S), and from the Generalitat de Catalunya ”Sistemas de Adquisición Remota de datos y Tratamiento de la Informaci´on en el Medio Marino (SARTI-MAR)” 2017 SGR 376Peer reviewe