Using immersive audio and vibration to enhance remote diagnosis of mechanical failure in uncrewed vessels.

There is increasing interest in the maritime industry in the potential use of uncrewed vessels to improve the efficiency and safety of maritime operations. This leads to a number of questions relating to the maintenance and repair of mechanical systems, in particular, critical propulsion systems which if a failure occurs could endanger the vessel. While control data is commonly monitored remotely, engineers on board ship also employ a wide variety of sensory feedback such as sound and vibration to diagnose the condition of systems, and these are often not replicated in remote monitoring. In order to assess the potential for enhancement of remote monitoring and diagnosis, this project simulated an engine room (ER) based on a real vessel in Unreal Engine 4 for the HTC ViveTM VR headset. Audio was recorded from the vessel, with mechanical faults synthesized to create a range of simulated failures. In order to simulate operational requirements, the system was remotely fed data from an external server. The system allowed users to view normal control room data, listen to the overall sound of the space presented spatially over loudspeakers, isolate the sound of particular machinery components, and feel the vibration of machinery through a body worn vibration transducer. Users could scroll through a 10-hour time history of system performance, including audio, vibration and data for snapshots at hourly intervals. Seven experienced marine engineers were asked to assess several scenarios for potential faults in different elements of the ER. They were assessed both quantitatively regarding correct fault identification, and qualitatively in order to assess their perception of usability of the system. Users were able to diagnose simulated mechanical failures with a high degree of accuracy, mainly utilising audio and vibration stimuli, and reported specifically that the immersive audio and vibration improved realism and increased their ability to diagnose system failures from a remote location