Exploring haptics for subsea vehicles: Haptic feedback for rate controlled vehicles in subsea environments

Remotely controlled subsea vehicles are frequently used for oil and gas applications. A potential future application requiring remote controlled vehicles is deep-sea mining. At the envisioned water depths beyond 1500m, rare minerals are accessible without deep excavation. However, the extreme hyperbaric conditions (i.e. high pressure), limited visibility and unpredictable soil properties pose immense challenges in controlling the excavation process. Such machines are expected to be operated manually by an operator using joysticks that manipulate the machine’s operational velocity (also known as ’rate control’). Large subsea vehicles are difficult to control due to their complexity and slow dynamic response. This thesis explores design choices for haptic feedback that can support the operator in controlling these machines. Offering haptic feedback (i.e. forces on the input device) can potentially improve task performance and operator awareness, by informing the operator of the naturally occurring (possibly scaled) interaction forces with the environment. Alternatively, artificial guidance forces based on a model or sensed environment can be used to guide or constrain the operator’s control actions. Force reflection in a rate controlled task poses a difficulty compared to a position controlled task, because the reflected forces are no longer directly related to the operator’s input position. The goal of this thesis is to provide design guidelines for haptic feedback, by designing and evaluating several haptic feedback algorithms, for a variety of remotely controlled sub-sea vehicles. First the thesis will present an analysis of the general task environment of deep-sea mining, including a choice for the most likely options for machinery to be used in the envisioned operations. Secondly, the design of natural haptic feedback is explored for controlling a large heavy backhoe dipper excavator, operating in a shallow subsea environment. And thirdly, the implementation of haptic guidance forces is studied for rate controlled devices and its effect on steering a deep-sea mining crawler. 1) General task environment of deep-sea mining, machines and mineralsDeep-sea mining applications require large heavy machinery, to excavate mineral-rich rock materials. Excavating rock in large water depths requires more energy than on land, due to hardening of the material in hyperbaric conditions (chapter 2). Two possible deep-sea mining approaches are compared: using a large suspended grab with two clamshells, and track-driven drum cutters. The suspended grab is shown to reduce energy consumption, due to a reduction of hyperbaric hardening-effect caused by slow loading of the material thereby allowing water to enter the effected deformed zone (chapter 2). Using a grab is a promising excavation method for deep-sea mining due to the low loading rates and only crushing parts of the material, leaving most intact. Controlling such a machine while exerting large cutting forces onto the seabed is a challenging task. Offering haptic feedback to the operator by means of natural force feedback and haptic shared control combined potentially improves the situational awareness and control effort (chapter 3). Further investigation into both types of support (i.e. natural and guidance feedback) needs to be done for these type of large subsea machines.2) Exploring natural haptic feedback for vehicles with a slow dynamic responseSubsea vehicles typically are large and heavy, thereby having a slow dynamic response. This requires predictive inputs from the operator for controlling the vehicles’ position. Natural haptic feedback increases the situational awareness of the operator, enabling better understanding of the state of the machine and anticipation of the required control inputs (chapter 4). It is shown that scaling of the reflected forces during position control does not affect the perception of the controlled vehicle’s response, thereof prediction of the required inputs. Using rate control has an unlimited workspace, required for steering heavy machines over the seabed. Offering natural feedback in rate control is however not as obvious as it is for position control, where the measured forces can be reflected directly related to the position. Implementing stiffness feedback showed promising results for offering natural haptic feedback in rate control for operating a slow dynamic system, compared to force-based feedback and static feedback of a centering spring (chapter 5). This was tested for the fundamental abstract subtask of positioning in free-space, a contact transition and force level tasks. Controlling a backhoe dipper excavator on a pontoon for excavation in harbors or offshore shallow waters is a challenging task due to the machine’s complexity and slow dynamics. A high fidelity force reflecting joystick was developed to demonstrate the effect of implementing stiffness feedback for controlling an excavator, based on the measured hydraulic cylinder pressures, representing the environment interaction forces (chapter 6). A human factors case study showed that several operating effects can be clearly reflected by means of stiffness feedback, such as making contact with the seabed and cutting through sand layers.3) Exploring haptic guidance feedback designsInstead of informing the operator of what the machine is doing, haptic guidance feedback based on a model or sensed environment can assist the operator in correct task execution. This thesis explores two types of design of guidance feedback, by means of a repulsive force field around forbidden zones or attractive forces towards a suggested path. The latter requires more sensed information from the environment, but showed most improvements for steering an abstract vehicle through a virtual maze (chapter 7).Haptic shared control is an attractive guidance towards a suggested path, sharing the control with the operator on the input device. For a deep-sea crawler maneuvering over the seabed haptic guidance is compared to semi-automated control and manual control (chapter 8). This showed that sharing the control is beneficial due to automation during normal operating conditions, but also from manual control in unexpected events such as obstacle avoidance or slip conditions.In conclusion, both natural haptic feedback and haptic guidance feedback were evaluated on abstract tasks as well as real-life tasks simulated in virtual reality. Combining natural haptic feedback and guidance feedback is recommended for rate controlled tasks, to inform the operator on interaction forces as well for as assisting in task execution. The combination of feedback can be offered to the operator by means of stiffness reflection combined with guidance by haptic shared control, which shifts the neutral position of the stiffness.Human-Robot Interactio