River segmentation for autonomous surface vehicle localization and river boundary mapping

We present a vision‐based algorithm that identifies the boundary separating water from land in a river environment containing specular reflections. Our approach relies on the law of reflection. Assuming the surface of water behaves like a horizontal mirror, the border separating land from water corresponds to the border separating three‐dimensional (3D) data which are either above or below the surface of water. We detect a river by identifying this border in a stereo camera. We start by demonstrating how to robustly estimate the normal and height of the water's surface with respect to a stereo camera. Then, we segment water from land by identifying the boundary separating dense 3D stereo data which are either above or below the water's surface. We explicitly show how to find this boundary by formulating and solving a graph‐based optimization problem using dense 3D stereo data near the shoreline and Dijkstra's algorithm. With the border of water identified, we validate the proposed river boundary detection algorithm by applying it to a chronologically sequential video sequence obtained from the visual‐inertial canoe data set. The intended purpose of the proposed river segmentation algorithm is to be used as a front‐end object recognition module for solving the simultaneous localization and mapping (SLAM) problem; therefore, using the extracted river boundary, we apply the recently developed visual‐inertial Curve SLAM algorithm to localize a canoe and create a sparse map that recovers the outline, shape, and dimensions of the shoreline of a river