Large-Scale Marine Image Annotation in the Age of the Web and Deep Learning

Zurowietz M. Large-Scale Marine Image Annotation in the Age of the Web and Deep Learning. Bielefeld: Universität Bielefeld; 2022.Digital imaging has become one of the most important techniques to non-invasively collect data in the field of marine benthic environmental monitoring and exploration. Traditionally, marine imaging data is analyzed by manual image annotation where domain experts mark objects of interest in the images and assign class labels to the marked objects. With technological advances of underwater carrier systems, digital cameras and digital storage technology, the acquisition rate of marine imaging data is rapidly increasing. Traditional purely manual image annotation cannot keep up with the volume of newly acquired data, as the availability of domain experts who can annotate the images is very limited. Hence, new (computational) approaches that increase both the efficiency and effectivity of marine image annotation are required. In this thesis, BIIGLE 2.0 is presented, which is a web-based application for image annotation with a special focus on marine imaging. BIIGLE 2.0 offers several novel concepts and annotation tools that enable highly efficient manual image annotation. Furthermore, the application architecture of BIIGLE 2.0 allows for a versatile deployment from a mobile single-chip computer in the field up to a large cloud-based stationary setup. The possibility to synchronize annotation data between multiple BIIGLE 2.0 instances and a federated search pave the way for the creation of a powerful collaborative network of annotation systems across research ships, monitoring stations or research institutes. In addition, the Machine learning Assisted Image Annotation method (MAIA) and its extension through Unsupervised Knowledge Transfer (UnKnoT) are presented. MAIA introduces a four-stage image annotation workflow that includes machine learning methods for computer vision to automate the time-consuming task of object detection. This allows human observers to annotate large marine image collections much faster than before. With UnKnoT, the first two MAIA stages for unsupervised object detection can be skipped for datasets with special properties that are often given in the benthic marine imaging context, accelerating the workflow even more. The combination of BIIGLE 2.0, MAIA and UnKnoT presents an advancement for marine image annotation that integrates manual annotation with specialized software, automated computer assistance and a sophisticated user interface for a highly efficient and effective annotation process. In addition, the method and tool Interactive Feature Localization in Deep neural networks (IFeaLiD) is presented, which offers a novel way for the inspection of convolutional neural networks for computer vision. IFeaLiD can be used, among other objectives, to judge the suitability of a particular trained network for a specific task such as object detection in the marine imaging context