Machine learning applied to big data from marine cabled observatories: A case study of sablefish monitoring in the NE Pacific

Ideas for this paper resulted from discussions during the international workshop “Marine cabled observatories: moving towards applied monitoring for fisheries management, ecosystem function and biodiversity”, funded by Ocean Networks Canada and co-hosted by ICM-CSIC, in Barcelona, Spain on 4–5 October 2018.-- 15 pages, 8 figures, 1 table, supplementary material https://www.frontiersin.org/articles/10.3389/fmars.2022.842946/full#supplementary-material.-- Data availability statement: The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found in the article/Supplementary Material. We made all training computations on the Google Colab notebook12. To repeat the training, please clone the Google Drive repository containing the annotated data at13. All detection, tracking, and time-series analyses are freely available for reproduction atOcean observatories collect large volumes of video data, with some data archives now spanning well over a few decades, and bringing the challenges of analytical capacity beyond conventional processing tools. The analysis of such vast and complex datasets can only be achieved with appropriate machine learning and Artificial Intelligence (AI) tools. The implementation of AI monitoring programs for animal tracking and classification becomes necessary in the particular case of deep-sea cabled observatories, as those operated by Ocean Networks Canada (ONC), where Petabytes of data are now collected each and every year since their installation. Here, we present a machine-learning and computer vision automated pipeline to detect and count sablefish (Anoplopoma fimbria), a key commercially exploited species in the N-NE Pacific. We used 651 hours of video footage obtained from three long-term monitoring sites in the NEPTUNE cabled observatory, in Barkley Canyon, on the nearby slope, and at depths ranging from 420 to 985 m. Our proposed AI sablefish detection and classification pipeline was tested and validated for an initial 4.5 month period (Sep 18 2019-Jan 2 2020), and was a first step towards validation for future processing of the now decade-long video archives from Barkley Canyon. For the validation period, we trained a YOLO neural network on 2917 manually annotated frames containing sablefish images to obtain an automatic detector with a 92% Average Precision (AP) on 730 test images, and a 5-fold cross-validation AP of 93% (± 3.7%). We then ran the detector on all video material (i.e., 651 hours from a 4.5 month period), to automatically detect and annotate sablefish. We finally applied a tracking algorithm on detection results, to approximate counts of individual fishes moving on scene and obtain a time series of proxy sablefish abundance. Those proxy abundance estimates are among the first to be made using such a large volume of video data from deep-sea settings. We discuss our AI results for application on a decade-long video monitoring program, and particularly with potential for complementing fisheries management practices of a commercially important speciesThis work was developed within the framework of the Research Unit Tecnoterra (ICM-CSIC/UPC) and the following project activities: ARIM (Autonomous Robotic sea-floor Infrastructure for benthopelagic Monitoring; MarTERA ERA-Net Cofound); RESBIO (TEC2017-87861-R; Ministerio de Ciencia, Innovación y Universidades); PLOME (PLEC2021-007525/AEI/10.13039/501100011033; Ministerio de Ciencia, Innovación y Universidades); JERICO-S3: (Horizon 2020; Grant Agreement no. 871153); ENDURUNS (Research Grant Agreement H2020-MG-2018-2019-2020 n.824348). We also profited from the funding of the Spanish Government through the ‘Severo Ochoa Centre of Excellence’ accreditation (CEX2019-000928-S). [...] Ocean Networks Canada is funded through Canada Foundation for Innovation-Major Science Initiative (CFI-MSI) fund 3019Peer reviewe