Deliverable D4.4 - Platform for the interactive pre-processing of camera trap images

This deliverable is a software platform at TRL 8 that processes observations from camera traps. It largely automates a major recurring action for Camera Trap Users to work through thousands of video clips or images that may be empty, filter those observations of interest, propose species names, and - if wanted - upload them to a chosen biodiversity Citizen Observatory, e.g. iSpot, with the proposed identification. The current best practice for automation of this kind is judiciously deploying specifically trained deep neural networks trained on suitable datasets. We chose the Caltech Camera Traps dataset (Beery et al., 2018) based on size, availability of bounding box information and diversity of images, to train and evaluate different neural architectures with a view to select the best state-of-the-art neural models for species identification. After careful experimentation we identified the Cascade R-CNN X-101 64x4d FPN model (Cai and Vasconcelos, 2019) as most suitable for species identification in camera traps. DynAIkon, in the Cos4Cloud framework, has developed an API at https://service.fastcat-cloud.org/api for use by machines and automated workflows to authenticate, upload camera trap images together with a set of species of interest, and to download corresponding annotations. The API is documented at https://service.fastcat-cloud.org/api/spec. The API is also used by our dedicated, interactive web service that allows a user to interactively authenticate, upload camera trap images, choose a machine learning model, download species identification lists, filter images of interest and upload a subset of observations to a citizen observatory. This document gives the necessary background information, user guide and examples on how to use our deliverable D4.4: Platform for the interactive pre-processing of camera trap images. Deliverable related to FASTCAT-Cloud service