Pre-processing of raw images for pose estimation using physical model-trained convolutional neural network model.

(a) Example of grayscale 8-bit images captured by the 3 orthogonal cameras for a free swimming experiment. (b) Each of the three camera images are background subtracted independently. A background image is created by computing the 90th percentile intensity of every pixel over the entire video. The raw image is subtracted from the background. (c) Background subtracted images for each of the three camera views are independently segmented. The background subtracted object is passed through a Gaussian filter (window size: 5x5 pixels, standard deviation: σ = 1) to reduce high frequency noise. Larvae are detected in each camera image independently using a binary threshold obtained using Otsu’s algorithm and scaling it such that larvae can be consistently detected across all frames. Segmented blobs corresponding to larvae detected simultaneously in all the three camera views are detected. The detection is performed using all the centroids (x0i, y0i) of the blobs (centroids indicated by red dots) detected in camera i (i = {1,2,3}) and the 2-D to 3-D mapping determined during camera calibration. (d) Cropped preprocessed images of a unique larvae (length = 3.8 mm) seen by the three cameras. The cropped images are passed as input to the convolutional neural network model. (PNG)