Automatic analysis of the content of cell biological videos, and database organization of their metadata descriptors

Abstract—We present a video content analysis and metadata organizational system for research videos arising from biological microscopy of living cells. Automated procedures are described to determine the position, size, shape and orientation of cells in each video frame. From the temporal changes in the values of these simple metadata parameters, high-level descriptors are derived that describe the semantic content of the video. This content information (specific intrinsic metadata) is of high information value, since it describes the behavior of cells and the timing of events within the video, including changes in environmental con-ditions experienced by the cells. When such metadata are properly organized in a searchable database, a content-based video query and retrieval system may be developed to locate particular objects, events or behaviors. Moreover, the availability of such semantic contents in the formal and generic format we propose will allow the application of data mining techniques and the amassing of more elaborate knowledge, e.g., species classification depending on behavior, patterns in response to environment changes, etc. The suitability and functionality of the proposed metadata model is demonstrated by the automated analysis of five different types of biological experiments, recording epithelial wound healing, bacterial multiplication, the rotations of tethered bacteria, and the swimming of motile bacteria and of human sperm. Index Terms—Biological videos, content analysis, content recog-nition, semantic metadata. I