Event based switched dynamic bayesian networks for autonomous cognitive crowd monitoring

Human behavior analysis is one of the most important applications in In-telligent Video Surveillance (IVS) field. In most recent systems addressed by re-search, automatic support to the human decisions based on object detection, track-ing and situation assessment tools is integrated as a part of a complete cognitive artificial process including security maintenance procedures actions that are in the scope of the system. In such cases an IVS needs to represent complex situations that describe alternative possible real time interactions between the dynamic ob-served situation and operators\u2019 actions. To obtain such knowledge, particular types of Event based Dynamic Bayesian Networks E-DBNs are here proposed that can switch among alternative Bayesian filtering and control lower level modules to capture adaptive reactions of human operators. It is shown that after the off line learning phase Switched E-DBNs can be used to represent and anticipate possible operators\u2019 actions within the IVS. In this sense acquired knowledge can be used for either fully autonomous security preserving systems or for training of new op-erators. Results are shown by considering a crowd monitoring application in a critical in-frastructure. A system is presented where a Cognitive Node embedding in a struc-tured way Switched E-DBN knowledge can interact with an active visual simula-tor of crowd situations. It is also shown that outputs from such a simulator can be easily compared with video signals coming from real cameras and processed by typical Bayesian tracking methods