A novel energy-logic model for multi-modality multi-agent data and information fusion

Surveillance systems heavily rely on data collected by multi-modality sensors to detect and characterize behavior of entities and events over a given situation. In order to transform the multi-modality sensors data into useful information leading to actionable decisions, there is an essential need for a robust fusion engine. A robust fusion engine should be able to acquire data from multi-agent sensors and take advantage of spatio-temporal characteristics of multi-modality sensors to create a better situational awareness ability and in particular, assisting with soft fusion of multi-threaded information from variety of sensors under task uncertainties. This dissertation presents a novel Energy-logic model for multi-modality multi-agent data and information fusion. The concept of this energy-logic fusion model is biologically-inspired by the human brain energy perceptual model. Similar to the human brain having designated regions to map immediate sensory experiences and fusing collective heterogeneous sensory perceptions to create a situational understanding for decision-making, the proposed energy logic fusion engine (EL-FE) follows an analogous data to information fusion scheme for actionable decision-making. The principle component of the new fusion engine is based on a transformation called Energy Logic Map. Where, features of sensor inputs are represented by time-varying exponential energy propagation models that collectively form a multi-sensor-feature hyperspace. This hyperspace is then subjected to a series of inference operations that transform sensor feature space to a situational awareness. This dissertation presents the theoretical foundation of the Energy-logic fusion model and describes its techniques for fusion of both homogeneous and heterogeneous sensors. The performance of the Energy-logic fusion model was tested and measured experimentally both in indoor and outdoor environments. A fuzzy logic fusion model was also developed for the purpose of performance comparison with the Energy-logic fusion model. Appropriate metrics are developed for assessment of performance reliability, uncertainty, and computational efficiency of the fusion models. The obtained results are very promising and strongly demonstrate superior performance of the energy-logic fusion model over the Fuzzy logic fusion engine. Applications of the energy-logic fusion engine can be realized for surveillance systems, in particular for homeland security as well as battlefield intelligence