Identifying Research Gap and Opportunities in the use of Multimodal Deep Learning for Emergency Management

With the proliferation of smart mobile devices, people are now increasingly using social media applications during disasters to share updates, check on loved ones, or inform officials. Additionally, there are many other sources such as remote sensing, CCTV monitoring, wireless sensor networks and mobile GPS which provide disaster-related data. As a result, an overwhelming amount of data is generated in different modalities (text, audio, video and images) during a crisis. However, extracting, analysing and interpreting a huge variety of multimodal data within a short period of time is a major challenge faced by emergency responders. Therefore, decisions still mostly depend on text-based reports prepared by field officers. In order to better understand the problem, to make more informed decisions and to have well-coordinated responses, it is necessary to process multiple modalities of data. As Multimodal Deep Learning (MMDL) techniques gain popularity among Artificial Intelligence (AI) researchers, it is timely to discuss the potential of such techniques for use in emergency management. MMDL addresses the problem of relating different representations from multimodal data to each other. Applying MMDL techniques on disaster data would help decision makers by improving the accuracy of data, reducing uncertainty in decision making, and reducing the time taken to analyse data thus assisting emergency responders to understand the “big picture” clearly and support effective disaster relief. Furthermore, the community may benefit by being better prepared, stronger and more resilient during emergencies. This work focuses on defining the research gap, opportunities and limitations of using MMDL techniques in disaster research