Towards Multimodal Data Analytics: Integrating Natural Language into Visual Analytics

The continuous interaction between users and the system in visual analytics can be considered a dialogue. We propose the use of multiple two-way channels facilitated by a multimodal interface as a central aspect of interactive visualization design, in particular, the use of natural language with interactive visualization. We discuss key related concepts, potential benefits, challenges and opportunities that emerge as a research agenda for multimodal data analysis