Framework for Semantic Integration and Scalable Processing of City Traffic Events

Intelligent traffic management requires analysis of a large volume of multimodal data from diverse domains. For the development of intelligent traffic applications, we need to address diversity in observations from physical sensors which give weather, traffic flow, parking information; we also need to do the same with social media, which provides live commentary of various events in a city. The extraction of relevant events and the semantic integration of numeric values from sensors, unstructured text from Twitter, and semi- structured data from city authorities is a challenging physical-cyber-social data integration problem. In order to address the challenge of both scalability and semantic integration, we developed a semantics-enabled distributed framework to support processing of multimodal data gushing in at a high volume. To semantically integrate traffic events related complementary data from multimodal data streams, we developed a Traffic Event Ontology consistent with a Semantic Web approach. We utilized Apache Spark and Parquet data store to address the volume issue and to build the scalable infrastructure that can process and extract traffic events from historical as well as streaming data from 511.org (sensor data) and Twitter (textual data). We present the large-scale evaluation of our system on real-world traffic-related data from the San Francisco Bay Area over one year with promising results. Our scalable approach was able to decrease the processing time of the test case we present in this work from two months to less than 24 hours. We evaluated our scalability method by varying input data loads and the system showed stability in the performance. Additionally, we evaluated the performance of our semantic integration method by answering questions related to traffic anomalies using multimodal data