Detecting Weak Signals of the Future: A System Implementation Based on Text Mining and Natural Language Processing

[EN] Organizations, companies and start-ups need to cope with constant changes on the market which are difficult to predict. Therefore, the development of new systems to detect significant future changes is vital to make correct decisions in an organization and to discover new opportunities. A system based on business intelligence techniques is proposed to detect weak signals, that are related to future transcendental changes. While most known solutions are based on the use of structured data, the proposed system quantitatively detects these signals using heterogeneous and unstructured information from scientific, journalistic and social sources, applying text mining to analyze the documents and natural language processing to extract accurate results. The main contributions are that the system has been designed for any field, using different input datasets of documents, and with an automatic classification of categories for the detected keywords. In this research paper, results from the future of remote sensors are presented. Remote sensing services are providing new applications in observation and analysis of information remotely. This market is projected to witness a significant growth due to the increasing demand for services in commercial and defense industries. The system has obtained promising results, evaluated with two different methodologies, to help experts in the decision-making process and to discover new trends and opportunities.This research is partially supported by EIT Climate-KIC of the European Institute of Technology (project EIT Climate-KIC Accelerator-TC_3.1.5_190607_P066-1A) and InnoCENS from Erasmus + (573965-EPP-1-2016-1-SE-EPPKA2-CBHE-JP).Griol-Barres, I.; Milla, S.; Cebrián Ferriols, AJ.; Fan, H.; Millet Roig, J. (2020). Detecting Weak Signals of the Future: A System Implementation Based on Text Mining and Natural Language Processing. Sustainability. 12(19):1-21. https://doi.org/10.3390/su121978481211219Zahra, S. A., Gedajlovic, E., Neubaum, D. O., & Shulman, J. M. (2009). A typology of social entrepreneurs: Motives, search processes and ethical challenges. Journal of Business Venturing, 24(5), 519-532. doi:10.1016/j.jbusvent.2008.04.007Ansoff, H. I. (1975). Managing Strategic Surprise by Response to Weak Signals. California Management Review, 18(2), 21-33. doi:10.2307/41164635Report on Weak Signals Collection. TELMAP, European Commission Seventh Framework Project (IST-257822) https://cordis.europa.eu/docs/projects/cnect/2/257822/080/deliverables/001-D41Weaksignalscollectionfinal.docDator, J. (2005). Universities without «quality» and quality without «universities». On the Horizon, 13(4), 199-215. doi:10.1108/10748120510627321Hiltunen, E. (2008). The future sign and its three dimensions. Futures, 40(3), 247-260. doi:10.1016/j.futures.2007.08.021Thorleuchter, D., Scheja, T., & Van den Poel, D. (2014). Semantic weak signal tracing. Expert Systems with Applications, 41(11), 5009-5016. doi:10.1016/j.eswa.2014.02.046Julien, P.-A., Andriambeloson, E., & Ramangalahy, C. (2004). Networks, weak signals and technological innovations among SMEs in the land-based transportation equipment sector. Entrepreneurship & Regional Development, 16(4), 251-269. doi:10.1080/0898562042000263249Xindong Wu, Xingquan Zhu, Gong-Qing Wu, & Wei Ding. (2014). Data mining with big data. IEEE Transactions on Knowledge and Data Engineering, 26(1), 97-107. doi:10.1109/tkde.2013.109Koivisto, R., Kulmala, I., & Gotcheva, N. (2016). Weak signals and damage scenarios — Systematics to identify weak signals and their sources related to mass transport attacks. Technological Forecasting and Social Change, 104, 180-190. doi:10.1016/j.techfore.2015.12.010Davis, J., & Groves, C. (2019). City/future in the making: Masterplanning London’s Olympic legacy as anticipatory assemblage. Futures, 109, 13-23. doi:10.1016/j.futures.2019.04.002Irvine, N., Nugent, C., Zhang, S., Wang, H., & NG, W. W. Y. (2019). Neural Network Ensembles for Sensor-Based Human Activity Recognition Within Smart Environments. Sensors, 20(1), 216. doi:10.3390/s20010216Huang, M., & Liu, Z. (2019). Research on Mechanical Fault Prediction Method Based on Multifeature Fusion of Vibration Sensing Data. Sensors, 20(1), 6. doi:10.3390/s20010006Awan, F. M., Saleem, Y., Minerva, R., & Crespi, N. (2020). A Comparative Analysis of Machine/Deep Learning Models for Parking Space Availability Prediction. Sensors, 20(1), 322. doi:10.3390/s20010322MohamadiBaghmolaei, R., Mozafari, N., & Hamzeh, A. (2017). Continuous states latency aware influence maximization in social networks. AI Communications, 30(2), 99-116. doi:10.3233/aic-170720McGrath, J., & Fischetti, J. (2019). What if compulsory schooling was a 21st century invention? Weak signals from a systematic review of the literature. International Journal of Educational Research, 95, 212-226. doi:10.1016/j.ijer.2019.02.006Chao, W., Jiang, X., Luo, Z., Hu, Y., & Ma, W. (2019). Interpretable Charge Prediction for Criminal Cases with Dynamic Rationale Attention. Journal of Artificial Intelligence Research, 66, 743-764. doi:10.1613/jair.1.11377Van Veen, B. L., Roland Ortt, J., & Badke-Schaub, P. G. (2019). Compensating for perceptual filters in weak signal assessments. Futures, 108, 1-11. doi:10.1016/j.futures.2019.02.018Thorleuchter, D., & Van den Poel, D. (2015). Idea mining for web-based weak signal detection. Futures, 66, 25-34. doi:10.1016/j.futures.2014.12.007Rowe, E., Wright, G., & Derbyshire, J. (2017). Enhancing horizon scanning by utilizing pre-developed scenarios: Analysis of current practice and specification of a process improvement to aid the identification of important ‘weak signals’. Technological Forecasting and Social Change, 125, 224-235. doi:10.1016/j.techfore.2017.08.001Yoon, J. (2012). Detecting weak signals for long-term business opportunities using text mining of Web news. Expert Systems with Applications, 39(16), 12543-12550. doi:10.1016/j.eswa.2012.04.059Yoo, S., & Won, D. (2018). Simulation of Weak Signals of Nanotechnology Innovation in Complex System. Sustainability, 10(2), 486. doi:10.3390/su10020486Suh, J. (2018). Generating Future-Oriented Energy Policies and Technologies from the Multidisciplinary Group Discussions by Text-Mining-Based Identification of Topics and Experts. Sustainability, 10(10), 3709. doi:10.3390/su10103709Kwon, L.-N., Park, J.-H., Moon, Y.-H., Lee, B., Shin, Y., & Kim, Y.-K. (2018). Weak signal detecting of industry convergence using information of products and services of global listed companies - focusing on growth engine industry in South Korea -. Journal of Open Innovation: Technology, Market, and Complexity, 4(1). doi:10.1186/s40852-018-0083-6Ben-Porat, O., Hirsch, S., Kuchi, L., Elad, G., Reichart, R., & Tennenholtz, M. (2020). Predicting Strategic Behavior from Free Text. Journal of Artificial Intelligence Research, 68. doi:10.1613/jair.1.11849Fink, L., Yogev, N., & Even, A. (2017). Business intelligence and organizational learning: An empirical investigation of value creation processes. Information & Management, 54(1), 38-56. doi:10.1016/j.im.2016.03.009Ilmola, L., & Kuusi, O. (2006). Filters of weak signals hinder foresight: Monitoring weak signals efficiently in corporate decision-making. Futures, 38(8), 908-924. doi:10.1016/j.futures.2005.12.019Doulamis, N. D., Doulamis, A. D., Kokkinos, P., & Varvarigos, E. M. (2016). Event Detection in Twitter Microblogging. IEEE Transactions on Cybernetics, 46(12), 2810-2824. doi:10.1109/tcyb.2015.2489841Atefeh, F., & Khreich, W. (2013). A Survey of Techniques for Event Detection in Twitter. Computational Intelligence, 31(1), 132-164. doi:10.1111/coin.12017Mehmood, N. Q., Culmone, R., & Mostarda, L. (2017). Modeling temporal aspects of sensor data for MongoDB NoSQL database. Journal of Big Data, 4(1). doi:10.1186/s40537-017-0068-5Bjeladinovic, S. (2018). A fresh approach for hybrid SQL/NoSQL database design based on data structuredness. Enterprise Information Systems, 12(8-9), 1202-1220. doi:10.1080/17517575.2018.1446102Čerešňák, R., & Kvet, M. (2019). Comparison of query performance in relational a non-relation databases. Transportation Research Procedia, 40, 170-177. doi:10.1016/j.trpro.2019.07.027Yangui, R., Nabli, A., & Gargouri, F. (2016). Automatic Transformation of Data Warehouse Schema to NoSQL Data Base: Comparative Study. Procedia Computer Science, 96, 255-264. doi:10.1016/j.procs.2016.08.138Willett, P. (2006). The Porter stemming algorithm: then and now. Program, 40(3), 219-223. doi:10.1108/00330330610681295Griol-Barres, I., Milla, S., & Millet, J. (2019). Implementación de un sistema de detección de señales débiles de futuro mediante técnicas de minería de textos. Revista española de Documentación Científica, 42(2), 234. doi:10.3989/redc.2019.2.1599Kim, J., Han, M., Lee, Y., & Park, Y. (2016). Futuristic data-driven scenario building: Incorporating text mining and fuzzy association rule mining into fuzzy cognitive map. Expert Systems with Applications, 57, 311-323. doi:10.1016/j.eswa.2016.03.043Mendonça, S., Pina e Cunha, M., Kaivo-oja, J., & Ruff, F. (2004). Wild cards, weak signals and organisational improvisation. Futures, 36(2), 201-218. doi:10.1016/s0016-3287(03)00148-4Ishikiriyama, C. S., Miro, D., & Gomes, C. F. S. (2015). Text Mining Business Intelligence: A small sample of what words can say. Procedia Computer Science, 55, 261-267. doi:10.1016/j.procs.2015.07.044Yuen, J. (2018). Comparison of Impact Factor, Eigenfactor Metrics, and SCImago Journal Rank Indicator and h-index for Neurosurgical and Spinal Surgical Journals. World Neurosurgery, 119, e328-e337. doi:10.1016/j.wneu.2018.07.144Thomason, J., Padmakumar, A., Sinapov, J., Walker, N., Jiang, Y., Yedidsion, H., … Mooney, R. (2020). Jointly Improving Parsing and Perception for Natural Language Commands through Human-Robot Dialog. Journal of Artificial Intelligence Research, 67, 327-374. doi:10.1613/jair.1.11485Tseng, Y.-H., Lin, C.-J., & Lin, Y.-I. (2007). Text mining techniques for patent analysis. Information Processing & Management, 43(5), 1216-1247. doi:10.1016/j.ipm.2006.11.011Gergelova, M. B., Labant, S., Kuzevic, S., Kuzevicova, Z., & Pavolova, H. (2020). Identification of Roof Surfaces from LiDAR Cloud Points by GIS Tools: A Case Study of Lučenec, Slovakia. Sustainability, 12(17), 6847. doi:10.3390/su12176847Badmos, O., Rienow, A., Callo-Concha, D., Greve, K., & Jürgens, C. (2018). Urban Development in West Africa—Monitoring and Intensity Analysis of Slum Growth in Lagos: Linking Pattern and Process. Remote Sensing, 10(7), 1044. doi:10.3390/rs10071044Thomson, E., Malhi, Y., Bartholomeus, H., Oliveras, I., Gvozdevaite, A., Peprah, T., … Doughty, C. (2018). Mapping the Leaf Economic Spectrum across West African Tropical Forests Using UAV-Acquired Hyperspectral Imagery. Remote Sensing, 10(10), 1532. doi:10.3390/rs10101532Samasse, K., Hanan, N., Tappan, G., & Diallo, Y. (2018). Assessing Cropland Area in West Africa for Agricultural Yield Analysis. Remote Sensing, 10(11), 1785. doi:10.3390/rs10111785Anchang, J. Y., Prihodko, L., Kaptué, A. T., Ross, C. W., Ji, W., Kumar, S. S., … Hanan, N. P. (2019). Trends in Woody and Herbaceous Vegetation in the Savannas of West Africa. Remote Sensing, 11(5), 576. doi:10.3390/rs11050576Jung, H. C., Getirana, A., Arsenault, K. R., Holmes, T. R. H., & McNally, A. (2019). Uncertainties in Evapotranspiration Estimates over West Africa. Remote Sensing, 11(8), 892. doi:10.3390/rs11080892Mondal, P., Liu, X., Fatoyinbo, T. E., & Lagomasino, D. (2019). Evaluating Combinations of Sentinel-2 Data and Machine-Learning Algorithms for Mangrove Mapping in West Africa. Remote Sensing, 11(24), 2928. doi:10.3390/rs11242928Meftah, M., Damé, L., Keckhut, P., Bekki, S., Sarkissian, A., Hauchecorne, A., … Bui, A. (2019). UVSQ-SAT, a Pathfinder CubeSat Mission for Observing Essential Climate Variables. Remote Sensing, 12(1), 92. doi:10.3390/rs12010092Zhang, W., Yoshida, T., & Tang, X. (2008). Text classification based on multi-word with support vector machine. Knowledge-Based Systems, 21(8), 879-886. doi:10.1016/j.knosys.2008.03.044Griol-Barres, I., Milla, S., & Millet, J. (2020). Improving strategic decision making by the detection of weak signals in heterogeneous documents by text mining techniques. AI Communications, 32(5-6), 347-360. doi:10.3233/aic-190625Dzedzickis, A., Kaklauskas, A., & Bucinskas, V. (2020). Human Emotion Recognition: Review of Sensors and Methods. Sensors, 20(3), 592. doi:10.3390/s20030592Haegeman, K., Marinelli, E., Scapolo, F., Ricci, A., & Sokolov, A. (2013). Quantitative and qualitative approaches in Future-oriented Technology Analysis (FTA): From combination to integration? Technological Forecasting and Social Change, 80(3), 386-397. doi:10.1016/j.techfore.2012.10.002Silva, V. O., Martins, C. A. P. S., & Ekel, P. Y. (2018). An Efficient Parallel Implementation of an Optimized Simplex Method in GPU-CUDA. IEEE Latin America Transactions, 16(2), 564-573. doi:10.1109/tla.2018.832741