Learning bayesian networks with large-scale problems and computing paradigms

Anyone working in machine learning requires a particular balance between multiple disciplines. A solid mathematical background, proficiency with computer science tools and a deep expert domain are one of the most commonly attributed. We can find a large number of real problems that can be solved using machine learning techniques and usually require the involvement of many experts, where each of them contribute their specialty to the particular solution. We can say that machine learning depends heavily on technology, however the opposite is true as well. Artificial intelligence has motivated some of the most important advances in computer science of the last years. From the hardware point of view, computing power has been key in the development of this discipline and machine learning is perhaps the field which utilises more specialized architectures such as GPUs and the recent TPUs for deep learning. In addition to computing advances, the last decades have witnessed the arising of the data revolution, with the advent of big data as a scientific and business discipline, it is perhaps one of the most widely adopted technologies since the Internet. Again, the stimulus of this field has propitiated the cheapening and optimization of data storage technology, leading humanity to produce and store more information in a couple of years than in all known history. Methodological computer science has evolved as well, with the appearance of new data science languages and libraries that have shaped the profiles of many professionals from different fields. Expressiveness and programming models that are closer to mathematical language are required in order to create the next wave of intelligent software and the ingestion and processing of such a vast amount of data. All of this, happening at an ever increasing rate that permanently men-aces every new technology with becoming obsolete. Scientific research in the field of machine learning studies new approaches to improve existing algorithms as well as new unexplored approaches to im-prove this domain. The field is reaching maturity with many results been already adopted by the industry. However, at the current rate of technological expanse we should revisit and adapt existing techniques to new software and hardware developments. In this dissertation we will focus on computational scalability, specifically in the approaches to learn from massive amounts of data in large distributed computing technologies. This has been an active field of research both in academia and industry with ground breaking results that have defined what we know today as big data. We will focus on a particular framework known as probabilistic graphical models and especially on Bayesian networks [Pearl, 2014]. This formalism combines a mathematically sound background such as the probability theory and many other desirable properties, like the capacity for providing explanation and interpretation of their results, something that many of the algorithms lack or provide artificially. This, in addition to a range of successful practical applications has gained it a recognised spot in the machine learning community. However, to the best of our knowledge, this models are not considered by the industry as part of the available big data solutions and are not present in any of the standard libraries that are being applied in real life problems. While other techniques are perhaps more easily portable to such scalable technologies, Bayesian networks are scalable both from a theoretical and practical point of view and have been studied from this perspective in the past prior to the popularity of new big data computing frameworks. We will explore several applications of this formalism to scalable problems in many domains and specific machine learning problem such as structural learning and supervised classification. 1.2 Outline of the dissertation In Chapter 2 we will begin with a thorough introduction to the basic concepts of probabilistic graphical models and Bayesian networks that we will use during the remaining of the dissertation. We will follow with an introduction to the technological development of computing architectures in Chapter 3, starting from the single computer and disk machine, to the distributed cluster and finally developing into their evolution to modern specialized big data architectures and cloud computing. We will describe the new computing paradigms that have been developed for these architectures and that are perhaps among the most demanded techno-logical skill in the development and machine learning community. These will be used as well during the dissertation as the technological base on which we will build our proposals. Chapter 4 proposes a new scalable structural algorithm for Bayesian networks, developed for scalability in traditional computing architectures. We will explore existing scalable algorithms and identify some problems for which we will propose novel solutions. The metrics and algorithms introduced in this sections will serve as a basis for other developments that will be introduced further during the dissertation. In Chapter 5 we explore applications of probabilistic graphical models to multi-dimensional supervised classification, a problem that is less known in the community but very common in practice. We propose a new framework to deal with this type of data and provide experiments and sound conclusions to apply it to real life problems in scalable architectures. Following with Chapter 6 we will shift our focus to supervised classification, which is one of the most demanded problems in the machine learning scalable libraries. We will provide a wide overview of the posibilites of Bayesian networks scalable applications to this problem, specifically for the current state-of-the-art in big data computing. We propose a general framework to deal with massive problems as well as a thorough experimentation in order to provide future practitioners with guidelines to use our results. The last contribution in Chapter 7 explores original approaches on Bayesian networks, specifically designed to work in big data problems and technologies. We will identify the common pitfalls of existing methods and provide new strategies to exploit the benefits of new highly distributed computing paradigms