Using machine learning in the adaptive control of a smart environment:Master's thesis

The purpose of this thesis is to study the possibilities and need for utilising machine learning in a smart environment. The most important goal of smart environments is to improve the experience of their inhabitants. This requires adaptation to the behaviour of the users and the other changing conditions in the environment. Hence, the achievement of functional adaptation requires finding a way to change the behaviour of the environment according to the changed user behaviour and other conditions. Machine learning is a research area that studies the techniques which make it possible for software agents to improve their operation over time. The research method chosen in this thesis was to review existing smart environment projects and to analyse the usages of machine learning within them. Based upon these uses, a model for using machine learning in a smart environment was created. As a result, four different categories of machine learning in smart environments were identified: prediction, recognition, detection and optimisation. When deployed to the environment, these categories form a clear loop structure in which the outputs of previous learning agents serve as inputs for the next agents, which ultimately enables the making of changes to the environment according to its current state. This kind of loop is called a control loop in adaptive systems. To evaluate the suitability of the model for using machine learning in a smart environment, two demonstrations were carried out in an environment using a Smart-M3 inter-operability platform, both utilising machine learning in one of the above-discussed categories. In the first experiment neural networks were used to predict query latencies in different situations in the environment. The predictions of the network were compared to the outputs of two simpler models. The results showed that the neural network approach was capable of adapting to rapid changes more quickly. However, it also made more false assumptions about the impact of the different parameters. The second experiment belongs to the optimisation category. In this experiment a decision maker was implemented for a resource allocation problem in a distributed multi-media streaming application. It used reinforcement learning with a look-up table and an implementation of the Q-learning algorithm. After the learning period the agent was capable of making optimal decisions. The experiments confirm that it is suitable to use the model described in this thesis in smart environments. The model includes the most important uses of machine learning and it is consistent with other results in the areas of smart environments and self-adaptive software