Handling abrupt changes in evolving time-series data

Forecasting and online classification are challenging tasks for the current day industry. Under the influence of many unobservable factors, the concepts that are derived from data tend to change over time. In the sales domain, for instance, the sales of a particular product can change continuously under the influence of temperature, preferences, and many other factors. This problem is known as concept drift and poses the operation of information systems with the challenge of handling this type of behavior. Traditionally, research in the field of concept drift has dealt with concept drift by either adapting to or detecting changes in the target concept. Yet, in large information systems there is, typically, a multitude of products for which we would like to derive a concept. In this dissertation we investigate and show that certain properties of real applications are not covered by state-of-the-art research in this field. Concept drift appears in many different shapes and forms. Each application domain has its own goals and challenges. The contribution of this dissertation is a common framework in which each of the information system applications can be fitted. Using this framework we investigate several use-cases in order to check whether the framework is consistent with their operational assumptions and propose refinements to the general framework. In general, these refinements fall into two categories; the change handling framework and the evaluation framework. In some domains, e.g. the food sales domain, the behavior of objects over time can be unpredictable. Even if an adaptive strategy is used to predict this behavior, the results will be poor due to the absence of information in the object. This, in turn, means that in order for the information system to be fully automated, we need to have a mechanism that is aware of this distinction apart from change handling mechanisms. One of such solutions is to learn the dependency between aspects of behavior and the best suited model for prediction. And even this process is subject to change. Some tasks assume the presence of the true target values. But in many domains this assumption is not valid. In this dissertation we will discuss at least two issues that arise from this mismatch. The first is that, in absence of true values, a learner still has to be on time to cast a prediction. This can be solved by monitoring both the found predictions and changes online and allow for some delay in prediction. The second is that in some cases we simply do not know how to evaluate the outcome of a prediction, even if we do observe the true value. In the case of the food sales prediction, we observe sales figures, but these figures might have a memory. Which means there is a dependency between subsequent instances. This, in turn, implies that simply computing the error over individual instances does not reflect the true performance of the predictors. This could be solved by using a utility based evaluation mechanism rather than one based on a pair-wise error computation. The main contribution of this dissertation is three-fold. The first is in the area of change detection. In the fuel mass prediction we present a change detection framework that uses a parametric, a non-parametric, and a heuristic approach to change detection. For food sales time-series we propose a method of catching abrupt changes in individual products. For stress detection case we propose an automatic labeling mechanism and describe a pipeline design. The second contribution is in the field of meta-learning. Our framework for the prediction of food sales time-series uses product categorization in order to improve the overall food sales prediction based on observed behavioral aspects. The third contribution is in the field of online prediction evaluation. We show that commonly used evaluation methods for time-series prediction do not capture all of the properties of real applications. We propose different views on the performance of predictors that do take these properties into account. These contributions together form the starting point for a better understanding of online performance of predictors under the influence of change and, eventually, for the proactive handling of changes based on observed properties of time-series and groups of time-series