Tackling Concept Drift by Temporal Inductive Transfer

The success of machine learning classification pales for real-world, time-varying streams of data. We define three subtypes of concept drift, and confirm that recurrent themes appear in the benchmark dataset Reuters2000. To encourage research in this difficult area, we define a ‘daily classification task ’ (DCT) problem formulation, in which a few random iid training samples are provided each day. Ideally, past training data could be leveraged to improve the current day’s classifier. Empirical results for Reuters2000 show that two likely methods are not successful: (1) the popular idea of a sliding window incorporating recent past training data, and (2) inductive transfer of the previously learned classifiers to provide additional predictive features for the current learning task. The former provides a method of characterizing the degree of concept drift. The latter excels if all past labels are given: ‘hindsight DCT.’