Online Learning From Finite Training Sets and

this paper, we give an exact analysis of online learning in a simple model system. Our aim is twofold: (1) to assess how the combination of non-infinitesimal learning rates j and finite training sets (containing ff examples per weight) affects online learning, and (2) to compare the generalization performance of online and offline learning. A priori, one Online learning can also be used to learn teacher rules that vary in time. The assumption of an infinite set (or `stream') of training examples is then much more plausible, and in fact necessary for continued adaptation of the student. We do not consider this case in the followin