Active learning for pipeline models

For many machine learning solutions to complex ap-plications, there are significant performance advantages to decomposing the overall task into several simpler sequential stages, commonly referred to as a pipeline model. Typically, such scenarios are also characterized by high sample complexity, motivating the study of ac-tive learning for these situations. While most active learning research examines single predictions, we ex-tend such work to applications which utilize pipelined predictions. Specifically, we present an adaptive strat-egy for combining local active learning strategies into one that minimizes the annotation requirements for the overall task. Empirical results for a three-stage entity and relation extraction system demonstrate a significant reduction in supervised data requirements when using the proposed method