Discriminative training by iterative linear programming optimization

In this paper, we cast discriminative training problems into standard linear programming (LP) optimization. Besides being convex and having globally optimal solution(s), LP programs are well-studied with well-established solutions, and efficient LP solvers are freely available. In practice, however, one may not have complete knowledge of the feasible region since it is constructed from a limited number of competing hypotheses based on the current model - not the final model which, by definition, is not known a priori at the time of hypotheses generation. We investigate an iterative LP optimization algorithm in which an additional constraint on the parameters being optimized is further imposed. Our proposed method is evaluated on the estimation of global and state-dependent stream weights and biases of a multi-stream hidden Markov model system. Results show that the stream weights and biases found by our iterative LP optimization algorithm may give better recognition performance than the ones found by a brute-force grid search. ©2008 IEEE