Generalising Combinatorial Discriminant Analysis through Conditioning Truncated Rayleigh Flow

Fisher\u27s Linear Discriminant Analysis (LDA) has been widely used for linear classification, feature selection, and metrics learning in multivariate data analytics. To ensure high classification accuracy while optimally discovering predictive features from the data, this paper studied CDA, namely Combinatorial Discriminant Analysis that intends to combinatorially select a subset of features and assign weights to them optimally. CDA extents the Truncated Rayleigh Flow algorithm (Tan et al. in J R Stat Soc: Ser B (Stat Methodol) 80(5):1057–1086, 2018) and improves LDA estimation under k-sparsity constraint. The experimental results based on the synthesized and real-world datasets demonstrate that our algorithm outperforms other LDA baselines and downstream classifiers. The empirical analysis shows that our algorithm can recover the combinatorial structure of optimal LDA with empirical consistency