Fast Robust Logistic Regression for Large Sparse Datasets with Binary Outputs

Although popular and extremely well established in mainstream statistical data analysis, logistic regression is strangely absent in the field of data mining. There are two possible explanations of this phenomenon. First, there might be an as-sumption that any tool which can only produce linear classification boundaries is likely to be trumped by more modern nonlinear tools. Sec-ond, there is a legitimate fear that logistic re-gression cannot practically scale up to the mas-sive dataset sizes to which modern data mining tools are This paper consists of an em-pirical examination of the first assumption, and surveys, implements and compares techniques by which logistic regression can be scaled to data with millions of attributes and records. Our results, on a large life sciences dataset, indi-cate that logistic regression can perform surpris-ingly well, both statistically and computation-ally, when compared with an array of more recent classification algorithms.