Efficient Integration of Data Mining Techniques in Database Management Systems

In this paper, we propose a new approach for apply-ing data mining techniques, and more particularly super-vised machine learning algorithms, to large databases, in acceptable response times. This goal is achieved by inte-grating these algorithms within a Database Management System.Weare thus only limited by disk capacity, andnot by available main memory. However, the disk accesses that are necessary to scan the database induce long re-sponse times. Hence, we propose an original method to reduce the size of the learning set by building its contin-gency table. The machine learning algorithms are then adapted to operate on this contingency table. In order to validate our approach, we implemented the ID3 decision tree construction method and showed that using the con-tingency table helped us obtaining response times equiva-lent to those of classical, in-memory software