The role of algorithms in profiling

Algorithms can be utilised to play two essential roles in the data mining endeavour. Firstly, in the form of procedures, they may determine how profiling is conducted by controlling the profiling process itself. For example, methodologies such as CRISP-DM have been designed to control the process of extracting information from the large quantities of data that have become readily available in our modern, data rich society. In this situation, algorithms can be tuned to assist in the capture, verification and validation of data, as discussed in the reply to Chapter 3. Secondly, algorithms, dominantly as mathematical procedures, can be used as the profiling engine to identify trends, relationships and hidden patterns in disparate groups of data. The use of algorithms in this way often means that more effective profiles can ultimately be computed than would be possible manually. In this chapter we show how algorithms find a natural home at the very heart of the profiling process and how such machine learning can actually be used to address the task of knowledge discovery.