Mining Evolving User Profiles in Noisy Web Clickstream Data with a Scalable Immune System Clustering Algorithm

Web usage mining has recently attracted attention as a viable framework for extracting useful access pattern information, such as user profiles, from massive amounts of Web log data for the purpose of Web site personalization and organization. These efforts have relied mainly on clustering or association rule discovery as the enabling data mining technologies. Typically, data mining has to be completely re-applied periodically and offline on newly generated Web server logs in order to keep the discovered knowledge up to date. In addition to difficulty to scale and adapt in the face of large data and continuously evolving patterns, most clustering techniques, such as the majority of KMeans variants, also suffer from one or more of the following limitations: requirement of the specification of the correct number of clusters/profiles in advance, sensitivity to initialization, sensitivity to the presence of noise and outliers in the data, and unsuitability for sparse data sets. Hence, there is a crucial need for scalable, noise insensitive, initialization independent techniques that can continuously discover possibly evolving Web user profiles without any stoppages or reconfigurations