Essays on Developing and Testing New Machine Learning Models for Dynamic Market Segmentation and Personalized Product Recommendation

This research comprises two essays on the development and application of machine learning models for marketing problems such as segmentation and personalized product recommendation. Prior market segmentation research has primarily focused on deriving customer segments based on demographics or brand choice data rather than on purchase sequences and attitudes. The first essay proposes a new machine learning approach that uses a mixture of hidden Markov models (MHMM) to cluster customers based on their purchase sequences of multiple items and predicts their cluster membership probabilities based on more than 200 attitude and demographic variables through a Lasso regression formulation. It shows the approach by estimating the models on a uniquely compiled dataset of transaction and attitudinal data in the salty snacks category from a large supermarket chain. Accurate prediction of customers’ next purchases is increasingly becoming important to retailers and manufacturers. Prediction of next purchases is a dynamic big data problem as it involves a large number of products and a huge base of customers, whose preferences may change over time. Current purchase prediction do not scale well and while machine learning topic models such as probabilistic latent semantic analysis (PLSA) and Latent Dirichlet Allocation (LDA) model show promise, but they do not account for dynamics in customer preference. The second essay combines topic modeling and dynamic purchase modeling to propose a new machine learning approach, termed the Topic Hidden Markov Model (THMM) that is based on consumer preference theory. The model has several unique features that distinguishes it from alternative models. First, the model can explicitly find latent segments of customers based on common preferences or motivations. Second, it captures heterogeneity by identifying idiosyncratic preferences. Third, the model incorporates dynamic patterns in customer preferences. Finally, it can learn correlated purchase patterns that exist in transactions of multiple items that are common in categories such as salty snacks and cereals. Using data from a supermarket retailer, the essay estimates the model, validates it, and benchmarks its product recommendations against those of alternative models. The model validation tests demonstrate the superior prediction accuracy of the new model and offer important implications for the theory and practice of marketing