Card counting meets hidden Markov models

The Hidden Markov Model (HMM) is a stochastic process that involves an unobservable Markov Chain and an observable output at each state in the chain. Hidden Markov Models are described by three parameters: A, B, and . A is a matrix that holds the transition probabilities for the unobservable states. B is a matrix that holds the probabilities for the output of an observable event at each unobservable state. Finally,  represents the prior probability of beginning in a particular unobservable state. Three fundamental questions arise with respect to HMM’s. First, given A, B, and , what is the probability a specific observation sequence will be seen? Second, given A, B,  and an observation sequence, what is the most probable sequence of hidden states that produced the output? Finally, given a set of training data, estimate A, B, and . There are a number of tools that have been developed to answer these questions. Woolworth Blackjack is a variation of Blackjack played with a deck consisting of 20 fives and 32 tens. The object is to get a close to 20 as possible without going over. The player using a basic strategy loses to the dealer. The aim of this research is to develop a winning counting strategy for Woolworth Blackjack and then attempt to improve upon the counting strategy with a HMM using well-established HMM analysis tools. A secondary goal is to understand when to use counting strategies and when to use HMM’s.Electrical EngineeringMastersUniversity of New Mexico. Dept. of Electrical and Computer EngineeringJordan, RamiroJayaweera, SudharmanSolomon, Otis Jr