Hierarchical Decision Making in Multi-time Scale Markov Semiconductor Fabs using Decision Processes

Absfmct- There are dimerent timescales of decision making in semiconductor fabs. While decisions on buyingldiscarding of machines are made on the slower timescale, those that deal with capacity allocation and switchover are made on the faster timescale. We formulate this problem along the lines of a recently developed multi-time scale Markov decision pro-cess (MMDP) framework and present numerical experiments wherein we use TD(0) and Q-learning algorithms with linear approximation architecture, and show comparisons of these with the policy iteration algorithm. We show numerical exper-iments under two different scenarios. In the first, transition probabilities are computed and used in the algorithms. In the second, transitions are simulated without explicitly computing the transition probabilities. We observe that TD(0) requires less computation than Q-learning. Moreover algorithms that use simulated transitions requite significantly less computation than their counterparts that compute transition probabilities. Index rem-Semiconductor fab, multi-time scale Markov decision process (MMDP), reinforcement learning, temporal difference (TD(0)) learning, Q-learning. I