An approximate dynamic programming approach for the production control of a random-speed machine

This thesis considers the production control of a production system that has random speeds. The system operator is given T time units to produce D units of a product. There are several production speeds for the operator to choose. Due to random noises in the system the actual production speeds of the system are random variables that depend on nominal speeds chosen. The operator does not have any idea on what the actual speeds are, though he knows the quantity produced at inspection epochs determined by him: at each inspection epoch, upon observing the quantity produced, the operator decides the timing of the next inspection epoch and the (nominal) production speed till the next inspection. There are operating, inspection, inventory holding, shortage, and surplus costs, and the sales revenue. The objective is to determine the optimal inspection epochs and production speeds so as to maximize the expected profit. We use dynamic programming (DP) to solve this problem, i.e., fi(d,t), the expected reward with d units to produce and t time units to go at the ith inspection, is found recursively from fi+1. To overcome the curse of dimensionality, two approximations are used in our DP approach. In the first approximation, we carry out DP in two phases; state aggregation is used in the first phase to round up potentially good decisions, and the rounded up decisions are examined in details in the second phase. In the second approximation, we approximate fi(d,t) by only calculating fi+1 at some chosen grid points. When compared to two existing heuristics, an upper bound, and the DP approach without any approximation, our approach shows promising results. It leads to higher expected profit than the two heuristics; closer to the upper bound; and requires an order of magnitude less computation time than the DP approach without any approximation