Using Approximate Dynamic Programming to Solve the Military Inventory Routing Problem with Direct Delivery

The United States Army uses Vendor Managed Inventory (VMI) replenishment to manage resupply operations while engaged in a combat environment; upper-echelon organizations (e.g., a brigade) maintain situational awareness regarding the inventory of lower-echelon organizations (e.g., battalions and companies). The Army is interested in using a fleet of cargo unmanned aerial vehicles (CUAVs) to perform resupply operations. We formulate an infinite horizon, discrete time stochastic Markov decision process model of the military inventory routing problem with direct delivery, the objective of which is to determine an optimal unmanned tactical airlift policy for the resupply of geographically dispersed brigade combat team elements operating in an austere, Afghanistan-like combat situation. An approximate policy iteration algorithm with Bellman error minimization using instrumental variables is applied to determine near-optimal policies. Within the least-squares temporal differences policy evaluation step, we use a modified version of the Bellman equation that is based on the post-decision state variable. Computational results are obtained for examples based on representative resupply situations experienced by the United States Army in Afghanistan