Optimal control of a large dam using time-inhomogeneous Markov chains with an application to flood control

We consider a decision making framework for a dam modelled by a controllable time-inhomogeneous Markov chain with a compound Poisson process as an input, which approximates the inflow pattern of real dams with seasonal rainfall. The controls are responsible for output flow explicitly and implicitly via a price on water consumption. We consider the problem with system balance and demand shortfall performance criteria under constraints. The state space of the dam is discretized while the time domain remains continuous. We also add a controllable water release to reduce the probability of the dam overflowing and give the procedure for determining if an optimal solution exists by showing the consistency of constraints