A trader wishes to execute a given number of shares of an illiquid asset. Since the asset price also depends on the trading behaviour, the trader main aim is to find the execution strategy that minimizes the related expected costs. We solve this problem in a discrete time framework, by modeling the asset price dynamic as an arithmetic random walk with drift and volatility both modeled as Markov stochastic processes. The market impact is assumed to follow a Markov process. We found the unique execution strategy minimizing the implementation shortfall when short selling is allowed. This optimal strategy is given as solution of a forward-backward system of stochastic equations depending on conditional expectations of future values of model par...