Learning dynamic prices in multiseller electronic retail markets with price sensitive customers, stochastic demands, and inventory replenishments

Abstract—In this paper, we use reinforcement learning (RL) as a tool to study price dynamics in an electronic retail market con-sisting of two competing sellers, and price sensitive and lead time sensitive customers. Sellers, offering identical products, compete on price to satisfy stochastically arriving demands (customers), and follow standard inventory control and replenishment policies to manage their inventories. In such a generalized setting, RL tech-niques have not previously been applied. We consider two repre-sentative cases: 1) no information case, were none of the sellers has any information about customer queue levels, inventory levels, or prices at the competitors; and 2) partial information case, where every seller has information about the customer queue levels and inventory levels of the competitors. Sellers employ automated pric-ing agents, or pricebots, which use RL-based pricing algorithms to reset the prices at random intervals based on factors such as num-ber of back orders, inventory levels, and replenishment lead times, with the objective of maximizing discounted cumulative profit. In the no information case, we show that a seller who uses Q-learning outperforms a seller who uses derivative following (DF). In the par-tial information case, we model the problem as a Markovian game and use actor-critic based RL to learn dynamic prices. We believe our approach to solving these problems is a new and promising way of setting dynamic prices in multiseller environments with stochastic demands, price sensitive customers, and inventory re-plenishments. Index Terms—Dynamic pricing, inventory replenishments, markovian game, multi-agent learning, online retail markets, price sensitive customers, reinforcement learning (RL), stochastic de-mands. I