We study bargaining models in discrete time with a finite number of players, stochastic selection of the proposing player, endogenously determined sets and orders of responders, and a finite set of feasible alternatives. The standard optimality conditions and system of recursive equations may not be sufficient for the existence of a subgame perfect equilibrium in stationary strategies (SSPE) in case of costless delay. We present a characterization of SSPE that is valid for both costly and costless delay. We address the relationship between an SSPE under costless delay and the limit of SSPEs under vanishing costly delay. An SSPE always exists when delay is costly, but not necessarily so under costless delay, even when mixed strategies are al...