This paper studies the evolution of both characteristics of reciprocity - the willingness to reward friendly behavior and the willingness to punish hostile behavior. Firstly, preferences for rewarding as well as preferences for punishing can survive evolution provided individuals interact within separated groups. This holds even with randomly formed groups and even when individual preferences are unobservable. Secondly, preferences for rewarding survive only in coexistence with self-interested preferences. But preferences for punishing tend either to vanish or to dominate the population entirely. Finally, the evolution of preferences for rewarding and the evolution of preferences for punishing influence each other decisively. The existence ...