We propose a decentralized method to perform mutual localization in multi-robot systems using anonymous relative measurements, i.e. measurements that do not include the identity of the measured robot. This is a challenging and practically relevant operating scenario that has received little attention in the literature. Our mutual localization algorithm includes two main components: a probabilistic multiple registration stage, which provides all data associations that are consistent with the relative robot measurements and the current belief, and a dynamic filtering stage, which incorporates odometric data into the estimation process. The design of the proposed method proceeds from a detailed formal analysis of the implications of anonymity ...