Towards a Comprehensive Testbed to Evaluate the Robustness of Reputation Systems against Unfair Rating Attacks

Abstract. Evaluation of the effectiveness and robustness of reputation systems is important for the trust research community. However, existing testbeds are mainly simulation based and not flexible to perform robust-ness evaluation, and none of them is specifically designed to evaluate the robustness of reputation systems against unfair rating attacks. In this pa-per, we propose a novel comprehensive testbed by simulating three types of environments (simulated environments, real environments with simu-lated unfair rating attacks, and real environments with detected unfair ratings). The testbed incorporates sophisticated deception models and unfair rating attack models, and introduces several performance metrics to fully test and compare the effectiveness and robustness of different reputation systems. We also provide two case studies to demonstrate the usage of partial features of our proposed testbed