Evaluation of Collecting Reviews in Centralized Online Reputation Systems

Abstract

Background: Centralized Online Reputation Systems (ORS) have been widely used by internet companies. They collect users’ opinions on products, transactions and events as reputation information then aggregate and publish the information to the public. Aim: Studies of reputation systems evaluation to date have tended to focus on isolated systems or their aggregating algorithms only. This paper proposes an evaluation mechanism to measure different reputation systems in the same context. Method: Reputation systems naturally have differing interfaces, and track different aspects of user behavior, however, from information system perspective, they all share five underlying components: Input, Processing Storage, Output and Feedback Loop. Therefore, reputation systems can be divided into these five components and measured by their properties respectively Results: The paper concentrates on the evaluation of Input and develops a set of simple formulas to represent the cost of reputation information collection. This is then applied to three different sites and the resulting analysis shows the pros and cons of the differing approaches of each of these sites.