How endogenous crowd formation undermines the wisdom-of-the-crowd
Research on the wisdom-of-the-crowd has shown that groups of judges can produce surprisingly accurate collective evaluations. Existing research has focused on settings where the crowd is fixed before judgment aggregation. Yet, in many naturally occurring environments crowds form as the result of a dynamic and endogenous process. We analyze the consequences of this process for the quality of collective evaluations. Using computer simulations, an experiment, and analyses of field data, we show that if additional judges are more likely to join the ‘crowd’ when the current collective evaluation is positive, a negative bias in collective evaluations emerges. This bias affects particularly strongly items that were evaluated by few judges. Our results imply that if decision makers rely on average ratings to choose among options, they will make systematic mistakes. We demonstrate the relevance of this endogenous crowd formation process in analyses of datasets comprising more than 80 million online ratings.