How Yelp-Type Ratings Could Be Used for Self Governance

Ofer Tchernichovski is a Zoologist at Hunter College, The City University of New York

Dalton Conley is a sociologist who teaches at Princeton University.

Humans can learn how to solve big problems from tiny creatures.

An individual ant has limited cognitive abilities, but collectively, ants can solve extremely complex tasks. They can sustain colony harmony and productivity across millions of individuals with little or no coercion and without leadership. Ants also have a highly distributed, non-hierarchical communication network, which is partly chemical and partly tactile. They can collectively perceive resources, and be constantly aware of changing colony needs. It seems as if our species is at the opposite extreme: individually we are highly intelligent animals, but collectively we are often stupid. We cannot solve even the most elementary social challenges of our era.

Interestingly, over the last decade, we have developed distributed communication technologies that make us a little bit more like ants. Our online communication networks provide us with collective perception of resources. We can "see" highway traffic via our phone, we can virtually visit 20 different restaurants in a couple of minutes and decide where to eat based on reports from current diners. Many everyday decisions are now guided via vast online networks of sharing and rating systems.

These systems are going beyond private companies—some state and local governments have implemented these networks for services. The question is: Can online communication networks and rating systems evolve into a form of distributed, continuous self-governance?

In a recent study, we gathered a team of sociologists, zoologists, and engineers to study this question, and we found that organizations that are looking for quality information in their online feedback and ratings systems have that potential, but they are going about it in the wrong way.

For example, companies such as Uber and Amazon try to encourage as many consumers as possible to contribute ratings by making it easy (or, in rarer cases, mandatory). While that approach increases the number of ratings, the quality of the information suffers greatly. Think about how often you have lazily just clicked on “five-stars” for an Uber or Lyft driver who, if you really gave it some thought, really didn’t deserve even four. Of course, Uber drivers can be penalized for even a four star review, so some of us give five star reviews simply because we don’t want them to lose their jobs.

Drawing from the ecological theory known as “costly signaling theory,” we argue instead that providing feedback should carry some cost—whether money, or time, or energy—to ensure that cheating (such as negative "review bombing" or pumping up a product's review rating) or lazy rating isn't worth it.

This principle explains, for example, why male peacocks advertise their quality by growing an elaborate tail, why baby birds signal their hunger with energy-intensive begging displays, and even why human job candidates advertise their quality with diplomas from elite schools.

In this vein, we tested the theory that imposing a “cost” to providing information—and more importantly, imposing higher costs on extreme ratings—would eliminate the no-risk, average-skewing "lazy" one-star and five-star ratings and result in more accurate feedback. We created an online virtual world, where participants "worked" to make money by driving their “car” along roads picking up coins that can be redeemed for one cent each. Every so often they hit a lake, whereupon they had to wait for a ferry to take them across before they could continue to hoover up coins again. A slow ferry cost our participants critical seconds in which they could have been earning coins—in this game, time is literally money.

Then, the participants scored their satisfaction with the ferry transportation services using a rating widget (a slider) with a built-in digital “friction." The friction determines the ease of moving the slider away from the initial position at the average rating. This way, we tested the effect of specific time costs (read: money) for reporting extreme scores on information quality.

We found that even a few seconds of cost led to much more reliable crowd estimates of quality. People became more moderate and thoughtful—they gave extreme rating scores only in truly extreme cases. As a result, collectively they became wiser: Their ratings more closely tracked the objective quality of the service that they were receiving.

Classic economic theory—which Yelp and other firms have been following—suggests that minimizing cost and effort in leaving a review would yield the best results. But behavioral ecology theory, and our recent data, suggest the opposite. Making it is too easy for people to report extreme rating scores doesn’t work.

How do the lessons of our experiment fit in with the dream of collective self-governance? Of course we are not advocating imposing costs to all forms of collective decision making. Nobody wants a return to discriminatory poll taxes. We don’t think laws making it difficult to vote make for a healthy society. But periodic elections are not the same as the fast-paced arena of e-commerce and social media. In our game, for example, giving the ferry a moderate rating had negligible time cost, and each participant decided, based on her level of enthusiasm (or frustration) with the service, whether to spend a little extra time to report an extreme event. This way, we were able to set an optimal 'cost' that increased information quality without suppressing voting.

The internet as currently configured is, perhaps, inherently too fast and inexpensive to nurture sustained, healthy collective deliberation. Social media such as Facebook, YouTube, and Twitter have given rise to chaotic, disconnected bubbles of rapid interactions. This has, of course, had some positive outcomes (e.g., the #MeToo movement), but for creating sustainable bottom-up governance, we will need to develop different types of platforms, where information accumulates to reliably guide collective decisions in appropriate time scales. Such information systems could be useful for making wise collective (i.e. bottom up) decisions about investments in pooled resources such as healthcare and infrastructure.