Scientists are Teaching Robots to Write Surprisingly Good Yelp Reviews

In the past few months, scientists have done some incredibly entertaining things with neural networks, training artificial intelligence systems to identify food in photos (although recognizing a Big Mac is still beyond its virtual brain capacity) and to name craft beers. But while one computer system was ready to raise a glass of just-christened Heart Compost, another was being taught how to write surprisingly believable Yelp reviews.

Ben Zhao and a team of researchers from the University of Chicago have written a paper explaining how they used existing restaurant reviews to teach a deep learning language model called a Recurrent Neural Network (RNN) how to write its own reviews, and illustrating how many people could not recognize those reviews as fake. Even more surprising—or unsurprising, depending on your level of cynicism about the internet—was that the AI-generated sentences "[provided] the same level of user-perceived 'usefulness'" as those written by real people.

They taught the RNN language model using a dataset of more than 4 million Yelp reviews that had been written by one million different reviewers, covering 144,000 restaurants in 11 cities and four countries. (So it was pretty large in scope, is what we're saying).

After the RNN was equipped and ready to automate its own reviews, the study participants were asked to mark the reviews as being either real or fake, and to rate the usefulness of the review on a scale from 1 to 5. They learned that the RNN-generated reviews were "effectively indistinguishable" from authentic ones, and their average usefulness score was 3.15 – which is close to the 3.28 usefulness average for real reviews.

Before you roll your eyes, how would you do? Would you have called out this review as being bogus:

Or how about this one?

The truth is, they're more coherent than the comments sections on most websites, and the syntax is no more or less complex than, say, the President's Twitter account. That, says Zhao, is the problem.

READ MORE: This is What Happens When You Let Robots Name Your Beer

"We're starting with online reviews. Can you trust what so-and-so said about a restaurant or product?" Zhao told Business Insider. "It is going to progress to greater attacks, where entire articles written on a blog may be completely autonomously generated along some theme by a robot, and then you really have to think about where does information come from, how can you verify […] that I think is going to be a much bigger challenge for all of us in the years ahead." No, that's not terrifying at all.

"We appreciate this study shining a spotlight on the large challenge review sites like Yelp face in protecting the integrity of our content, as attempts to game the system are continuing to evolve and get ever more sophisticated," Yelp spokesperson Rachel Youngblade told MUNCHIES. "Yelp has had systems in place to protect our content for more than a decade, but this is why we continue to iterate those systems to catch not only fake reviews, but also biased and unhelpful content […] We encourage the authors to continue research on this important topic so consumers can continue to rely on review site content."

In the meantime, I promise that this post was written by an actual human, one who will constantly be looking over her shoulder to see if the RNNs are gaining on her.