Eating the Uncanny Valley: Inside the Virtual Reality World of Food

When the robots roam the land as our omnipotent overlords, I imagine the only thing that will keep the waning group of ragtag rebels moving forward is the memory of how things once were. But would you be more inclined to stand up to your cybernetic conquistadors if you had the ability to actually experience first-hand what, say, maguro collar looked and tasted like before its memory was obliterated from the world?

A recent virtual reality resurgence could thankfully help you achieve such a watershed moment during humanity's inevitable decline. VR tech and its practitioners are redefining how we think about and interface with food, and one of the myriad applications could be to preserve the taste of extinct or lost foodstuffs for future generations of post-apocalyptic gastronomes. But perhaps even more interesting to those of us who aren't Elon Musk are the strange and intriguing ways in which VR is working with food in the here and now.

Back in the 90s, virtual reality was pitched as an amazing, otherworldly realm transcending human scope, but the sweat-stained and popcorn-scented virtual reality behemoths that littered movie theaters and arcades fell utterly flat. And if you had the good fortune to own your very own Virtual Boy, Nintendo's 1995 answer to the future of gaming? That worked out pretty well for you—if you were the type who enjoyed a monochromatic, crimson tinted hellscape. We all know nothing stimulates the imagination better than vertigo and the possibility of permanent eye damage!

But this time around, things are different; virtual reality is growing out of its awkward adolescence. Facebook's $2 billion acquisition of Oculus VR in 2014 and the strategic partnership between HTC and gaming giant Valve signaled a change in the tide. Sony has also thrown its hat into the ring with a 2016 release date for its Project Morpheus; and Samsung has become the first to have a physical VR device on retail shelves, albeit a device intended for developers. While all of the major players have yet to release retail versions of their headsets for the general public, numerous applications are in the works.

Among these new applications are several targeted at the subset of techies who also happen to love food. To learn more about the emerging intersection between virtual reality and all things culinary, I found myself heading from New York to Boston on the cheap bus to attend Boston Virtual Reality, a monthly developer meetup held at MIT.

iPads mounted to the walls act both as security touchpads and a reminder that tech is king in these hallowed halls. Around 100 students, engineers, tech entrepreneurs, and fans of all things VR showed up for presentations and networking. But the reason I sat on a sweltering bus reeking of Chinese buffet for four hours was to see Matheus DePaula-Santos. He and his Brookline-based Myo Studios are the creators of Perception Fixe, and they are hoping to both change the way we think about food and how we interact with it socially.

Perception Fixe, as DePaula-Santos explained to me, will operate as a hybrid website, which has been billed as the world's first VR food blog. Users without a headset will interact with the food blog through a 3-D model viewer that will be built into the site; those who have a VR headset, however, will have a significantly more mind-melting adventure.

Myo Studios is banking on the notion that providing an enhanced visual experience through virtual reality will markedly up its food blog's ante. Users will be able to "sit down in front of a steak from some restaurant, even though there's no reservation for three months." The VR food blog will also be able to "teach someone to cook from a first-person perspective." DePaula-Santos told me, "One of my hopes is to not just take photographs of food, but also be able to animate it. If you see a sizzling steak in front of you, that's just one way of stimulating more senses."

This is accomplished through the magic of photogrammetry. Myo Studios takes photographs of food and then generates 3-D models from them, thereby creating a sense that the viewer is actually in the presence of food—cooking it, eating it, or otherwise fawning over it.

An audience member at the meetup asks, "What about smell?" Perception Fixe is not quite there yet—it is only a visual virtual reality experience. Yet DePaula-Santos points out the centrality of visuals in our experience of food: "When I was a kid I could have sworn high and low that when I got a box of Froot Loops, there was lemon, orange, lime. I thought I could taste it. But Kellogg has admitted that those fruit-flavored loops aren't 'f-r-u-i-t" they are 'f-r-o-o-t.'"

Perception Fixe is still in development and is far from being ready for public consumption. DePaula-Santos is the first to point out that "the challenge of working with VR is that all of the headsets haven't been finalized." He says that as the technology continues to advance, Perception Fixe will "get to grow with the industry to understand what constitutes 'real' or ethical viewings of the food versus something that is doctored or taking certain advantages."

While DePaula-Santos's work with virtual reality may change the future of food blogs, cookbooks, and food reporting, what about the all-important act of actually consuming food in the VR world? Word of mouth led me to an ongoing project out in LA, one that is taking the leap and aiming to change the very way we consume food through the use of virtual reality.

Project Nourished—created by Kokiri Labs and its founder, the ever-entrepreneurial Jinsoo An—bills itself as a "gastronomical virtual reality experience." This mashup of molecular gastronomy and virtual reality allows users to "experience fine dining without concern for caloric intake of other health-related issues."

An works out of a lab on Skid Row that is packed to the brim with more gizmos than Inspector Gadget's estate sale. "I'm an experienced designer and I consider myself a bit of a researcher, tinkerer, technologist, food enthusiast, nerd, and mad scientist," An told me. His resume documents a career in systems analysis, advertising, and wearables design, with plenty of esoteric awards and speaking engagements.

As for Project Nourished, here's the deal: You put on your VR headset, Oculus or otherwise; you lift your "food detection sensor," which, at this stage of development, looks like nothing more than a wooden fork with two prongs wrapped in tinfoil; you eat a hydrocolloid—a flexible substance that is "viscous, emulsifiable, and low caloric"—that has been shaped in a 3-D printed mold to "add physical characteristics to the 'faux' food." Then, with the help of a motion sensor, an aromatic diffuser, and a bone conduction transducer (more on that later) you experience a gourmet meal with no downside in the way of calories, carbs, or allergy-inducing ingredients.

The first three foods to be replicated in hydrocolloid form? Sushi, steak, and apple pie. An says he would also like to do a virtual reality form of a Krispy Kreme donut one day.

It is also conceivable that Project Nourished could be used to preserve and replicate the characteristics of some of today's best dining experiences for future generations. Imagine your great-great-great grandchildren enjoying the delights of the salmon tartare cornets from The French Laundry, or the flavors of a by-then long-extinct bluefin tuna.

Despite its humble roots as a one-man endeavor, Project Nourished has ballooned into a multi-headed beast with a total of 11 creatives, each with varying backgrounds, including Mike Resendez, the molecular mixologist from The Bazaar's Beverly Hills outpost, and Mark Manguera, the co-founder of LA's Kogi and Chego.

Among these collaborators is a husband-and-wife duo who are no strangers to the DIY mentality: Nguyen and Thi Tran. The Trans are the proprietors of the now-legit but once-illegal Starry Kitchen, an LA restaurant that was originally run out of their apartment. They have jumped on board to help create recipes for Project Nourished.

The Trans explained that Project Nourished works on multiple sensory levels besides just sight. Flavor is key, of course, and is addressed by those hydrocolloids, which recreate the taste and texture of various flavor compounds. Smell is dealt with by an aromatic diffuser "which uses ultrasonic frequency to dissipate various layers of scents near the diner." But arriving at the perfect scent is not easy, as the noncorporeal ghost of Coco Chanel would be happy to tell you. An said, "One of the things that we learned is that smell can have different layers and timing," so each dish will be accompanied by one or more scents.

As hard as it is to get smell right, hitting the mark when it comes to replicating texture or toothsomeness is even more difficult. That's where the bone-conducting transducer, in association with those 3-D printed molds, comes in. The transducer "recreates sound vibrations that travel within your bones when you're chewing." Got that?

An traces his food-centric ambitions to the film Hook, citing that famous scene in which a grown-up Peter Pan (played by Robin Williams) uses his imagination to create an over-the-top meal. But An also has a more serious reason for getting into the virtual reality food business: his own history of severe allergies and his stepfather's diabetes. "To people like us, eating is a chore. And I didn't want it to be like that," he told me. "I'm sure millions of people around the world with any kind of dietary restriction would feel the same way." This is why Project Nourished is being pitched as a possible remedy for anorexia, bulimia, diabetes, heart disease, obesity and allergies.

Although Perception Fixe and Project Nourished still have far to go in making their ideas a reality—virtual as it may be—they are following in the footsteps of generations before them who have used technology to push culinary boundaries. Just as molecular gastronomy has been accepted as a credible fine-dining tool in the kitchen, so too may virtual reality enter the culinary lexicon. Cooking and technology have always gone together, ever since man put flame to food.

And if nothing else, after the apocalypse, we can all happily chow down on our hydrocolloid pellets of extinct maguro collar.