How Facebook sees the future of social virtual reality

Facebook bought VR hardware maker Oculus in 2012 amid jokes of VR newsfeeds. Now, the company is talking about what social interactions in VR could really be like.

Image: Facebook

A long-standing joke (or concern, depending who you talk to) about virtual reality is that the technology will further isolate people from each other as they choose to spend time in a fantastic virtual world instead of interacting with others in the real one.

Facebook doesn't see it that way. In fact, Oculus' chief scientist Yaser Sheikh called virtual reality the ultimate social platform during his talk at the day two keynote for Facebook's F8 developer conference.

At the start of the keynote, Facebook CTO Mike Schroepfer showed off a demo of a social VR experience in which he put on an Oculus Rift headset and talked with fellow Facebooker Mike Booth, who was back at the office in Menlo Park, California. They interacted with each other in real time using the Oculus Touch controllers, and virtually stepped into several 360 photos, including one of Big Ben in London.

The moment that got the buzz was when Booth and Schroepfer's avatar posed for a selfie in front of the British monument together and then Schroepfer was able to post the picture to his wall.

SEE: Convrge signals the future of social in virtual reality

The big idea the demo teased could be summarized by Sheikh: "Proximity would no longer determine who you spend your time with."

That can affect a variety of situations from hanging out with distant college friends, to meeting with a remote team, or attending a business meeting. People could meet with each other across the country and across the world everyday with a sense of actual presence.

However, Sheikh, who is also an associate professor at Carnegie Mellon, said there are three main problems that have to be solved, and they all work hand in hand.

First, there's the problem of capturing all the subtleties and nuances of human expression in both the face and the body. There are myriad, nameless clues that the human brain processes in order to make sense of others -- it's how we might know someone is lying or uncomfortable in a split second.

There's not a VR rig that exists that can capture all of that, and because VR must be convenient and affordable in order to survive, the answer can't be asking folks to strap on motion capture suits or something of that like.

The second obstacle is display -- how does that info get communicated to someone on the other side of a social VR experience?

And third, there's the matter of prediction. This works two ways. One, the human brain makes predictions about things all the time. For example, Sheikh showed a picture of two people's frames represented by dots walking toward each other, having just joined hands. The brain knows that the next likely movement is a handshake. A similar process must happen in social VR, partly because there will be some amount of lag in the delivery of this quantity of information back and forth between people in a social VR experience. So, Sheikh said there needs to be a computation understanding of this "elaborate code" of human behavior and interaction.

At Carnegie Mellon, Sheikh has something called the Panoptic Dome, which is a dome covered in cameras that helps them study the nuances of interaction.

"The point of this is obviously not to install a crazy dome in everyone's living room," he said. Rather, the hope is that one day there's a way to combine those three elements and close the distance from one person to another.

Also see