We’ve had an Oculus Rift developer kit since it first came out, and have even managed to pair it with a prototype of a Kickstarted 360-degree video camera, the Sphericam. We wanted to know what it feels like to watch 360-degree video immersively with VR goggles. Does the view change in a natural way when you turn your head? Will this soon become a compelling way to experience another location “by feeling truly present,” as Zuckerberg put it? Are we at the cusp of immersive telepresence?
In a word, No.
360-degree content needs to be taken from a single focal point, to avoid stitching errors from parallax effects. As a result 360-degree video looks oddly flat from within the Oculus Rift. You are inside a sphere, looking at its surface, onto which a video is being played. This can be new and interesting, but it is not compelling. You are not “truly present”, whether or not this is a live feed, or prerecorded.
This is because a 360-degree video feed does not contain any depth perception. Within the Rift, each eye is fed a slightly transposed view of the same image projected onto the inside of the sphere, but that sphere is the same for both eyes.
A 3D view of the real world requires two aligned cameras, one for each eye. In a conventional 3D film, you don’t get to choose your direction of view — it is always the camera’s. To make truly immersive real-world content for a VR goggle, you’d need to be able to provide 360-degree 3D video that mimics your head movements.
It’s clear from these early efforts that innovators face huge challenges as they try to use virtual reality to make telepresence immersive enough to be social.
That’s why quite possibly VR is not the holy grail of social. Let’s count the ways:
1. You can’t video chat with VR goggles on
With AR solutions such as Google Glass you are visible and accessible to others. With today’s VR goggles, you aren’t. Never bet against technology, but I think it is unlikely that there will soon be a device that can give the user a fully immersive experience without also obstructing her face — probably not until we jack the video feed directly into the brain’s visual cortex. This means that VR is geared towards the consumption of social information in an immersive context, not the display of such information, because nobody can see you cry or wink when you’ve got your VR goggles on. You might as well be wearing a burqa. (And so much for “consulting with a doctor face-to-face”, as Zuckerberg puts it.)
2. Virtual worlds can’t do social as well as real worlds
VR is eminently suited for the social exploration of digitally constructed environments — the very first VR games for Oculus Rift were notably multiplayer. Collaborative fantasy worlds like World of Warcraft and Minecraft are good for teaching social skills such as sharing and cooperation, and Second Life allows for much odder consensual efforts.
If VR has a holy grail, it is the holodeck: I cannot wait until VR technology starts getting precise input from position and motion detection tools, so that my goggles can transform my livingroom into a dungeon or spaceship, to be explored via physical movement together with others.
But in all these cases, players are represented by avatars. That’s not a problem when there is questing or building to be done, because these avatars will always act out your deliberate commands — you choose which emotive states to act out. This quirk leads to interesting use cases, but it also narrows down the technology’s usefulness as a mediator of genuine emotive states.
Voice chatting already does a remarkable job of mediating emotions like surprise or amusement, but we’ve had that since the telephone. Your virtual avatar won’t any time soon be mediating your true emotive state; until then, video chatting is far more effective — for business meetings, long-distance dates, and for showing off grandchildren.
3. 360-degree 3D live-action immersive video is very hard
Today, If you want to have a 360-degree 3D immersive video experience, you would need to build two front-facing cameras in a rig that can quickly mimic your head movements with minimal lag, to preserve the parallax effect that lets your brain calculate depth of field. In fact, a Swedish drone manufacturer is tantalizingly close to a proof of concept for this, done from a drone. But in a situation where you want to record 360-degree 3D video for later, or where multiple users will want to explore the same live stream, it is today impossible to simultaneously record stereo video from all possible viewing angles from the same location.
So when Zuckerberg talks about “enjoying a court side seat at a game” in a manner that lets you feel “truly present”, he’s actually asking a lot. Still, for controlled environments such as a basketball stadium, we will conceivably one day be able to blanket the court with hundreds of cameras and then use all these viewpoints to automatically reconstruct a live simulacrum of a game in 3D space, which we can explore at will, immersively. Think Microsoft Photosynth, but live. That would be awesome as sports entertainment. But is it a social innovation?
AR vs. VR:
Notwithstanding the social opprobrium being heaped on Google Glass right now, augmented reality’s focus on layering context onto the real world and providing an always-on communications channel to friends is far more likely to have an impact on our social lives than VR, if only because it has a much greater potential to become ubiquitous. After all, no matter how great VR gets, its unlikely you’ll ever be walking down the street with your VR goggles on any time soon.
Unless, of course, they find a way to fuse the best of both worlds — AR and VR.
/ Posted by Stefan Geens (Employee 2014-2017)