We’re spending about a quarter of our waking hours on our mobile devices, with about half of that time dedicated to social media. Facebook, Instagram, Slack, Fortnite … our digital experiences are becoming more immersive, and they’re capturing more of our attention and emotions.

Enthusiasts have been working for decades to blur the boundaries between our digital and physical lives - now we have VR headsets in our homes, smart glasses, Pokemon Go, and selfie filters.

Journalist/novelist Keith Stuart wrote on Medium: “As a child of the late 1970s and early 1980s, it hit me a few weeks ago that Fortnite feels like a skatepark. Or if you prefer, a drag strip. Or a surfing beach. Or a roller disco. It has a central function that draws people in, but more important, it provides a safe place to hang out, experiment, and mess around. To be free.”

The term we use for this broad span of technologies that meld the real and the virtual is extended reality (XR). It encompasses the whole spectrum of experience from the wholly digital (virtual reality, or VR) to digital overlays on the real world (augmented reality, or AR). In the middle is mixed reality (MR), where the user can interact with virtual objects as if they were in the real world; there’s also the possibility of actual connection between these virtual and real things via the Internet of Things.

These technologies are maturing rapidly – we’re starting to see calls for standardization, Microsoft is building an entire HoloLens ecosystem to bring XR to the front-line worker, and VR arcades are opening around the world. The fully blended world seems like it’s just over the horizon, but for XR to truly hit its inflection point and become ubiquitous, I believe at least four things need to happen.

1. The tech needs to get out of the way. Standing in line at the coffee shop, what’s the first thing we do? Pull out the phone, open Facebook, do a quick scan. That always-on and easy access turned checking social media first into a habit, then into an addiction. Right now, the technological interfaces for XR are creating enough friction to keep out many people. Those that do make it in can tire depending on how comfortable the interface is; for example, it’s untenable to hold a phone up to eye level for long, VR in particular can induce nausea, and headsets are not terribly compatible for glasses-wearers, although companies like North are making glasses that correct vision and create XR experiences.


2. XR needs to support genuine human-to-human interaction. “Twitter friends” are now just friends. With the notable exception of VR, XR is a one-to-one experience – it’s very difficult to share with a friend, dependent as it is on individual display ports and inputs. Humans are hard-wired for social interaction; once XR truly supports connection and sharing, more people will be willing to overcome the technological barriers to access it. It will also need to do this 10x better than what we use now. Given the current state of Twitter and Facebook, that may be a lower bar than it first appears. As mentioned above, VR is actually pretty good at doing this right now but it requires complicated technology and a safe space in which to participate, so for the foreseeable future it will likely remain more of a specialized experience.


3. XR needs to democratize self-expression. It’s no surprise that the some of the first and most widely-adopted uses of AR are selfie filters. How many iPhone Xs are sold simply so people can create their own Animoji? We flock to social media and other tools that help us express who we are, where we can share pictures of what’s important in our lives (even if that’s just our lunch), where we can use character limits to inspire works of fiction or political movements. Right now, content that works in XR is has to “feel real” in some way, so it’s complicated and expensive to produce. We’re starting to see creative ways people are addressing this; note this entrepreneur who built a collaborative space to make music. Algorithms are being written that convert 2D imagery to 3D, and Apple is rumored to be releasing a 3D-capture camera in its next iPad; the Structure sensor from Occipital quickly scans real world objects and converts them into 3D images. Once we have accessible and easy-to-use tools to craft our own unique content, the sky’s the limit as to what experiences we’ll build and interact with.


4. It needs to be done responsibly. Blending virtual and physical worlds has implications for the nature of our data, our privacy, even the basic concept of “ownership” (is it still considered vandalism if the graffiti on a building is only visible through a device?). Immersive experiences can help develop empathy, but what are the long-term effects of interacting in virtual versus physical space? We don’t know, and movement forward should encompass taking a long hard look at what we learn along the way.

It’s easy to envision a fully blended world, where our digital creations are just as real as the world around us. And XR is heading there. But first, the technology needs to get out of its own way so that it is a means to an end – a way for us to more fully connect and express ourselves as humans – rather than a stumbling block. There is hope; in the World Economic Forum’s latest report on the top emerging technologies, they highlight advancements in optics that will lead to much smaller camera units, as well as the potential for collaborative telepresence. Based on the enthusiasm, creativity, and passion of the folks working in XR today, I’m confident we’ll get there eventually. But we also need to learn from Twitter and Facebook and figure out how to do it mindfully so that we’re not creating more social ill than social good.