Tantra Analyst

How Photorealistic VR And 5G May Change The Future Of Media

I witnessed an impressive 5G-based artificial reality/virtual reality (AR/VR) demo at this year’s Qualcomm 4G/5G Summit in Hong Kong. Indeed, AR and VR together, referred to as extended reality (XR), is possible today, and 5G’s higher speeds, greater capacity and lower latency will naturally improve the experience. But this demo, referred to as “boundless XR,” clearly showed how transformational 5G could be and gave further credence to the “wireless edge” concept that I discussed in my previous article.

(Full disclosure: Qualcomm is a client of my company, Tantra Analyst.)

Today’s XR, which I consider second generation, can be divided into two categories: smartphone-based and console-based. The former is the easiest and the most convenient way to experience XR — and it’s mobile. The latter offers an immersive, realistic and uncompromised experience — but with limited mobility because of cables. Each form has its own fan base. But why not combine both and have the best of both worlds?

Typically, console-based XR needs a beefy gaming-quality computer that processes all the content, prepares the “views” (this process is called rendering) and sends them to the headset. The headset displays these “views” on VR glasses. As evident, you need an extremely high-speed and low-latency link between the console and the headset. In today’s setups, this is achieved by HDMI or USB cables. However, cables restrict movement — and can even cause a user to fall — destroying the complete immersive experience. Moreover, aesthetically, cables dangling all over make the headset look more like a science project than a sleek consumer device.

That’s why 5G’s excellent performance makes it an ideal replacement for the cables. While 4G and Wi-Fi can accomplish this for today’s XR, it is evolving at a tremendous pace. By the time 5G arrives, VR will be in the next phase, what I call “version 3.0” and what Qualcomm calls “photorealistic VR.”

The evolution will not only improve performance but also change the architecture of how XR experience is created and delivered.

What is photorealistic XR with 5G, and what does it mean for marketers?

The best way to illustrate photorealistic XR is to consider a highly interactive multiplayer game, such as a first-person shooter game. In such a game, your 360-degree views must not only be perfectly synchronized with your movements but also with your opponent’s movements. If you quickly turn or even shake your head, all the views should move accordingly, without any lag. That’s what photorealistic VR is.

Simply put, it’s experiencing VR as if you were right there, living the moment, not just watching as a passive onlooker. To make it happen, the rendering of visuals must be so realistic that they are indistinguishable from reality. This might seem trivial, but if you consider all the details in the views, such as light shades, reflections and everything else that needs to be perfect, it is quite a monumental task.

Two things are extremely critical to delivering photorealistic XR: tremendous processing power and extremely low latency. There is a maximum of 20 milliseconds of allowable delay, called motion-to-photon latency, between the actual movement and the rendering of the view. Anything beyond that will make users dizzy and destroy immersiveness.

The best option for XR is to do all processing in the headset. However, for that, a user has to carry a huge computer, a large battery and a massive heatsink on their head, which is obviously impractical. So, the optimal approach is to keep enough processing in the headset and offload everything else, including rendering, to the console or cloud, so that headsets are sleek and attractive, not heavy and hideous. The link between the two sides becomes extremely important. Making it wireless takes the challenge a notch higher — a perfect recipe for 5G.

Why 5G?

If you look at the trend, content is going online, and XR is no exception. There might still be some things on the console, but for XR to be mainstream, its content will have to be in the cloud and accessed through wireless — 5G makes that a realistic possibility. Its low latency liberates the content from the console and allows it to reside in the cloud. But mind you, it has to be close enough (edge cloud) so that the latency is still within the budget. Although latency is the lynchpin, 5G’s sustained high data speeds are also essential to create the photorealistic XR experience. Indeed, 5G’s high data capacity will enable such XR experiences to scale to millions of users and make it mainstream.

The imperative for marketers is to start considering XR as another marketing medium. The immersiveness of XR imparts unparalleled abilities for marketers to engage and interact with their customers. This means not only that they should include XR as part of their marketing campaigns, but also that they should rethink and retool their strategies.

What does ‘wireless edge’ have to do with all this?

The notion of wireless edge is to move the action — in this case, processing, intelligence and value — toward the edge of the wireless network. For photorealistic XR, the wireless edge consists of the device itself, as well as the edge cloud servers that host the content and rendering. The edge cloud also improves the overall network efficiency and has the content available close to the users where and when they need it.

You might ask: why bother moving XR content to the cloud at all? First, to make XR mobile and cost-effective (consoles are expensive). Second, when you have a silver bullet such as 5G available, why not use it to scale VR and make it mainstream? Additionally, photorealistic XR applications give real opportunity to cellular operators to compete. Such services also give more options to operators to monetize their investments in 5G. But they must be quick to capitalize on their early advantage so they do not meet the same fate as they did with 3G and 4G services. It indeed will be interesting to watch how the future of XR unfolds with 5G.