Marvel’s What If on Vision Pro Is a Free Taste of the Future

The thrill is undeniable: The Marvel fanfare plays, but this time the logo is all around me. With the Apple Vision Pro headset on, I feel like I’ve stepped into the MCU.

In the three months since the Vision Pro’s release, it’s been hard to find amazing experiences outside of movies and cinematic 3D video. The arrival of Marvel’s What If…? — An Immersive Story feels momentous for that reason. It’s a unique experience — half-game, half-film — and just the sort that the Vision Pro desperately needs more of.

This What If…? experience is a taste of a possible future. Like a shard of the multiverse, it glimmers and astonishes. But it’s gone too soon. The free hour-long story (free as long as you own a $3,499 Vision Pro headset, that is), is broken down into roughly a dozen interactive chapters. It feels like a very special episode of What If…? that you’ve suddenly stepped into. You are, in fact, the central character.

The What If…? VR experience is the latest from ILM Immersive, formerly ILMxLab. The company created a number of VR experiences I’ve loved in the past: Vader Immortal and Star Wars: Tales From The Galaxy’s Edge on the Quest, and the now-gone location-based VR experiences Disney collaborated on with The Void. All three were almost theme park-like ways to step into Disney experiences. What If…? continues that exploration, but it feels like it’s jumping through the fourth wall of mixed reality even more than before.

Those other Star Wars and Marvel VR adventures put me in other characters someplace else. The fun part of What If…? is that it jumped into my home, with portals (and a tall Watcher, along with Wong) appearing and looking right at me.

The magic tricks of the whole experience include eye tracking, which makes all the characters actually look at you no matter where you are glancing or moving, and hand tracking to cast spells like a sorcerer in mid-air. They’re my actual hands, too: the Vision Pro’s excellent hand and arm occlusion overlays my hands into the world like I’ve materialized there. I raise my hands, and I conjure shields and collect Infinity Stones.

Instead of the sensation I had in Vader Immortal or Tales from the Galaxy’s Edge, which felt like explorations of spaces, What If feels like theater. I’m standing among characters, watching stories play out. Sometimes I interact with them, too, using my hands. 

These hand tracking moments are hit and miss. Some feel amazing, like when I lift my arm and create a glowing shield of energy beams. Glowing hand ghosts often appear to guide me to what to do next, and by following them, I create portals and reverse time and fire bolts. My choices aren’t infinite, and mostly I’m playing along in a narrow storyline, like I’m an invited guest in scenes of immersive theater.

The really fun ending, which I won’t share here, clearly leads to the rest of the MCU and makes me wonder how an experience like this could really start to feel like a connected part of TV, films, and even theme parks. For Disney and Marvel and ILM Immersive, it’s clearly a first stepping-stone for more.

David Bushore, director and executive producer of Marvel Studios, and Ian Bowie, experience design director of ILM Immersive, admit that this new free app is also a way of starting to figure out what can be done in newer mixed-reality headsets with hand tracking. Bushore worked on Marvel’s Disney theme park attractions at Walt Disney Imagineering, and helped create an AR iOS experience based on The Eternals that felt, a few years ago, like hints of how AR characters might someday appear in your home in a headset.

“We can start thinking about how things fit in your home in an actual present way. And now that opens up storytelling potential of things down the line that we’re all very excited about,” Bowie said to me in a Zoom conversation. “And this is just the first step in that direction. We have many, many ideas of how to take it to like further levels and be able to kind of expand all that. But hitting the high notes of it of being able to just think about your space as a location, something that other characters can come into, is really cool.”

I asked about when these experiences might even overlap more with theme parks, something I also asked the Tales of the Galaxy’s Edge team a few years ago. 

“The idea of story living really is where we’re at now. We’re starting to crossover into that tipping point — we’re not there yet. But when I go to the park, I have an experience when I watch a movie I have an experience,” Bushore said. “And those are where our memories are made. This type of technology and immersion is just a different way to make memories.”

But Bowie and Bushore also admit, despite years of experience working on VR and AR previously, that the landscape now sort of feels new all over again. This may lead to new types of apps on Vision Pro, and on other mixed-reality headsets to come. 

“There’s definitely an education that everyone’s going through right now. A lot of the cool opportunities the [Vision Pro] gives, especially for us in the storytelling realm, of being able to have characters looking straight in the eye when they’re talking to you, be able to use your hands and make magic with your hands, be able to move between like your actual environment and artificial environments, all of these were kind of the core features that we’ve been thinking about and working with in some capacities, just waiting for the threads to tie a lot of these things together in one product for a while,” says Bowie.

I’m curious about what’s next: Will there be another episode of this immersive series? (The ending suggests so.) Will experiences like this arrive on Meta and maybe Google and Samsung’s headsets? And will mixed reality apps finally find a language and interface that feels unique from VR? Apple’s WWDC conference, where the next version of Vision Pro’s OS may be announced, could introduce even more ideas into the mix. In that sense, this Marvel app only feels like an early start.