In 2014, Mark Zuckerberg acquired Oculus VR for billions of dollars on the premise that virtual reality would become the foundation of personal computing.
In 2026, virtual reality is really starting to take root on these foundations with OpenXR and Flatpaks. Operating systems based on VR headsets with eye tracking as a key feature are currently receiving updates from Google, Valve, and Apple.
New walled gardens are rapidly being built while old walled gardens are falling down. Valve is coming for gaming, Google is relying on Android APKs, and Apple is building new kinds of live sports and TV experiences, all using VR as a display for the entire landscape.
Hundreds of people lost their jobs earlier this week as Meta announced its most dramatic strategic change to date. The future of VR has never been brighter, but given the weight of Meta’s shift, some may believe that “Oculus VR” here is a “legendary misfortune” and that virtual reality is dead again.
That couldn’t be further from reality. If you care about the future, you were probably reading UploadVR yesterday.
As I look back over the past decade and try to piece together how Meta got here, I notice one key piece of technology is noticeably missing from nearly every headset and glasses design, with the exception of the failed Quest Pro.
Here’s a look at why the lack of eye tracking limited VR’s scale and why Zuckerberg’s ambitions for a new social network clouded Oculus’ vision.
Eye tracking in 2017
In 2017, I attended two eye-tracking demos at GDC, one of which was inside Valve’s booth. From these demos, I began to realize how powerful eye tracking can be for VR software designers.
“Additional information (eye tracking) provides creators with the ability to create games that are fundamentally different from the current generation,” I wrote. “It was like suddenly being given a superpower, and it was fun, so I naturally started using it. If the game knows exactly what it’s interested in at that moment, it’s up to the designer to decide how much skill is required to accomplish a particular task.”
Designing an entire VR platform over a decade without a solid plan for the default implementation of eye tracking is a study in balancing short-term execution with long-term vision.
“Apple’s eye tracking is really cool,” Zuckerberg said on Instagram in 2024, after saying he tried Apple’s headset. “These sensors were actually put back in the Quest Pro. We removed them for the Quest 3, but we plan to bring them back in the future.”
Mark Zuckerberg: Quest 3 is better than Apple Vision Pro
Mark Zuckerberg tried Apple Vision Pro and claims the Quest 3 is “suitable for most things people use mixed reality for.”
This small comment could be seen as his first public admission that he is beginning to see something seriously wrong with his current strategy.
When Facebook discontinued sales of the Oculus Go, the company confirmed it would no longer make 3DoF headsets. The same thing was supposed to happen after the Quest Pro launches in 2022 with eye-tracking capabilities. It wasn’t. By the time Meta ships its second VR headset with eye tracking, it will have been roughly five years since the first one. After Apple first made headlines with its eye-tracking-enabled FaceTime and Persona, perhaps the company is going all in on offering full-body codec avatars as prizes to immerse users in its vision of the Metaverse.
I think we now have proof that VR headsets that don’t allow you to see what you want to see according to your eye’s intent are not serious platform competitors. Valve, Google, and Apple all value modern headset technology for slightly different reasons. If you go back far enough, you’ll see that Steam Frame’s DK1 is 2016’s HTC Vive, and Valve Index’s is 2019’s DK2.
Following Apple’s decision to release Vision Pro in 2024, Valve has decided that SteamOS for VR will be ready for primetime in 2026 with a consumer release in Steam Frame. Both use eye tracking to do things that are important to you.
From the lift to the quest
For Zuckerberg’s organization, increased investment over the past decade has built the technology needed for a complete computing platform, starting with just a few billion dollars to acquire the Oculus Rift development team. Michael Abrash left Valve and decided to invest in expensive long-term research and development to create a modern Xerox PARC within Zuckerberg’s larger organization.
Meta has built these technologies in a fairly public manner by showing the progress of its work, both in sharing its research and selling its products. During this early period, solid ideas like Oculus Medium were created and continued by Adobe and others.
Starting in 2020, Facebook tried to forcibly link accounts to use the Quest headset, and in early 2021 tried advertising in virtual reality. VR users quickly rejected both efforts.
Facebook executives launched a rebranding effort to Meta in parallel with a new account system developed as a fresh start for Zuckerberg’s new computing platform in headsets and glasses. By the end of 2021, Facebook has become meta.
Quest 2 sold well. Although there were well-curated stores and hand tracking was rapidly approaching the cutting edge, there was no reliable competitor shipping standalone VR headsets in the United States. All the stink associated with Facebook was being pushed into the meta background by Zuckerberg’s bold new vision of the Metaverse.
And the high-end Quest Pro with eye tracking was still scheduled to arrive in late 2022.
A legendary misfortune?
“Starting to build a metaverse is actually not the best way to ultimately complete it,” technical guide John Carmack warned in 2021. “The Metaverse is a honeypot trap for architectural astronauts… Mark Zuckerberg decided now is the time to build the Metaverse… What I worry about is that we could probably spend years and thousands of people and end up with something that doesn’t really contribute much to the way people actually use devices and hardware today… We need to focus on the actual product,” rather than the technology, architecture, and initiative. ”
Carmack left Meta in 2022, saying he was “tired of fighting,” and four years later, thousands of people left Meta as his leadership reorganized the company in the form of VR and AR technology. Until the layoffs occurred in 2026, Meta’s leadership and design failures did not reek of the kind of failure Carmack specifically warned about. Now we do.
Will Beat Saber and Population: One be the last bastion to keep Horizon Worlds alive after Meta fired most of the game developers it hired and left the remaining developers to work on Horizon? Meta’s latest move in December, when some of the first Steam Frame kits arrived with the developers, brought Population: One back to Steam. It pointed out that this is a move to stop “unfair play” by cheaters who take advantage of the openness of PC to destroy the multiplayer experience.
The legendary misfortune here was the whole Horizon Worlds effort, which tried to brute force the wrong technology at the wrong time in the wrong way onto social networks. 2026 certainly represents a reset for Meta’s efforts, but the question is: exactly how far back in this timeline does Meta need to go to figure out what went wrong, and what structural changes need to be made to fix it?
Game studio instead of eye tracking
Meta acquired Beat Saber in November 2019 and over the next few years doubled down, hiring dozens of developers skilled in using Oculus Touch controllers. Some of these decisions were made based on unusual behavioral patterns as a generationally significant pandemic keeps people at home near their headsets.
From 2016 to 2018, NextVR livestreamed NBA games to VR headsets, a startup called Spaces opened a walk-around Terminator VR attraction, and decent eye tracking was demonstrated in consumer VR hardware for the first time. Apple released a headset in its 2024 product that combines all the technologies mentioned from 2017.
At Meta, after shipping a single headset with eye tracking, someone made the decision to ship a headset without eye tracking. They all have their reasons, but whatever they are may be the reason Meta lost some of its lead in VR that it acquired with Oculus in 2014. Whatever was going on with Meta’s decision-making process, management sought to rectify it by the end of 2025 by hiring key executives from Apple.
Meta now faces a world where it may increase production of non-VR glasses products. Meanwhile, Apple, Google, Samsung, and Valve are shipping or plan to ship VR headsets with eye-tracking capabilities.
From real to virtual and from virtual to real
Thirty years after this graph appeared in the research literature until Apple created a dial to move the visible across the real-virtual continuum.
From Paul Milgram’s seminal 1994 paper, imagine two types of eyewear at opposite ends of this particular continuum. The one on the right is a relatively heavy VR headset that is basically all display. The other on the left is a pair of ultralight frames without a display. Currently, Meta ships Quest 3, 3S, and Ray-Ban glasses in each of these categories, all of which lack eye tracking.
Currently, Apple only ships the Vision Pro with eye tracking, but it’s a $3,500 device and not many people have tried it. The headset performs a little magic trick on this graph. This is routed to the right edge of the chart, but by default the software starts from the left. By turning a dial on a headset, the world can go from an environment that is completely “real” to completely “virtual” across the continuum.
Apple is certainly up to something to secure the left side of the chart. What features will you focus on at launch and how will Apple and Meta eyewear differ?
point the camera in the wrong direction
If the Vision Pro is a spatial computer, I wish Apple’s answer to the Meta Ray-Ban glasses was something that worked more like a spatial mouse. There is no display, everything is entered.
Apple could take the sensors that track hand and eye movements from the Vision Pro and put that technology into a slim frame with Bluetooth and a battery. The thin clear glass can collect the same eye and finger input as a larger, closed VR headset. It’s certainly difficult, but it’s more convenient than introducing a display system in one eye. The differentiating feature is a universal remote control for everything that is so advanced that it can feel magical almost anywhere.
The same way you navigate the Vision Pro’s menus, you should be able to navigate your iPad, Apple TV, and even other Apple devices. Just pinch and drag outdoors. Example: While washing the dishes with your hands, while watching a movie on your iPad, look around and pause the movie without touching the tablet with your dirty hands. You should be able to move your finger along a flat surface as a virtual trackpad for the computer you’re looking at. You should be able to touch type on any surface.
Google told me that the problem of being able to touch type on any surface will be solved within a few years, by the end of 2024. With such a focused design, Apple thinks it could potentially replace the mouse, trackpad, and keyboard with eyewear as opposed to the Vision Pro. This literally means that the lack of a display means you can only see your real-world environment through a pair of frames, but the glasses still track your eye movements just like Vision Pro.
Apple Vision Pro uses eye tracking to target what you’re looking at, so when you pinch your finger to make a “mouse click,” the entire system responds to exactly what you need in that moment. It is also used to drive an outward-facing display system that displays the included persona avatars and recreated eyeballs to external viewers. That’s the amount of technology, weight, and expense that Apple introduced with Vision Pro to completely corral people into a focused virtual location, represented as Apple’s home environment, and then use dials and software to completely confine them elsewhere on the mixed reality spectrum.
None of this sounds like a mass-market need, unless you’ve experienced instantly feeling like a superhero with a VR headset in 2017. Why do VR headsets need eye tracking? For the same reason, computers need mice. It’s a way of telling the computer what you want in a graphical user interface, even if it needs something else to select what you’re looking at.
