June 12, 2023, 12:58 pm | Read time: 9 minutes
The new mixed-reality headset that Apple has been working on for years is now a “real” reality. With the device, the company sets new standards–but it comes at a price.
Apple’s mixed-reality headset is official and is called Vision Pro. It is a completely new product category for the company, but it could significantly shake up the market for virtual and augmented reality.
Overview
Launch in the U.S. in early February 2024
For a long time, it was unclear whether Apple could meet its self-imposed goal of launching Vision Pro in early 2024. But now the company has announced the launch date in a press release. The headset will go on sale on February 2, and pre-orders will be available starting January 19.
However, Vision Pro will initially only be available in the U.S. There is no information yet about a possible global release, including in Germany.
A dedicated operating system for the virtual home screen
At first glance, the user interface of the Apple Vision Pro resembles that of the iPhone or iPad. Many functions familiar from these devices are also available here. The dedicated operating system, visionOS, also enables video conferences with FaceTime in virtual meeting rooms. But how can other call participants see the headset wearer? When setting up the Vision Pro, users must film themselves once with the device, which then creates a digital persona. This avatar mimics all the wearer’s facial and hand movements and has volume and depth to appear as realistic as possible.
The Vision Pro includes Apple’s first 3D camera. The headset offers new ways to interact with photos and videos. When files are opened, the room automatically dims. Self-taken panoramic photos appear life-sized. The headset gives 3D photos and videos a depth effect, allowing moments to be almost relived. The device also supports 3D movies like “Avatar.” Users can watch videos as if on a big screen and even adjust the window size as desired.
Apple makes it quite easy for developers to optimize their existing apps for visionOS. Although the new operating system has its own app store with its own applications, compatible iPhone and iPad apps are also available here. Similar to Face ID and Touch ID, the Vision Pro now has Optic ID, which scans the iris to unlock the device. This enables functions like Apple Pay, App Store purchases, and password access. For those who are critical of the eye-tracking inside the headset, Apple reassures them with the promise that apps and websites cannot track it. An additional layer registers the input and only then passes it on.
Apple has started a collaboration with the Walt Disney Company, which is celebrating its 100th anniversary this year. Not only does Disney use Vision Pro headsets to create new 3D experiences, but Disney+ will also be available on the device from day one.
Operation possible without any accessories
Apple has developed a completely new control system for the Vision Pro that does not require controllers or similar devices. Operation is done exclusively through eyes, hands, and voice. Cameras inside the headset track eye movements and can recognize what users are looking at. These objects are then highlighted in the interface. Pinching the thumb and forefinger confirms the input. Swipe gestures with the fingers are used for scrolling. To search, it is enough to look at a text field and dictate the input. A virtual keyboard is also available. Siri commands can also open apps, just like on the iPhone.
visionOS is fully 3D-compatible. For example, a 3D object in an email attachment can be easily dragged with a hand gesture and then viewed in augmented reality from all directions. But that’s not all: the system can also connect with other Apple devices. For instance, if you look at a MacBook, the headset automatically projects the laptop’s screen. Apps can be launched directly from there–without installation.
Nevertheless, the headset is compatible with Bluetooth hardware. Keyboards, trackpads, and mice can be used with it. It can also connect with gaming controllers like Sony’s DualSense, as the device supports Apple Arcade. Although it has built-in speakers, AirPods can be used for more private use.
Wearers get digital eyes
A unique feature of the Vision Pro is a function called Apple Eyesight. To use the headset as naturally as possible in the real world, it is important to communicate with the wearer. To avoid having to remove the headset, it includes a curved OLED screen that displays the eyes outward. Through a lenticular lens over the screen, the eyes are visible from any angle to the other person.
When used in augmented reality mode with apps or similar, the headset places a slight shimmer over the eyes. However, when full virtual reality is activated, the eyes are covered with a veil. This allows others to see when you are fully immersed in the virtual interface. If a person approaches, the headset recognizes them and only makes their shape visible to the wearer.
Read also: Apple introduces first 15-inch MacBook Air
Powerful technology inside
Apple itself refers to the Vision Pro as a “spatial computer.” The headset must process 23 million pixels, which is more than 4K resolution per eye. This enables razor-sharp text for reading and writing, which has always been a weakness of VR/AR headsets.
For the necessary performance, the company relies on its own M2 chip, which is accompanied by a completely new chip: the R1. This chip is specialized in processing sensor data. It can process data from 12 cameras, 5 sensors, and 6 microphones without delay. Even the slightest delay can contribute to discomfort in a mixed-reality headset.
On the front is a single, curved piece of glass that transitions into an aluminum frame, housing the camera button and the Digital Crown. Under the glass are cameras for hand tracking and the 3D camera. Inside are the two lenses, behind which are the two displays for the eyes. Glasses can be magnetically installed on the lenses to correct individual vision errors. A “light seal” is available in various sizes for sealing. In the arms are two audio pods that support 3D audio. Through “audio ray tracing,” the pods are supposed to adapt the sound to the room. The rear part consists of braided, stretchable, and breathable textile.
Read more: What is the difference between VR, AR, and MR?
Opinion on Samsung’s 3D Monitor at IFA: “This Technology Leads to a Dead End”
What We Expect from Apple’s “Special Event” on June 9
Who should buy the Apple Vision Pro?
Apple aims to combine the real and digital worlds with the Vision Pro. The company has developed a range of new technologies to make the transition as seamless as possible.
According to Apple, the main argument for using the Vision Pro is that users are no longer limited to a simple screen. The virtual interface offers virtually unlimited screen space thanks to a 360-degree environment. Apps can be used anywhere within it, resized, and freely placed. They have volume, react to light, and cast a shadow. However, users still see their real environment–the headset offers not just virtual but augmented reality.
It is still capable of virtual reality. Through a Digital Crown, also found in the Apple Watch, users can determine how much of their real environment they want to see. With a turn, a kind of veil is placed over the field of view and behind the displayed content, which adjusts according to the type and colors of the open windows. The degree of coverage is freely selectable.
Apple plans to position its headset alongside the iPhone, iPad, Mac, and Watch as a new product category–but the mixed-reality market is still quite young. The existing technologies are promising but remain underdeveloped and expensive, making them hardly suitable for the mass market. This also applies to the Apple Vision Pro, which at $3,499 is no bargain for consumers. The company is at least trying to market the device as a replacement for high-end TVs, computer monitors, audio systems, 3D cameras, and more.
The “New York Times” reports that many Apple employees are skeptical about the new product. Some have dropped out, while others were let go due to a lack of progress. Internally, there seems to be disagreement about what purpose the headset should serve. This stands in stark contrast to Apple’s otherwise determined product marketing–such as with the first iPhone: “an iPod, a phone, an internet communicator.”
Apple reportedly plans a cheaper version of its headset
Not only did the Apple Vision Pro polarize at its unveiling at WWDC 2023, but the price of the headset was also a statement. Regardless of how good the technology inside and how extensive the possibilities, users must also be able to afford the device. And at around $3,500, the Apple Vision Pro is anything but cheap. So how well will it sell and establish itself in the market?
Due to its high investment in the AI/AR sector and the estimated $15 billion cost of product development, Apple cannot currently offer its headset much cheaper. But in a few years, according to Mark Gurman, this could change. By then, Apple could have recouped a large part of its investment, he writes in his newsletter. This circumstance and the name Vision Pro suggest that Apple is apparently planning another headset.
This could then be called simply Apple Vision or Apple Vision One and be significantly cheaper than the Pro version. The company could save costs, for example, by using a simpler headband and relying on its AirPods for audio output with spatial audio instead of built-in speakers. The cheaper headset might also omit features like the 3D cameras. However, features like EyeSight, which displays the eyes while wearing, and the tracking system itself are expected to remain part of the headset.
However, it will take some time before this is ready. Gurman does not expect the cheaper headset before 2025.