Apple Vision Pro development: what Apple Vision Pro developer needs to know

Now that we’ve learned about what Apple Vision Pro is and what it can do for your business in the first blog of our new and exciting series, we figured it’s time to do a little guide for Apple Vision Pro for developers.

So, we've discovered that Vision Pro is a mixed-reality headset that seamlessly merges the immersive elements of virtual reality (VR) and augmented reality (AR). This groundbreaking device features high-resolution displays, state-of-the-art sensors, and robust processing capabilities, offering boundless opportunities for users.

If you plan on developing your own application for Apple Vision Pro, it’s high time to get prepping, starting with learning everything you need to know about Vision app development. And we’ve got you covered.

visionOS: what you need to know

visionOS is the operating system powering the Vision Pro headset, tailor-made for spatial computing, enabling the creation of captivating apps and games that seamlessly merge digital elements with the real world. It is built on the foundation of Apple's established operating systems.

At the architectural level, visionOS shares fundamental components with MacOS and iOS but introduces a unique "real-time subsystem" dedicated to interactive visuals on Vision Pro. Its three-dimensional interface liberates apps from conventional display constraints, enabling them to coexist side by side at varying scales. The UI dynamically responds to natural light, casting shadows to convey scale and distance.

At launch, visionOS will offer a variety of apps, including native Unity apps for Vision Pro, Adobe's Lightroom, Microsoft Office, medical software for anatomy renders, and an engineering app visualizing physics phenomena on real-world objects.

Secrets of Apple Vision Pro for programming

To understand how to create apps for Vision Pro, you need to know that the concept behind spatial computing is to have your work environment suspended before you, creating a floating virtual space. The "shared space" refers to the area where apps coexist side by side, akin to multiple windows arranged in midair rather than on a traditional desktop.

In visionOS, users can open one or more windows, which appear as planes in the virtual space. These windows offer support for both traditional 2D views and controls, as well as the integration of 3D content alongside 2D elements. Beyond windows, apps can create three-dimensional volumes that house objects and scenes. Volumes have the unique feature of being movable in 3D space and viewable from all angles.

Let’s go over some of the “secrets” of developing for Apple Vision Pro:

1. All apps on VisionOS exist in 3D space, including basic 2D apps ported from iOS or iPad OS. Whether in a user's real-world room view or a simulated environment, apps will "float" in 3D space.
2. Consider the Field of View and opt for a landscape screen, as it is easier for users to see side by side rather than up and down, keeping in mind the unique characteristics of Apple Vision Pro.
3. Prioritize user comfort and posture by placing content within their line of sight at an optimal distance, neither too close nor too far. Exceptions can be made for specific experiences.
4. Older UIKit apps can be recompiled for VisionOS, gaining some 3D presence features. While not fully immersive, they will have a native VisionOS look, coexisting well with SwiftUI-based applications.
5. Be mindful of users' physical surroundings, avoiding excessive movement demands and large space requirements for the app's usage.
6. Make strategic use of depth for specific tasks, placing controls closer for ease of use or enabling interactive 3D objects.
7. Object lighting adapts to spatial conditions, with objects inheriting lighting and shadows from the user's environment. You can customize lighting with image-based lighting assets.
8. ARKit on VisionOS adds skeletal hand tracking and accessibility features, allowing gestures to control the virtual experience. Accessibility features enable interactions via eye movement, voice, head, and hand movements.

Considerations for Vision Pro app development

Since Vision Pro is a new and intriguing technology with lots of potential but still somewhat unknown to the masses, here's what you should remember as a developer:

• Familiarize yourself with visionOS tools and technologies to create connected, productive, entertaining experiences.
• Ensure compatibility with existing iPadOS and iOS apps, appearing as scalable windows in visionOS. Check out the official Apple Vision Pro compatibility evaluations.
• Design straightforward interactions based on eye and hand gestures, supporting standard system gestures and external game controllers.
• Prioritize privacy by informing users about data usage, offering a privacy statement, and securing data using Apple's security technologies.
• Clearly explain the purpose of data collection and its benefits to users.

What you need for developing apps for Apple Vision Pro

To begin developing apps for Vision Pro, you'll require a Mac computer running macOS Monterey or a later version, along with the latest Xcode release and the Vision Pro developer kit.

The development process involves downloading the visionOS SDK and utilizing familiar tools like SwiftUI, RealityKit, ARKit, Unity, Reality Composer Pro, and Xcode, which are also used for building on other Apple operating systems.

Adapting your existing apps for Vision Pro with the visionOS SDK is possible, although some adjustments in code for platform differences will be necessary. Most macOS and iOS apps are compatible with Vision Pro, maintaining their appearance but presenting content within the user's surroundings as a separate window.

Let’s go over the essentials for your own Apple Vision Pro dev kit:

• SwiftUI is an Apple framework for user interfaces. It's ideal for Vision Pro apps, creating immersive experiences by overlaying 3D models onto the real world with precise user movement tracking.
• Xcode, Apple's macOS integrated development environment (IDE), is vital for creating apps across Apple platforms, including Vision Pro. Its abundance of tools, notably the simulator, allows easy app development and testing in a virtual setting.
• RealityKit, exclusively designed for Vision Pro, is a 3D rendering engine enabling the creation of lifelike, interactive 3D content. It’s packed with features tailored for Vision Pro apps, including spatial computing and augmented reality support.
• ARKit, Apple's augmented reality framework, will help you craft apps overlaying digital content onto the real world. It's great for creating diverse, immersive experiences, from games and educational apps to training simulators.
• Unity, a renowned tool for visually stunning games, is also excellent for developing Vision Pro games. With support for spatial computing and augmented reality, Unity offers great potential.

Apple Vision Pro SDK

The visionOS software development kit (SDK) is now available, empowering Apple's developer community to create groundbreaking app experiences for Vision Pro. Developers can leverage the unique capabilities of Vision Pro and visionOS across diverse categories such as productivity, UI/UX design, and gaming.

For optimal 3D content integration in visionOS apps and games, the all-new tool, Reality Composer Pro, accessible via Xcode, enables previewing and preparing 3D models, animations, images, and sounds to deliver stunning visuals on Vision Pro. Additionally, developers can interact with their apps in the visionOS simulator, exploring and testing various room layouts and lighting conditions.

Moreover, each developer framework includes built-in support for Apple's innovative accessibility features, ensuring that spatial computing and visionOS apps are inclusive and accessible to all users.

How to apply for an official Apple Vision Pro developer kit

To get an official Apple Vision Pro developer kit, you need to apply for it on Apple’s website. To participate, the developers must have an Account Holder status and provide information about their team's development expertise and current applications. Apple states that the preference will be given to those whose app will focus on the features and capabilities of visionOS the most.

Developers who obtain a Vision Pro will receive comprehensive assistance, including device setup and onboarding, regular check-ins with Apple experts for UI design and development advice, as well as support for refining their applications. Additionally, each developer granted access to the kit will be entitled to two code-level support requests to address any troubleshooting issues they encounter.

It is essential to note that Apple considers the Apple Vision Pro headsets and designated development devices as their property. It means they must remain in the private and secure workspace where they were delivered, accessible solely to the approved developers working with them. Apple will maintain consistent communication with the developers utilizing the kit, and they may request the kit's return whenever they want.

Apple also establishes Apple Vision Pro developer labs across various locations, including London, Munich, Shanghai, Singapore, Tokyo, and Cupertino. These labs will offer developers the opportunity to visit and test their applications. For those unable to travel to Apple sites, developers can submit requests for Apple to evaluate and test their app builds, receiving valuable feedback remotely.

Want to be an Apple Vision Pro developer?

Want to develop for Vision Pro? These are just a few valuable guidelines for app developers venturing into the realm of Apple Vision Pro. If you have any questions or insights about app development for Apple's VR headset, feel free to get in touch. And stay tuned for our next piece in this series, where we delve into Vision Pro interface and how it will change the user experience.