Developing Games for Apple Vision Pro: A Guide

At Mind Studios Games, we were lucky to get a chance to work on Apple Vision Pro game development. So let us talk a bit about it.

Highlights:

• Apple Vision Pro is posed to become a game changer for AR and VR game development;
• Apple Vision Pro game app development is possible in Unity as well as in Apple’s native environment;
• The challenges in game development for Apple Vision Pro are entirely manageable with a skilled team;
• Apple’s tools for Vision Pro development make the work of game artists more seamless.

Apple Vision Pro game app development has been gaining a lot of traction following the device’s launch in early February 2024. And it so happened that we’ve got a client with ambition and a plan to be part of the emerging market that is this blend of AR and VR.

In this article, we’d like to share some intricacies, practices, and challenges we’ve come across.

Fruitopia Watermelon Game is an XR-enabled casual game with falling and merge mechanics — players need to drop fruits into a basket, where identical fruits will merge into something new. The more you merge, the higher your score is — and the higher you are on a leaderboard. Mind Studios Games crafted 2D art, UI/UX design, 3D art, and animations for this title.

But first things first, what is Apple Vision Pro, and why should you bother with it?

Is Apple Vision Pro for gaming?

Short answer — yes, very much so.

“The Apple Vision Pro has made an innovation among headsets, as it is the first to implement an AR mode. Previously, all headsets were limited to VR, while AR was only available on phones. The Apple Vision Pro has several unique features that make it appealing for the development of games and interactive applications.”

— Ivan Dyshuk, Head of Gamedev at Mind Studios Games

Apple Vision Pro is a mixed reality headset — it blends augmented reality (AR) and virtual reality (VR). The unique offer of this new device is that it allows developers to create games with both AR and VR modes or levels. At the same time, the device has a lot of other innovative features to offer as well.

Spatial interface

Apple Vision Pro uses a spatial interface, enabling deeply immersive games where users can interact with virtual objects in real space. This approach creates opportunities to develop games that integrate seamlessly into the real environment.

Spatial audio

The device supports spatial audio, allowing users to hear sounds positioned precisely within their surroundings. This adds realism to games, as sounds adapt to the user’s position and the location of objects within the virtual environment.

Extended Augmented Reality (XR)

Vision Pro combines the capabilities of VR and AR to create extended reality (XR), where users can transition from fully virtual environments to semi-transparent elements that blend into the real world. This opens up possibilities for new gameplay mechanics, where virtual objects can interact with and respond to the real world.

Eye and gesture Control

Vision Pro uses eye tracking and gesture recognition as primary control methods. This means that games can adapt to the player’s gaze direction, allowing for precise targeting, item selection, or navigation with just eye movements. Gestures also enable players to control games without controllers, making interactions feel more natural.

High resolution and performance

Vision Pro is equipped with high-resolution displays and a powerful processor (Apple M2 and a dedicated R1 chip), enabling high-quality graphics and smooth FPS. This is crucial for games with high levels of detail and realistic graphics.

Integration with the Apple ecosystem

The device integrates with other Apple devices and services, such as iPhone, iPad, Apple Watch, and Mac. In other words, it’s your access to the App Store and its millions of dedicated users. This makes for a massive market, a significant part of which is considered a premium tech segment.

On top of that, this allows developers to create games that sync across different devices, utilize their sensors or cameras, or even work as cross-platform projects.

Apple APIs and developer tools

Apple provides access to ARKit, RealityKit, and other development tools, significantly simplifying the creation of games for Vision Pro.

All in all, Apple Vision Pro is kind of a game-changer for the gaming and entertainment industries.

Besides the popularity granted by novelty (which is always temporary), many game developers believe the immersive quality of the gameplay Apple Vision Pro offers will captivate players for a while.

So who is the target audience for Apple Vision Pro?

Apple targets this headset mostly at a tech-savvy and tech-driven population — young adults and older professionals, ages between 18 and 45. The second segment (professionals in their 30s ans 40s) tend to be relatively well-off too. This makes developing games for Apple Vision Pro a lucrative undertaking with a potentially lasting ROI.

Now, how do you get access to all that? Let us offer our considerations for Vision Pro game app development.

Vision Pro game tools: Development environment

First, let us briefly lay out what you (or your hired developers, if you’re not one yourself) will need to make Apple Vision Pro VR games.

Hardware requirements for developing games for Apple Vision Pro

1. Mac computer. Seems obvious, but we’re still adding it. For the smoothest Apple Vision Pro development process, Apple recommends game developers to have a Mac with Apple Silicon M1 or later, meaning a Mac built in 2020–2021 at the earliest.
2. Apple Vision Pro Developer Kit. Now, the Vision Pro headset is quite pricey — at the time of me writing this, it’s listed on the Apple website at $3,499. Developers can also apply to get it on loan from Apple. Finally, the current version of Xcode has an Apple Vision Pro simulator for those who can’t get their hands on the developer kit itself.
3. Wi-Fi and Bluetooth. For testing and debugging over the air if you do have the actual hardware.
4. Game controller. Not mandatory, but to test games with additional inputs, game controllers compatible with macOS and Apple platforms will be worth getting.

Software for developing games for Apple Vision Pro

• Xcode (version 15 or later). It’s the integrated development environment (IDE) for building and testing Vision Pro games. Since Vision Pro is a newly released device, it stands to reason that the latest version of Xcode is the best option for building and testing Vision Pro-enabled games. Check it out here.
• visionOS SDK. The software development kit (SDK) that provides the necessary tools, libraries, and APIs for Vision Pro app development.
• Reality Composer Pro. A tool for designing 3D scenes and environments for visionOS. We’ll cover a bit on the specifics of integrating 3D models into Apple Vision Pro projects in the next section.

Both Unity and Unreal Engine support Vision Pro, so you don’t necessarily need to hire Apple iOS app developers for your game to support Vision Pro integration. Moreover, being engines created and geared towards game development, Unity and UE offer tools for building both simple and more complex games that you can further integrate with the new VR/AR tech.

At Mind Studios Games, we work with Unity, so if you’re looking for game developers who are adept at this game engine and already have experience with Apple Vision Pro game development, look no further — contact us right away for a free first consultation.

Designing for Apple Vision Pro

Designing games for Apple Vision Pro requires a deep understanding of both technical and user experience (UX) principles. Apple’s XR introduces new opportunities but also new challenges when it comes to interaction, immersion, and system performance. Below are the best practices for designing games for the Apple Vision Pro, with key considerations in UX/UI and immersion.

UI/UX in XR game design

UI and UX are always of the utmost importance in game development because they affect the player's ease of use and immersion into the game. And unlike traditional gaming, XR engages the player’s entire field of vision and movements, making it more physically and emotionally interactive.

Poor UI/UX can quickly lead to frustration and confusion in any game, but in VR and XR gaming, it can even cause motion sickness! On the other hand, a smooth and intuitive experience will increase player satisfaction manyfold. Here’s what our designers suggest Apple Vision Pro game developers to keep in mind.

Best practices for UI/UX
Intuitive controls	In VR mode, controls should feel natural and minimize the learning curve. Vision Pro supports gestures, eye-tracking, and voice commands, so developers should use them to create seamless interactions. Avoid complex control schemes or actions that might confuse players.
Eliminate UI clutter	Too many on-screen elements can overwhelm players in XR. Use spatial elements like holograms or 3D objects for UI, and make them context-sensitive so they only appear when needed.
Diegetic UI	This is a UI approach where in-game objects act as interfaces. For example, rather than floating menus, a character’s smartphone or some permanent object in the environment could display game stats and options. This will increase immersion significantly.
Comfortable interactions	Keep menus and interactions within a player’s natural field of view (which is usually 30–60 degrees both vertically and horizontally). Avoid designs requiring excessive head or body movements to navigate UI elements.
Accessibility	The Vision Pro’s advanced sensors enable more inclusive input methods. Try to accommodate players with different physical abilities by providing options like gaze-based controls, larger interactive zones, or voice commands.

Designing for maximum immersion

The four key elements of immersion in VR and XR are:

• Presence: The illusion of “being there” is the cornerstone of VR gaming. Every element in your game should reinforce the player’s feeling of being present in the virtual world, in the flesh.
• Interactivity: Allow players to interact with the environment. The more players can affect the world around them, the more connected they will feel to the experience.
• Audio: Apple Vision Pro’s spatial audio capabilities allow game developers to create rich, immersive environments where sound sources can be positioned very precisely. This makes it possible to use sound to guide players’ attention, increase tension, and enhance storytelling.
• Realistic physics and feedback: Simulating realistic physics and offering haptic feedback through controllers can significantly improve the immersion, as players will see and feel the direct effects of their actions.

How to approach XR and VR game design to get the best result for your game’s immersion? We can think up several tips.

#1 No non-interactive objects

When players load into VR, they expect to be able to manipulate every object in their line of sight, just like they can in real life. So creating objects players can’t interact with will result in lower immersion. Not to mention that you’ll waste your artists’ and developers’ time and effort.

Since Apple Vision Pro supports both full VR and mixed reality, you can blend virtual elements with the real world. Consider creating scenarios where players can interact with virtual objects overlaid on their physical environment.

#2 Invest in good graphics and animation

While not all VR games are photorealistic, all of them need good graphics even more than traditional games do. Low-quality and glitchy animations can and will disrupt the player’s immersion in the game, possibly resulting in abandonment.

#3 Design with first-person perspective in mind

While there are a number of VR games with third-person perspective (Hellblade: Senua's Sacrifice has a VR Edition, and it’s every bit as haunting as the non-VR one), usually when people think VR they imagine first-person perspective. This extends to XR as well, in most cases. Mechanics, UI elements, and certain narratives for a first-person game will differ from those in third-person games. You have to account for that when you plan virtual reality game development.

Tip: Apple Vision Pro’s eye-tracking allows for gaze-based selection. This can be used for precise targeting, reducing the need for hand or head-based pointing.

#4 Try dynamic environments

Environments that react to the player’s presence or actions can make the world feel alive. This can include physics-based responses as well as effects like changing lights or sounds.

Moreover, in mixed reality, players will be moving through their real space. Ensure your game is aware of physical obstacles and use tools to create boundaries or alert players if they’re nearing real-world objects.

#5 Consider body presence

Simulating the player’s body in VR, where they can see their arms or legs move in accordance with their real-life actions, can greatly contribute to the game’s immersion. However, this must be done carefully to avoid causing a disconnect if the body does not move as expected.

Integrating 3D models into Apple Vision Pro

The format for 3D objects to be added to games based on visionOS is .usdz, a proprietary format Apple uses for AR and VR. However, it’s easy to convert 3D models to .usdz from the .obj, glTF, and .usd formats using Reality Converter Pro, a free currently-in-beta drag-and-drop tool designed just for that. And you can obtain your 3D models as .obj files from almost any 3D editing software, like Blender.

Tip: Blender and Autodesk Maya actually provide .usdz as a format for exporting scenes, so you might not even need a converter.

Now, object integration in Apple Vision Pro requires developers to use both Xcode and Reality Composer Pro, but the latter is already included in the former, so you won’t be able to miss it.

Basically, Reality Composer Pro is a scene editor for Xcode, it allows you to preview how your 3D model will work in a real environment. You can adjust scale, rotation, textures, add animations and interactions, and design spatial scenes tailored for Vision Pro

Just import your .usdz file into either Reality Composer Pro or Xcode and do whatever you require with the models — RealityKit allows you to load and manipulate 3D models, handling real-time physics, lighting, animation, etc.

When you’re done, use Xcode to deploy your AR/VR scenes directly to the Vision Pro to view them in spatial computing mode, where you can employ eye-tracking, hand gestures, and spatial awareness to enhance interactions.

That’s it!

Apple Vision Pro game app development: 7 tips from Mind Studios Games experts

Our current project in Apple Vision Pro game development is focused on visuals — we did UI/UX, 2D art for interfaces, 3D models, and animation. In this section of the article, we’d like to focus on creating 3D models for our client’s Apple Vision Pro game, as this process has some peculiarities, which we’d like to cover.

As we’ve already mentioned, the uniqueness of Apple Vision Pro is that it blends virtual reality with augmented reality. This poses a bit of a challenge for 3D modeling and UI/UX design in games. It’s nothing an experienced artist can’t solve, though.

Here’s what our Lead 3D artists Andrii Koval has to say about his experience with 3D modeling for an Apple Vision Pro game:

Now, as for the tips on how to make better 3D models that work well with mixed reality, Andrii offers the following:

1. Make an effort to build your models as seamless as possible. Work closely on the UV map to make as few extra vertices and seams as possible — this will make the game’s performance better.
2. Optimize core animation to use things like shadows, blurs, and masking only as much as it is necessary for the players to have a great experience with your game. The reason for that is effects can cause offscreen passes — the need to render effects invisible to players. This, in turn, can result in frame drop and lagging performance of the game.
3. Create your models in real-world scale for a seamless integration into mixed reality. Apple Vision Pro uses a highly immersive environment, so objects need to appear realistic when interacting with physical surroundings.
4. Prioritize spatial awareness. Include collision meshes for physical interactions, and use tools like ARKit Scene Understanding to test how models interact with real-world surfaces. Avoid overly complex geometry for objects meant to rest on or interact with real surfaces.
5. Design detailed areas of your model where users are likely to focus; less visible areas can have lower texture quality or fewer details. This is done to make full use of the Apple Vision Pro’s foveated rendering (prioritizing high resolution where the user is looking).
6. Use baking techniques (lighting, ambient occlusion) to pre-render details and improve performance.
7. Always test extensively in XR environments and iterate based on feedback.

Overall, as per our artists, 3D modeling for Apple Vision Pro benefits from the same tricks as modeling for mobile games. That’s because performance is an issue for spatial apps — just like it is for mobile games. However, Apple Vision Pro offers quite a selection of tools to deal with performance drops. These tools have a bit of a learning curve, but the results are worth the trouble.

Best practices for Apple Vision Pro game development

When developing games for Apple Vision Pro, ensuring a smooth gaming experience is essential for maintaining immersion and preventing player discomfort, such as motion sickness or frustration. Performance optimization plays a key role in achieving this. Additionally, troubleshooting common issues during development and ensuring accessibility and inclusivity are critical for creating a welcoming experience for all users.

VR games demand far more computational resources than traditional games due to the necessity of rendering two images (one for each eye) at high frame rates to prevent motion sickness. During planning for how to develop apps for Apple Vision Pro properly, one of the important tasks for developers is to carefully balance performance vs. quality of graphics to maintain immersion and comfort. How do we at Mind Studios Games do that?

Performance optimization and resource management for apple vision pro games
#1 High frame rates (90 FPS Minimum)	Low frame rates in VR can cause motion sickness. Aim for a stable frame rate of at least 90 FPS, and ideally higher, to ensure a smooth experience. This might require dynamic resolution scaling to adjust the resolution in real time based on the current workload: when the game detects high GPU usage, it can temporarily lower the resolution to maintain a steady frame rate.
#2 Foveated rendering	Foveated rendering is a technology supported by Apple Vision Pro’s eye-tracking. This technique renders in high detail only where the player is looking, reducing the workload on the GPU for peripheral areas.
#3 Asynchronous time and space warp	Asynchronous time warp reprojects frames to account for head movement when new frames are not ready, while asynchronous space warp generates synthetic frames to maintain smooth motion at lower frame rates.
#4 Asset optimization	Optimize 3D models and textures by using LOD (level-of-detail) techniques, where objects that are farther away are rendered with lower detail. Compress textures and optimize 3D models to reduce the processing power required while keeping the visual fidelity high. One of the techniques we use is texture atlasing — combining multiple textures into a single large texture. It can reduce draw calls and improve performance.
#5 Optimization of shadows and lighting	Shadows and dynamic lighting are hard to render. Use baked lighting for static scenes to reduce GPU load, and consider reducing the resolution or quality of shadows in dynamic scenes.
#6 Memory management	VR requires a lot of memory to manage detailed 3D environments and textures. Stream assets efficiently to avoid excessive memory consumption, especially with high-res textures.
#7 Latency minimization	High latency can break immersion or cause discomfort. Ensure that input actions, like moving or interacting, are instantly reflected in the game world. Aim for input-to-output latency under 20 milliseconds.
#8 Efficient physics and animation systems	Physics calculations and animations can be resource-intensive. Optimize these systems by reducing complexity when possible and using simplified collision meshes for objects.

Common challenges in Apple Vision Pro game development

During Apple Vision Pro game development, you may encounter various performance or usability issues. Below are common problems and tips for troubleshooting them.

Motion sickness / discomfort

Problem: If players report motion sickness, it is usually due to frame rate issues or unnatural movements in-game.

Solution: As mentioned in the section above, it’s recommended to ensure a steady frame rate of at least 90 FPS. Also, avoid jerky camera movements, and make sure the player’s movement speed feels natural and comfortable. Provide smooth maneuvering alternatives, such as teleportation, for players sensitive to motion.

Poor performance in complex scenes

Problem: Performance drops in scenes with many objects or dynamic lighting.

Solution: Implement occlusion culling to prevent the rendering of objects not visible to the player, reduce the complexity of environments with LOD, and use baked lighting where possible.

Tracking problems

Problem: Inconsistent tracking of head, hand, or eye movements.

Solution: Ensure that the Apple Vision Pro’s sensors are being utilized correctly. For hand tracking issues, consider reducing the precision required for hand gestures or expanding the interactive hit boxes to improve detection.

Unexpected crashes or memory leaks

Problem: Games crash or slow down after extended play sessions.

Solution: Monitor memory usage and implement proper memory management techniques. Release unused assets and avoid memory leaks by cleaning up assets between scenes.

Unintuitive controls or interaction

Problem: Players struggle with controls or don’t know how to interact with objects.

Solution: Simplify control schemes. Provide clear visual and audio feedback when interacting with objects, and ensure the game provides adequate onboarding or tutorials.

9 tips for accessibility and inclusivity in VR games

Accessibility and inclusivity are essential for any games, XR included. Players of all abilities and backgrounds should be able to enjoy XR games. Apple Vision Pro offers powerful tools like voice commands, eye-tracking, and hand tracking to enhance accessibility. To make your Apple Vision Pro game more accessible, you can employ the following:

1. Multiple input options. Provide alternative input methods for players with mobility challenges. In addition to traditional VR controllers, use eye-tracking for gaze-based selection, voice commands for menu navigation, and hand tracking for gesture-based interactions.
2. Customizable controls. Offer adjustable controls and sensitivity settings. This ensures that players with different needs or preferences can tailor the controls to suit their abilities. Allow for adjustment of gestures, input sensitivity, and interaction zones.
3. Seated and standing modes. Consider making your game playable both seated and standing. Some players may have mobility restrictions or may prefer seated play. Always offer a comfortable mode for players who cannot stand for long periods.
4. Subtitle and audio preferences For players with hearing impairments, offer customizable subtitles (adjustable size, background, and color) for all audio-based information. Also, ensure that critical information is provided visually as well as aurally. Use spatial audio to assist players in localizing sounds, and provide optional audio cues for visually impaired players.
5. Colorblind modes. Offer colorblind settings to adjust in-game colors. Implement colorblind-friendly UI elements to ensure that critical gameplay elements are distinguishable to players with various forms of color vision deficiency.
6. Motion sensitivity options. Possible motion sickness is a significant issue for Apple Vision Pro game app development in general, and some players are more sensitive to motion in VR than others. Provide motion sensitivity options, such as the ability to turn off camera movement or reduce the field of view during fast movement to prevent motion sickness.
7. Visual comfort adjustments. Allow players to adjust the brightness, contrast, and scale of the VR world to accommodate different levels of visual comfort. Providing a high-contrast mode and scaling UI elements can improve readability for players with visual impairments.
8. Narrative or cognitive assistance. Provide difficulty options or cognitive assistance features, such as hints or tooltips, for players who may struggle with complex puzzles or fast-paced gameplay. Ensure that the game can be enjoyed by players of varying cognitive abilities.
9. Avoid rapid flashes and patterns. A lot of games warn their players about flashes and flickers, as those might trigger photosensitive epilepsy. Be mindful of players with photosensitivity and offer the option to turn off such effects in the settings menu.

Conclusion

Apple Vision Pro game development is on its way to conquer the world of VR gaming with its unique blend of VR and AR — besides the overall novelty, it offers convenience and variability to both developers and players. Moreover, with the tools Apple offered, it’s not strictly necessary to spend big money on the hardware for developers, meaning the development costs are entirely sensible even for indie developers.

With all this in mind, a lot of visionaries will most likely want to embark on this journey. If you’re one of them and wish to give life to a game idea that will look best in XR, and you’re looking for Apple Vision Pro game developers — check out Mind Studios Games. We’ve got 9+ years of game development experience, and we’ve already dipped our toes in the workings of this breakthrough tech. Schedule a consultation with our Head of Gamedev to discuss your idea and see what we have to offer.

1