Add support for visionOS

[vanlooverenkoen]

Use case

The apple vision pro was released yesterday. It would be nice to have support for that. Even though most apps will be supported out of the box. This ticket can track the evolution of vision pro for Flutter

Proposal

This should just work out of the box.

[stuartmorgan]

At least some of this tvOS comment and this watchOS comment will almost certainly be applicable here. The extent to which visionOS differs from iOS remains to be seen as we learn more about the target and SDK.

[Update Oct 2023]: Some specific differences at the engine level:

• All uses of UIScreen would need to be conditionalized. In many cases we probably want to move away from UIScreen anyway (e.g., Audit use of UIScreen scale #134062), and this is an area where someone motivated to maintain a custom engine build could upstream some PRs, but other would need different codepaths.
• A new OS type needs to be plumbed into the build, including dependencies like Dart and Skia. Parts of Skia would need conditional compilation (e.g., MTKView).
• Various specific areas of code would need to be conditionalized (keyboard handling, status bar handling)

Based on the initial investigation, it is the case that most of the work would be in the long term support, as discussed in the linked issues above.

[MousyBusiness]

I feel these is a great opportunity for Flutter here as iOS tooling is lacking and Vision OS is the perfect environment for a free rendering framework like Flutter. If Flutter doesn't compete, React Native will charge ahead. Vision OS won't be for everyone (at least at this stage), but it would be excellent for developers who will likely be early adopters of this technology. Getting the jump on this could be pivotal in Flutter growth.

[34r7h]

Agree with sentiments above. Making a strong commitment to immersive reality early in the game could solidify utility in the future. I think it'll become the standard form of computing within 10 years; replacing the desktop environment first then mobile. I also see the potential to rethink development environments all together and jumping into an immersive development platform being the way to go. As it stands, Unity and Unreal have a big head start.

[BraveEvidence]

Good to see people interested in apple vision pro but why not support other vr headsets like meta quest which is based on android and has been in the market for quite a while now.

[kaljitism]

As shown in the impeller demo, flutter could support 3D scenes within the widget tree, which is an amazing in the direction. More work could definitely be put to improve Spatial UI elements such that in future - if not vision pro, but other AR devices could run Flutter app as a better option to already existing ones.

[thipokch]

Great news! VisionOS is released along with Xcode 15 beta 2. Although, Flutter has yet to support VisionOS SDK, it seems to be working on visionOS via iOS (Design for iPad) target.

You can also do hot reload just like Design for iPad target using flutter attach. See #84411 .
For better DX, I also think we should properly support flutter run 'Design for iPad' as a target.

[Mambsy]

Managed to get the flutter demo to run natively (not as a iPad app) through a few little tweaks, convinces me that it would be possible and actually quite trivial to add VisionOS as a target, only mystery to me is how it would work with the eye gaze

Here is a video where I show it in action: https://twitter.com/MrMambwe/status/1671969959553933318

[stuartmorgan]

Since this is a broad umbrella request, I've filed some specific issues for targeted requests that would apply to iOS Flutter apps running on visionOS. I would encourage anyone interested in those issues to vote for them specifically so that we can gauge interest in specific items (which have much different scope from some of the comments above):

• Support visionOS devices for iOS apps in flutter devices, flutter run, etc.
• Support eye tracking when running on visionOS

[bvoq]

I would love to see a demo with Impeller 3D models working on visionPro!
In particular, it would be great if we can have windowed flutter applications but with 3D elements inside the windowed app (for example spinning 3D models inside the mobile UI).

[Jordanlaubaugh9]

Managed to get the flutter demo to run natively (not as a iPad app) through a few little tweaks, convinces me that it would be possible and actually quite trivial to add VisionOS as a target, only mystery to me is how it would work with the eye gaze

Here is a video where I show it in action: https://twitter.com/MrMambwe/status/1671969959553933318

Very cool. Would love to see the code!

[stuartmorgan]

Managed to get the flutter demo to run natively (not as a iPad app) through a few little tweaks

@Mambsy Is this using a custom engine build? I wouldn't expect that running in native visionOS mode would be possible without one.

[edeuss]

@Mambsy Is this using a custom engine build? I wouldn't expect that running in native visionOS mode would be possible without one.

@stuartmorgan You can find what he did in this thread: https://twitter.com/MrMambwe/status/1672339427136249857

[stuartmorgan]

If that's a complete set of steps, then all that's actually demonstrating is that the visionOS simulator will, as has long been the case with the iOS simulator, run code that an actual device would not (because it's a simulator, not an emulator). It's not valid to draw any conclusions about actually supporting visionOS (e.g., that it's "actually quite trivial to add VisionOS as a target") from that.

[squirelboy360]

I just saw a React native app for VisionOS. I think if flutter supports Vision OS and probably other wearables, it would be the best cross platform framework to support Vision Os considering how performant flutter is. Just think about making an immersive flutter flame game that runs and fits the style of visionOs.