UI does not work in AR Session (IOS-XRViewer)

[andresgutierrez17]

Before entering AR mode the UI works perfectly, but when switching to AR the UI does not work. I tried to set the left ar camera in the Canvas component and it does not work. I created a script to set the camera when switching mode and it does not work too.

To Reproduce
Steps to reproduce the behavior:

1. Export example AR Hit test with a UI.
2. Open in XRViewer on iPhone
3. Interact with the UI before switching to AR, this works perfectly
4. Switch to AR session and the UI does not work now

What I believe happened is that the UI is bigger than the screen in the AR session because a button on the left top works more down in axis Y and the area biggest.

Screenshots

Unity info:

• Editor version: 2022.3.4f1
• WebXR package version: 0.18.0
• WebXR Interactions version: 0.18.0
• Git or OpenUPM: OpenUPM

Smartphone:

• Device: iPhone 12 Pro Max
• OS: IOS 17
• Browser: XRViewer

Additional context
The UI works perfectly on Android, with or without the script I created to set the camera.

[De-Panther]

I don't have iOS device to test with.
If you can create a project that use Input.GetMouseButtonDown(0) and prints Input.mousePosition + Screen.width + Screen.height,
it can help debugging the issue.
My guess is that pointer/mouse down events are invoking, but the positions are wrong.

[De-Panther]

Well, the XRViewer manipulates the HTML Canvas element.
In other iOS devices with different resolution and scale, it worked as intended.
Maybe needs to consider the devicePixelRatio when calculating the position.

[GDaniel00]

Performing the test you mention, we realized that the HTML canvas scales almost three times as much, but this happens when activating AR. We were trying to scale the canvas from the index.html that it gives us when exporting from unity but it doesn't work, we also tried using windows.devicePixelRatio = window.devicePixelRatio || 1, but it didn't work either, do you have any idea how we could fix that problem?

The total area of the device is 1284X2229, but due to the canvas scale, only the red area is working, the rest of the canvas leaves the device screen.

[De-Panther]

There can be 2 causes for that:

• The XR Viewer setting the canvas with this size. As the XR Viewer is not maintained, it wasn't built and tested on some newer iOS devices, and have this issue. Maybe you can change the canvas size after entering AR?
• The Unity emscripten/SDL wrapper has a recursive method that updates the canvas size, maybe the change of the canvas when switching to AR is triggering that method, which results in enlarging the canvas more and more... I don't have a suggestion of a fix for this case.

Please keep updating how it progress.

[andresgutierrez17]

Thanks for your prompt response!
I see that you have a listener "onARSupportedCheck" in the HTML template, is there a listener for after entering AR?, I want to do a test using as below:
document.addEventListener("onAREntering", function (event) {
//Code to change the size of canvas after entering AR
}, false);

What is the name of the event listener before entering AR mode?

[De-Panther]

I don't remember if there is any.
I guess you would like to check OnXRChange

[jesuscfp]

Hi, there is any advance with this issue? I have the same problem using a iPhone XS. I've checked that with the official WebXR live demo is not possible take the ball and boxes when the app starts the AR mode.

[andresgutierrez17]

Hi, I beleave that the issue will be solved when you put this tag in your html file.
<meta name="viewport" content="width=device-width, initial-scale=1" />

https://developer.mozilla.org/en-US/docs/Web/HTML/Viewport_meta_tag

[jesuscfp]

Hi, I beleave that the issue will be solved when you put this tag in your html file. <meta name="viewport" content="width=device-width, initial-scale=1" />

https://developer.mozilla.org/en-US/docs/Web/HTML/Viewport_meta_tag

Hi andresgutierrez17, thank you for the help. I putted the meta line in the html head tag but the issue continues appearing, also the screen view in normal web is too big and it doesn't adjust to the screen.