aframe-xr won't load in WebXR-Viewer w/ aframe v0.8.2

[Danpollak]

Hey everyone,

Ever since upgrading our app from aframe v0.8.1 to v0.8.2, we are experiencing issues with running aframe-xr.
With further debugging we found out that the "enterAR" method won't load the scene correctly and will eventually crash it.
I suspect that it might be caused by an event emitter, but I'm still trying to find out what might cause that.

(note that the app works fine on the WebARonARCore browser)

Tested on iPhone 7, iOS v. 11.3.1 (15E302).
Latest https://github.com/mozilla-mobile/webxr-ios build.

Any ideas where to start looking?

[blairmacintyre]

Not sure where to start. The version in vendor is still showing 0.7.x, I've never tried upgrading to even 0.8.

If you figure it out, I'd love to updating the version here to 0.8.x, but I (alas) don't have time to look into this.

[Danpollak]

I can confirm that the 0.8.1 version of aframe works with aframe-xr - I am using it as dependency and not from vendor. You still need to add three.xr.js, though.

About the 0.8.2 version, I am looking into that - still not sure if the problem lies in the WebXR Viewer or somewhere in between aframe-xr to aframe (or both).

[vincentfretin]

With 0.8.2 I get the following error on WebARonARCore with S8:

aframe-xr.js:1 Uncaught TypeError: Cannot assign to read only property 'enterVR' of object '#<a-scene>'
    at o.wrapSceneMethods (aframe-xr.js:1)
    at Object.module.exports.fireEvent (index.js:120)
    at HTMLElement.value (a-node.js:228)
    at a-node.js:118

This is apparently this line:

aframe-xr/src/systems/xr.js

Line 35 in 50326e4

sceneEl.enterVR = function (fromExternal) {

maybe enterVR is now a read only property in aframe 0.8.2?

[vincentfretin]

This seems to be a random issue on WebARonARCore, I don't have the error anymore... or maybe I changed something, I don't remember.

[vincentfretin]

I updated my last two comments to say the error I saw was on WebARonARCore.
On WebXR Viewer with aframe 0.8.2, I saw the same stacktrace via the react exception catcher because I have some part of the application with react, not the aframe part. Sometime I don't have the stacktrace, but the background is white, sometime it even works and I have a VR button at the bottom, sometimes I have the camera feed and the AR and VR buttons overlapping.

[Danpollak]

I've bisected the aframe commits to find the breaking change and found out that since aframe upgraded the webVR-polyfill from 0.9.40 to 0.10.3 it won't work properly. Still not sure this is the thing that is causing the problem, but I'm getting there!

@vincentfretin during the bisect I've experienced the overlapping ar/vr buttons also, but tapping either won't work. Couldn't see any errors though.

[blairmacintyre]

Glad to see some progress ... sorry I don't have much insight here!

[vincentfretin]

Same here, tapping doesn't work for me on WebARonARCore, no error. Tapping works on WebXR Viewer.

[vincentfretin]

On WebARonARCore, I have touchstart event on window, but no click event on my raycaster entity with aframe master and aframe-xr.
I have raycaster click event working with the same aframe master and aframe-ar (not aframe-xr), so this is probably a three.xr or aframe-xr issue. I already found an issue with component tick function not triggering #18

[vincentfretin]

For the error #17 (comment) on WebARonARCore, it was actually on 0.7.1 when I got this error, not 0.8.2. In aframe 0.7.1, the enterVR, exitVR are using writable: window.debug, in 0.8.2, it is writable: true.

For the click not working with WebARonARCore, with aframe 0.8.0 or 0.8.2, can it be related to a dom element that is above the aframe canvas? I had a similar issue with a full width and height transparent div above the aframe canvas in my project. I'll look closer into webxr-polyfill. Maybe you have some pointers @blairmacintyre

For the issue on WebXR Viewer with aframe 0.8.2 where the background stay white (do you have this behavior @Danpollak ?), it is probably a conflict or race condition between latest webvr-polyfill and webxr-polyfill? Actually I have sometime the camera feed working when I enable some of
aframe-ar code I'm trying to port, if I do window.webkit.messageHandlers.initAR.postMessage(data) myself for example (see code in https://github.com/chenzlabs/aframe-ar/blob/master/src/mozilla-xr-ar.js). From the webxr-polyfill code I read, it may not be a good idea to use window.webkit.messageHandlers.initAR and watchAR myself though, it may lead to other issues I think. I'll try to use updateFrame event instead when porting this code, like in the reticle component.

[vincentfretin]

Ah I'm going somewhere. My intuition about the element above the canvas seems correct. It seems the a-scene element if above the canvas on WebARonARCore.
Looking at the click event with

document.addEventListener('click', function(e) {
        console.log(e.target);
});

without aframe-xr, no logs, with aframe-xr, I have logs saying the click is on a-scene.
With aframe-xr and the following code:

      document.addEventListener('DOMContentLoaded', function() {
        setTimeout(() => {
          console.log(AFRAME.scenes[0].canvas);
          AFRAME.scenes[0].canvas.style.zIndex = 50;
        }, 4000);
      });

the click event doesn't go to document (so no logs) and tapping works correctly, the issue is fixed. I think this has to do with the way the canvas is created for arcore, but I don't really know what is the difference between simple aframe and aframe+aframe-xr though.

[vincentfretin]

With aframe alone (no three.xr), the canvas is created as a child of a-scene. With three.xr and aframe-xr, the canvas is created as a child of div.webxr-sessions which is the second element in body. There is another canvas inside the div.webxr-realities (first element of body) for the camera video frame I think?

[vincentfretin]

So I continued to debug with WebXR Viewer
with

...
<script src="https://aframe.io/releases/0.8.2/aframe.min.js"></script>
<script src="./dist/three-xr.js"></script> <!-- latest version found in the repo -->
<script src="./dist/aframe-xr.js"></script> <!-- latest version found in the repo -->
</head>
<body>
<div style="position:absolute; top: 0; width: 100%; height: 3em; background-color: white; color: black; z-index: 999" id="debugMessage">debug</div>
...

In aframe-xr/src/systems/xr.js I put in cameraActivated the following line to debug:
document.getElementById('debugMessage').innerHTML = navigator.XR;
navigator.XR is actually undefined, this weird because ./dist/three.xr.js should contain vendor/webxr-polyfill.js and so define navigator.XR. So I finally edited my index.html file to include webxr-polyfill like this

    <script src="./dist/webxr-polyfill.js"></script><!-- the file from vendor folder in three.xr.js repo -->
    <!--<script src="./dist/three-xr.js"></script>-->
    <script src="./dist/aframe-xr.js"></script><!--my own build-->

and included three.xr.js by editing aframe-xr/src/systems/xr.js and put the following lines at the top

import WebXRManager from '../../../three.xr.js/src/WebXRManager';               
import WebXRUtils from '../../../three.xr.js/src/WebXRUtils';                   
                                                                                
THREE.WebXRManager = WebXRManager;                                              
THREE.WebXRUtils = WebXRUtils;

and rebuilt aframe-xr.

Now navigator.XR is defined and I go in aframe-xr/src/systems/xr.js initXR, I have two displays, the first supports ar, the second one doesn't support ar. I have the iPad 2018.
The code in THREE.WebXRManager https://github.com/mozilla/three.xr.js/blob/644c8489dd091f02ed9972c8f14f0a57aa405d26/src/WebXRManager.js#L205-L212 verify
if we have 1 ar and 0 vr or 0 ar and 1 vr, so it doesn't work for me, and this.startSession is never called.
If I modify the first condition to

if (arSupportedDisplays === 1 && this.options.AR_AUTOSTART) {

it works properly.

[vincentfretin]

This fix is not enough, WebXRManager this.startPresenting go to the VR branch instead of AR branch because displayVR is defined. startPresenting should have a reality parameter I think.

[vincentfretin]

And in aframe-xr/src/systems/xr.js, disable look-controls on camera to not update the orientation on every tick (because it's enabled now with #18):

    if (this.supportAR && this.data.arAutostart) {
      sceneEl.camera.el.setAttribute('look-controls', 'enabled', false);
    }
    sceneEl.renderer.xr = new THREE.WebXRManager(options, displays, sceneEl.renderer, sceneEl.camera, sceneEl.object3D, this.updateFrame);

Well we should probably enable it back when ending the AR session, similar to what aframe-ar does https://github.com/chenzlabs/aframe-ar/blob/ff7202b838355be603ecd0722093f3c7586f4e08/src/ar-camera.js#L15-L20

@blairmacintyre in sessionEnded

aframe-xr/src/systems/xr.js

Lines 129 to 136 in 50326e4

	sessionEnded: function (data) {
	this.activeRealityType = 'magicWindow';
	this.el.emit('realityChanged', this.activeRealityType);
	if (this.activeRealityType === 'ar') {
	// To show camera on iOS devices
	document.documentElement.style.backgroundColor = '';
	document.body.style.backgroundColor = '';
	}

there is an error no?
Shouldn't it be the following?

sessionEnded: function (data) {
    if (this.activeRealityType === 'ar') {
      // Remove the transparency set in sessionStarted on iOS devices
      document.documentElement.style.backgroundColor = '';
      document.body.style.backgroundColor = '';
    }
    this.activeRealityType = 'magicWindow';
    this.el.emit('realityChanged', this.activeRealityType);

[vincentfretin]

I'll clean up my changes (removing all debug info :)) and do a PR for three.xr.js and aframe-xr to be easier to review.

[Danpollak]

@vincentfretin amazing work!!

I won't be able to test it till Sunday (no iOS available), but it seems like you nailed it! Will give a more in-depth feedback once I get my hands on an iPhone.

[vincentfretin]

For the look-controls disabling and sessionEnded fix, I did it in the commit a24c874 in my PR #18

[vincentfretin]

Oh man I included dist/three-xr.js instead of dist/three.xr.js this is why navigator.XR was undefined. :(

[vincentfretin]

For the two displays issue, see mozilla/three.xr.js#11

[vincentfretin]

For the raycaster click that wasn't working on both WebXR Viewer and WebARonARCore, I did this c0544e7 to fix it.
I believe #18 and mozilla/three.xr.js#11 fixes all the issues we discussed here.