UI buttons not working in AR :(

Help & Support
1

I did a little test here: PlayCanvas | HTML5 Game Engine
Where there’s a white button “Text” in the center of the screen, which when pressed, restarts AR. It works flawlessly. Great.

Now let’s look at a real project: PlayCanvas | HTML5 Game Engine
Open the scene “small_v2”. When you run the project you can select AR from the “Raum” dropdown in the topleft corner of the screen. When AR starts, a new UI shows up with three buttons. However, these can not be interacted with. Why?

I have tried to recreate the same setup as in the other project, but I can’t get it to work.
If I show the buttons outside of AR, they work. The fact that the UI is disabled by default doesn’t seem to be at fault either, because it behaves the same when it is always visible and never toggled.

The buttons even light up when I press them, so clearly the button works, but my code doesn’t seem to register the click.

(For reference, what should happen when you press the “Neu ausrichten” button is that XR should end for now).

What am I doing wrong, I can’t really see what could be at fault for the difference in behaviour here :frowning_face:

2

Looking into this and very confused about whats going on. Still investigating.

3

Looks like it’s related to the camera you are starting AR on. If I startXR with a new camera that I created in the Editor, the button in XR works fine.

Still investigating

4

I’ve narrowed it down to something in your cameraController.js script is causing an issue with the raycasting for the UI button while in AR

5

After commenting out the update code in the cameraController, the buttons work again

// update code called every frame
CameraController.prototype.update = function(dt) {
    /*if (!this.lerping) {
        this.entity.setPosition(this.targetPos);
        this.entity.setEulerAngles(this.targetRot);
    }

    if (this.trailingSpeed.x != 0) {
        this.trailingSpeed.x *= 0.95;
        this.trailingSpeed.y *= 0.95;

        var mouseEvent = {
            dx: 0,
            dy: 0,
            forceUpdate: true
        };
        this.onMouseMove(mouseEvent);
    }*/
};

My best guess is the following: When in XR, the XR manager in PlayCanvas moves the camera in position and rotation based on the data it gets from the phone about where it is in the real world.

All the XR controller inputs (that are needed to interact with the UI buttons) are positioned in the real world based on the phone data.

However, each frame you are moving the camera in this update loop so when it comes to converting to screen space, the XR controller input is no longer in the bounds of the screen.

TLDR, don’t move the camera when in XR :slight_smile:

6

Wow, I would never have guessed that’s the issue.
Thanks a lot for investigating and finding the culprit!