Clicking UI buttons when gazing using lookCamera

Help & Support
1

Unfortunately UI element buttons are not working in a (3D) screen when locking the mouse and use it for gazing.

My goal is to let the user click a button while locked in.

I am using the lookCamera script to lock the mouse and look around. Idealy I want to register a click in the middle of the screen when clicking and while locked in. I have looked around the forum but the only solution I came across was to disable the mouse lock. This is however not what I am looking for.

How do I let the user click while locked in, in the middle of the screen, no matter where the mouse was on the screen when starting to lock the mouse?

A nice extra would be to make hovering work as well from the center of the screen.

I made a small demo project:
https://playcanvas.com/editor/project/1308170

2

Could you use ray casting?

3

Hey @Codeknight999 ,

Thanks for your reply.
I could use ray casting by disabling the the button component and add a rigidbody component. Then scripting the hover and click states and logic.
But wouldn’t that be an intensive way to go about this?

4

I don’t think so, especially in your scenario. Another option would be using the player looking at a button and then clicking it or something like that

5

You can use the Frame Buffer Picker example to pick entities with meshes. Looking at [SOLVED] Picking 3D/2D Screens/UI elements?, I assume that OP could pick elements as well.

Just make sure that you change the picker’s x and y and you check if the entity is an element or whatever suits your needs.

Example changes:

PickerFramebuffer.prototype.onSelect = function (event) {
    var canvas = this.app.graphicsDevice.canvas;
    var canvasWidth = parseInt(canvas.clientWidth, 10);
    var canvasHeight = parseInt(canvas.clientHeight, 10);

    var camera = this.entity.camera.camera;
    var scene = this.app.scene;
    var picker = this.picker;

    picker.resize(canvasWidth * this.pickAreaScale, canvasHeight * this.pickAreaScale);
    picker.prepare(camera, scene, this.layers);

   // Center of picker
    var selected = picker.getSelection({
        x: Math.floor(picker.width / 2),
        y: Math.floor(picker.height / 2)
    });

    if (selected.length > 0) {
        var entity = selected[0].node;
        if(entity){ // You can check if it's an element or the type of element, etc
            // Do what you want
        }
    }
};
1 Like