[SOLVED] Raycast not working as expected in AR mode

Help & Support
1

I have a simple FPS setup where clicking a button fires a raycast straight out of the camera in the direction the player is currently facing. I am creating a red box entity at the place the raycast ends for debugging. It works as expected normally but when I enter AR mode the red box entity does get created where I expect it to be, it gets created right next to the player.

When I enter AR mode and tap on the screen to fire the raycast, the cube get spawned at the right of the player and it pushes the player a bit to the left.

Demo:
https://gamerwael.github.io/playcanvas-xr-fps/
Top Left button to shoot
Top Right button to enter AR
Bottom Right Button to jump

GitHub Source:


(Most of the relevant code is in index.html and touch-input.js)

My shoot function:

TouchInput.prototype.shoot = function() {
    console.log("shooting");

    var screenCenterX = (this.app.graphicsDevice.width / 2);
    var screenCenterY = (this.app.graphicsDevice.height / 2);

    var from = this.camera.camera.screenToWorld(screenCenterX, screenCenterY, this.camera.camera.nearClip);
    var to = this.camera.camera.screenToWorld(screenCenterX, screenCenterY, this.camera.camera.farClip);


    var results = this.app.systems.rigidbody.raycastAll(from, to);
    //var result = this.app.systems.rigidbody.raycastFirst(from, to);

    if (results.length > 0) {
        var temp = results;
        //Sort results by proximity
        results.sort(function(a, b) {
            var a1 = a.point.distance(from);
            var a2 = b.point.distance(from);
            return a1 - a2
        });

        result = results[0];
        //If first raycast result is the player itself then select second result
        if (results[0].entity.name == "characterController")
            result = results[1];
        else
            result = results[0];

        var pos = result.point;

        var entity = new pc.Entity("Box");
        entity.name = "Cube";

        entity.addComponent("model", {
            type: 'box',
        });
        entity.addComponent("collision", {
            type: 'box',
        });
        entity.addComponent("rigidbody", {
            type: pc.BODYTYPE_STATIC,
        });
        entity.setLocalPosition(
            pos.x,
            pos.y,
            pos.z
        );
        entity.model.meshInstances[0].material = this.red;

        this.app.root.addChild(entity);
    }
};

How I’m handling AR:

xrBtn.button.on('touchstart', function(e) {
    if (app.xr.isAvailable(pc.XRTYPE_AR)) {
        camera.camera.startXr(pc.XRTYPE_AR, pc.XRSPACE_LOCAL);
    }
}.bind(this));

app.xr.input.on('add', function(inputSource) {
    characterController.script.touchInput.shoot();
}.bind(this));
2

Not sure, but it looks like the center of the screen is different between AR on and off modes. For example, device pixel ratio can affect the final resolution, so dividing by 2 is not enough to get a true center on all devices. Perhaps, you could look into that.

1 Like
3

Thanks a lot @LeXXik ! That does seem to be the problem. I might have to calculate the screen center differently by taking into consideration the screen resolution for non-AR mode and the device’s camera’s resolution for AR mode.

4

I can’t seem to figure out a way to solve this. I seem to have gotten it to work on my phone by guessing values and then fine-tuning them until it looks like its shooting straight, but this obviously isn’t an optimal solution and I doubt that it will work on other devices(It could work if the XR session uses a fixed height and width for all devices irrespective of camera resolution, but I’m not sure if that’s how it works). So there’s a couple of problems here:

The numbers that I got don’t make any sense. I expected them to be the same as my cameras resolution but they’re not.

There doesn’t seem to be any way to access the resolution of the device’s camera or the height and width of the XR session.

I’m not sure how to proceed with this. Any help will be greatly appreciated!

This is how I’m doing it right now. The values seem to work both in regular as well as in AR mode.

var screenCenterX = 391/2;
var screenCenterY = 768/2;

var from = this.camera.camera.screenToWorld(screenCenterX, screenCenterY, this.camera.camera.nearClip);
var to = this.camera.camera.screenToWorld(screenCenterX, screenCenterY, this.camera.camera.farClip);
5

Does the ray always start from the middle of the screen? If so, can you do a raycast from the camera world position instead of using screen to world?

1 Like