Conversion of Mediapipe AI coordinates to match Playcanvas perspective camera FOV

Help & Support
1

Hi
I’m sending a video stream from my MacBook webcam to Mediapipe AI and getting landmarks back.
The x,y coordinates are normalized based on the video size.

I then open the video stream as fullscreen background in Playcanvas.
My goal is to place an entity right onto the identified landmark.

With the camera set to orthogonal and a height of 1 it’s working with the code below.
But when I switch to the perspective camera, I can’t find a solution so far to get it work.

Any help would be appreciated.

const normalizedX = results.multiHandLandmarks[0][14].x;
const normalizedY = results.multiHandLandmarks[0][14].y;
const playcanvasX = (normalizedX - 0.5) * 2; 
const playcanvasY = (normalizedY - 0.5) * 2 * self.video_p; //(videoHeight / videoWidth) 
//
const finalX = playcanvasX * self.window_p; //(window.innerWidth / window.innerHeight)
const finalY = playcanvasY * self.window_p;
/*
                const cameraHeight = 10;
                const cameraFOV = 18.6;
                var distanceFromCamera = cameraHeight / Math.tan((cameraFOV / 2) * (Math.PI / 180));
*/
window.globalsModels.globalModel01.setPosition(new pc.Vec3(finalX * -1, 0, finalY));