I’m sending a video stream from my MacBook webcam to Mediapipe AI and getting landmarks back.
The x,y coordinates are normalized based on the video size.
I then open the video stream as fullscreen background in Playcanvas.
My goal is to place an entity right onto the identified landmark.
With the camera set to orthogonal and a height of 1 it’s working with the code below.
But when I switch to the perspective camera, I can’t find a solution so far to get it work.
Any help would be appreciated.
const normalizedX = results.multiHandLandmarks.x; const normalizedY = results.multiHandLandmarks.y; const playcanvasX = (normalizedX - 0.5) * 2; const playcanvasY = (normalizedY - 0.5) * 2 * self.video_p; //(videoHeight / videoWidth) // const finalX = playcanvasX * self.window_p; //(window.innerWidth / window.innerHeight) const finalY = playcanvasY * self.window_p; /* const cameraHeight = 10; const cameraFOV = 18.6; var distanceFromCamera = cameraHeight / Math.tan((cameraFOV / 2) * (Math.PI / 180)); */ window.globalsModels.globalModel01.setPosition(new pc.Vec3(finalX * -1, 0, finalY));