Hello,
I’m trying to build an AR project / proof of concept where user can tap on the screen and have different items appear (and stay). Example of what I’m talking about: place-ground.
Since im very new, and this is a bit overwhelming, I’ve decided to divide everything into smaller milestones, and here they are:
- Create a simple project with couple of primitive entities.
- Implement something like an “event listener” that would console log entities that I click on.
2.2 Do something with the entity once it is clicked, e.g. change its color. - Have the event listener spawn a new entity, rather than console logging an existing entity.
- Have all of the above work on a phone (when user taps on the screen).
- Have all of the above work in an AR environment.
So, I’m currently at #2 and I’ve been sitting for few hours unable to progress.
Having looked at the forums and the docs/tutorials sections, I see that there are at least two ways of selecting an entity:
- Using collision picking (raycasting).
- Using frame buffer picking.
I’ve come to the conclusion that it would be beneficial for me to know both ways, so I started with the latter.
I am following this tutorial: entity picking, and although some of the in the snippets seem deprecated vs what the docs say, I’ve managed to get it to work.
Here is my current project: /overview/augmented-reality-basics
At this stage, could anybody kindly help me understand in plain English how the whole frame buffer picking works?
Following the documentation:
- What is a “pick buffer”?
- Why is 1024 width and 1024 height hardcoded in the tutorial (pick buffer’s width/height)?
- How does the getSelection method work exactly?
3.1 I know it has 2 required and 2 optional arguments: x, y = the left and top edges of the “rectangle” and width, height of the rectangle. But what it the rectangle and how is it drawn? Is it something like a “selection box” in RTS games where you can click and drag your mouse to select multiple units?
This is the most difficult part for me right now; since I cannot image / don’t know how the virtual “rectangle” is drawn, I don’t know what I’m really doing in the code.
Here is my code so far:
(console logs are just for debugging purpose).
// jshint esversion: 6
const pickViaFrameBuffer = pc.createScript('frameBuffer');
pickViaFrameBuffer.prototype.initialize = function () {
const mouse = this.app.mouse;
this.picker = new pc.Picker(this.app, 1024, 1024);
mouse.on("mousedown", this.handleClick, this);
};
pickViaFrameBuffer.prototype.handleClick = function(e) {
const canvas = document.querySelector('#application-canvas');
const canvasWidth = canvas.clientWidth;
const canvasHeight = canvas.clientHeight;
const camera = this.entity.camera;
const scene = this.app.scene;
const picker = this.picker;
picker.prepare(camera,scene);
const selection = picker.getSelection(e.x, e.y, canvasWidth, canvasHeight);
console.log(`mouseX: ${e.x}, mouseY: ${e.y}` );
console.log(`picker width: ${picker.width}, picker heigth: ${picker.height}`);
selection.forEach(meshInstance => console.log(meshInstance ? meshInstance.node.parent.name : "undefined element"));
};
This is how it works right now:
I want to have it so I can click on an entity and log its name/details/whaterver.
The way it is done in the tutorial, it seems like the first MeshInstance is picked from the array of MeshInstances.
if (selected.length > 0) {
// Get the graph node used by the selected mesh instance
var entity = selected[0].node;
(...)
I think I’m fine with this solution, but I would appreciate if somebody could kindly explain to me how this method works.
Also, I’m curious as to why is there always one undefined element in the array. It picks up the ground, sphere, box, capsule and the cone, but also an undefined element and I don’t know what that is.
Apologies for the lengthy post and I hope my questions although very basic are still appropriate for these forums.
