Mouse Click on Model and Map To Texture

Help & Support
1

I’m working on a product builder. The model is a jacket with multiple Mesh Instances. However, there is only one Material which comes from two textures (a Diffuse and a Normal) that is applied to all the mesh instances. The Material maps out correctly to each MI and it looks great.

Where I’m stuck is when a user clicks the model I want to update the Diffuse texture at the corresponding spot. What I’m missing, and having a very hard time tracking down (I’m newish to 3D coding so I may just not know the terms to search for), are the following:

  1. How to capture the position on the model where the user clicked

  2. How to (reverse) calculate that mouse position to the corresponding position in the material / texture. Given that there are multiple mesh instances, a click even one pixel adjacent could relate to a total different MI and Material map.

I have a feeling the answer, at least for #2, is very obvious so thank you for your patience. I greatly appreciate any help.

2

You can probably use the functionality of the Picker. See the API:
https://developer.playcanvas.com/api/pc.Picker.html

and an example:
https://playcanvas.github.io/#/graphics/area-picker

This gives you a meshInstance that was picked.

1 Like
3

Thank you for the tip. I’m not sure why but that wasn’t working for me. However, this example did help:

https://playcanvas.com/project/436809/overview/entity-picking-without-physics

So I now have the hit position (on the mesh instance) in world space. I’d be grateful for insights into how to take the pc.Vec3 and find the position on the 2D texture from which the material is made.

I dynamically build the diffuse texture with multiple layers, then update the material, so the “response” to the click needs to appear on the texture at the spot clicked but in the middle of some of those dynamically drawn texture layers. (drawn to an image buffer)

4

You’d need to get all vertex positions of that mesh instance, and also all indices (which means triangles) for the mesh instance. See how this can be done here: PlayCanvas Examples

The API is this: Mesh | PlayCanvas API Reference

Then you need to loop over all triangles (3 elements of indices array a time), get referenced 3 Vec3 positions for the triangle, and find out if your point is inside that triangle.

Then you would convert the position of the point to barycentric coordinates (within the triangle), and use that to look compute UV coordinate of the pixel based on UV coordinates of vertices.

Some math involved, but I don’t see any ready to use solution.

1 Like
5

Thanks for that lead. Just knowing to look at the UV coordinates is a good help. Also this demo:

https://developer.playcanvas.com/en/tutorials/character-damage-demo/

Gives me another angle to consider. I’ll dig in and try to figure this out. thanks.