Paint on Object at runtime(UV Painting)

[Shubham_D]

Hello everyone,

I want to make a project in which I can simulate a spray paint on the object at real time. So, can anybody suggest me how I can do UV paint inside the Play Canvas.

I’m also putting a link to the asset video, we use in unity for similar kind of task -
https://www.youtube.com/watch?v=Bsj8lax_kNU

Please have a look at it, do let me know we anybody knows how to do it here.

Thanks

[Leonidas]

Hi @Shubham_D, that’s cool!

Though there isn’t a Playcanvas API to do UV painting on runtime based on mouse click coords.

You will have to write a shader to compute the new UV coordinates to update the material maps.

[OmarShehata]

I’m trying to do the same thing, and I am able to accomplish this by creating a 2D canvas, drawing onto that, and using that as the material for my mesh. It works but it’s very slow due to uploading of the texture to the mesh every frame.

My canvas size is 1024x1024. If I set it to really small (like 100x100) it works very smoothly, but then I don’t have enough precision. Is there a better way to do this that could be done faster?

Edit: it runs a lot smoother when I try it on a device with a dedicated GPU - so I guess this is just a particularly slow operated on integrated graphics cards?

[Leonidas]

Hi @OmarShehata,

Using a 2D canvas to draw on and grab the resulting pixels (getImageData) to update a pc.Texture indeed will work. But as you found out it’s a very expensive operation, usually not meant to be executed per frame. Unless the cost (time it takes) per frame is something that your app can live with.

Usually to do something like this in the most performant way you will use pixel shaders. To offset in real-time the coordinates of a texture with almost zero performance cost.

[OmarShehata]

Just to make sure we’re on the same page, I’m not doing a getImageData call (unless PlayCanvas does this under the hood?) I first set up my texture with a 2D canvas source like this:

var texture = new pc.Texture(this.app.graphicsDevice, {
        format: pc.PIXELFORMAT_R8_G8_B8_A8,
        autoMipmap: true
});
texture.setSource(canvas);

Then I draw into the canvas, and update it with:

ctx.restore();
this.texture.upload();

@Leonidas

Usually to do something like this in the most performant way you will use pixel shaders. To offset in real-time the coordinates of a texture with almost zero performance cost.

Do you mean each frame I would pass a mouse position to this fragment shader as a uniform, and then this shader would be drawing into a texture, that I can then use as a uniform to finally render on my object?

[Leonidas]

@OmarShehata

Then I draw into the canvas, and update it with:

ctx.restore();
this.texture.upload();

Yes, Playcanvas uses internally a method to copy the current contents of the canvas and upload them to the GPU. This can be expensive, but you can profile this method and see if this is ok for your app:

playcanvas/engine/blob/3d2e7d431ea2eaf38d81a0210c4dcf0e4b51c8bc/src/graphics/device.js#L1833

@OmarShehata

Do you mean each frame I would pass a mouse position to this fragment shader as a uniform, and then this shader would be drawing into a texture, that I can then use as a uniform to finally render on my object?

Yes, you got it right. You can take a look at shaders on shadertoy.com, many use the mouse coordinates to create interactive effects.

A pixel shader is already drawn as part of a material so you can use it automatically on any model in your scene.

[FBplus]

@Leonidas Hi, Leonidas
As you mentioned here:

@Leonidas

Usually to do something like this in the most performant way you will use pixel shaders. To offset in real-time the coordinates of a texture with almost zero performance cost.

I want to implement a shader like this but I am new to shader programming ,after a period of learning,I’v known how to change the color of specific pixel,but have no idea how to fill the whole object with this color.I guess it has something to do with the uv coordinates?but how?is this difficult to implement?

my test project is here:
https://playcanvas.com/project/704805/overview/leran-shader

Can you give me some advice？thanks so much

[Leonidas]

Hi @FBplus, nice test best for this shader!

This is a bit advanced, haven’t done it myself. Though you are in luck, take a look at the new character destruction tutorial. It’s doing something quite similar to “paint” the damage on a texture and project that later on the model.

https://developer.playcanvas.com/en/tutorials/character-damage-demo/

[FBplus]

@Leonidas Oh,It looks awsome!I hope I can find the solution,thanks

[vgupgrade]

wow this is very cool, where can i rescue the data of a painting to save an xml, json or DB? it is possible?

[vgupgrade]

congratulations

[Mustafa_Ali]

Hi @FBplus & @Leonidas ,

I forked the UV painting project but can’t get it to work, I did some shader changes to make it up to date but still of no use, I don’t see a single error, it still isn’t working, any help would be appreciated.

[mvaligursky]

There’s a simple example that could be useful:
https://playcanvas.github.io/#/graphics/paint-mesh

[Mustafa_Ali]

Oh and I see this one too,
https://playcanvas.github.io/#/graphics/painter

This is great, Thank you @mvaligursky !