I’d like to explore using vertex animation textures in Playcanvas. While I’ve now done a fair amount making custom frag shaders in PC, I’ve yet to do anything on the vertex side. Which shader chunk/s would I need to modify in order to use textures to modify the vertex positions/normals?
This would be the main one to play with:
2 Likes
see MORPHING_TEXTURE_BASED in that file … you’d do something similar
Ah, fantastic thank you!
Follow up question: how can I bring in a texture at 16bit (half float)? Every format I’m trying it seems to convert it to an 8-bit version. It’s going to be tricky to do vertex animations using 8bit textures!
study this file: engine/morph.js at main · playcanvas/engine · GitHub
This uses either float or half float texture to store morph targets, depending on what your system supports.
In the past, I’ve done a VAT implementation using 8bit, and the quality was very good - and it could run about 10k skinned characters on a normal PC. I stored x, y and z offset in rgb, normalized to a bounding box of the skinned mesh. And .w was storing a normal ID, which I then use ti generate the normal on the GPU. It would do 2 texture samples per vertex. One with linear interpolation to get interpolated vertex offset, and one with point sampling to get exact normal index (those cannot be interpolated).
2 Likes
Great, this is really helpful, thanks!
I’ve figured out how to encode/decode a 16-bit float to/from two 8-bit RGB textures. I’m not sure I understand how you would calculate the normals as you’re suggesting - do you have a resource which describes that process?
Alternatively I could encode the normals into another texture, in which case which shader chunk would I use to set them directly using a texture? Edit: I just found normal.js which seems to be the place.
I’ve hit another wall I’m afraid.
I’m trying to figure out how to actually access the vertex id inside the shader. Looking at how it’s done in transform.js, there is this line:
float vertexId = morph_vertex_id;
When I search for how morph_vertex_id is defined, I find this in lit-shader.js:
// vertex ids attributes
this.attributes.morph_vertex_id = SEMANTIC_ATTR15;
code += "attribute float morph_vertex_id;\n";
So it seems this is being set according to some value which has been put in SEMANTIC_ATTR15.
Which then brings me to these lines in morph.js:
// create vertex stream with vertex_id used to map vertex to texture
const formatDesc = [{ semantic: SEMANTIC_ATTR15, components: 1, type: TYPE_FLOAT32 }];
this.vertexBufferIds = new VertexBuffer(this.device, new VertexFormat(this.device, formatDesc), ids.length, BUFFER_STATIC, new Float32Array(ids));
So I tried essentially copying this code, replacing ids.length with the number of vertices in my mesh, and the ids array with an array of all the vertex ids (in my case 0-507).
But I’m getting errors trying to use SEMANTIC_ATTR15 and BUFFER_STATIC because I can’t import things from a module in a Playcanvas script. So now I’m confused about whether I’m going about this in the right way at all.
How can I get the vertex id during the transform vert shader?
For the normal generation, I used Fibonacci sphere
See the code we have in PlayCanvas: engine/random.js at d3e1b4a32c3eced83fb464e29aeef53fc9e5704c · playcanvas/engine · GitHub
On the cpu during VAT generation, I’d generate all 255 normals using that code, compare the normal I’m encoding to find out the nearest of those, and store the index in .a of the texture.
During runtime, I’d simply use the index and the same code (running in a shader) to evaluate a normal.
Num points would be 255, start and end 0 and 1, the result is pretty fast to run code.
1 Like
those are all exported into pc name space. So this should work:
pc.SEMANTIC_ATTR15
pc.BUFFER_STATIC
pc.TYPE_FLOAT32
but this bit is tricky ;
this.attributes.morph_vertex_id = SEMANTIC_ATTR15;
and I’m not sure I have a solution here. Internally, when we generate the shader, we make it use ATTR15, but this is not at the moment available to you I suspect. This is the missing bit: Custom attributes on material's shader definitions. · Issue #3485 · playcanvas/engine · GitHub
As a workaround for now could be to use gl_VertexID - but note that this is WebGl 2 only features. See here: WebGL2 Drawing Without Data
1 Like
I did try using gl_VertexID, but even though I have ‘Prefer WebGL 2.0’ enabled in the rendering settings I’m getting this error:
ASSERT FAILED:
Vertex shader attribute "gl_VertexID" is not mapped to a semantic in shader definition.
So are you suggesting I should be able to potentially make the code like this?
// create vertex stream with vertex_id used to map vertex to texture
const formatDesc = [{ semantic: pc.SEMANTIC_ATTR15, components: 1, type: TYPE_FLOAT32 }];
this.vertexBufferIds = new VertexBuffer(this.device, new VertexFormat(this.device, formatDesc), ids.length, pc.BUFFER_STATIC, new Float32Array(ids));
Essentially using pc.SEMANTIC_ATTR15 and pc.BUFFER_STATIC.
And then potentially pass that to the shader as a parameter?
PS. That Fibonacci sphere is a very cool idea, thanks!
how did you use `gl_VertexID’ in the shader? I would not expect this error.
Ah OK, if I understand correctly I also need to put this at the top of the shader:
#version 300 es
However, obviously if I put that at the top of the transform shader, it is not actually at the top of the entire shader which gets generated. So I need to figure out where the start of the vert shader is, which file, and put it there. I think…?
Spector JS is your friend … install it as a Chrome plugin. Then you capture your code and see the whole shader generated.
By the way, all shader on GL2 already have that in them.
#version 300 es
Your problem is probably that you declare
`attribute gl_VertexID’ or something similar, which you should not.
I was just trying to test whether the value was even pulling through with these lines:
float vertexId = float(gl_VertexID);
float vertexCount = 508.0;
float u = vertexId / vertexCount;
vec2 testUV = vec2(u, 0.5);
vec3 vatPos01 = texture2D(vatPositionTexture01, testUV).xyz;
Edit: though now I’m wondering if it’s because there isn’t actually a texture in vatPositionTexture01 yet? I need to hook that up.
Edit 2: Nope, that didn’t make any difference. Seems as soon as I try to hook that value into the uv for a texture lookup it breaks.
This is the full error I’m getting any time I actually try to use gl_VertexID to do anything at all:
Vertex shader attribute "gl_VertexID" is not mapped to a semantic in shader definition.
[[playcanvas-1.56.0.dbg.js:15808]](https://code.playcanvas.com/playcanvas-1.56.0.dbg.js): Cannot read properties of undefined (reading 'substring')
TypeError: Cannot read properties of undefined (reading 'substring')
at new ShaderInput (https://code.playcanvas.com/playcanvas-1.56.0.dbg.js:15808:13)
at WebglShader.postLink (https://code.playcanvas.com/playcanvas-1.56.0.dbg.js:15976:22)
at WebglGraphicsDevice.setShader (https://code.playcanvas.com/playcanvas-1.56.0.dbg.js:18353:49)
at ForwardRenderer.renderForwardInternal (https://code.playcanvas.com/playcanvas-1.56.0.dbg.js:25292:42)
at ForwardRenderer.renderForward (https://code.playcanvas.com/playcanvas-1.56.0.dbg.js:25426:11)
at ForwardRenderer.renderRenderAction (https://code.playcanvas.com/playcanvas-1.56.0.dbg.js:25948:13)
at ForwardRenderer.renderPassRenderActions (https://code.playcanvas.com/playcanvas-1.56.0.dbg.js:25876:13)
at RenderPass.execute (https://code.playcanvas.com/playcanvas-1.56.0.dbg.js:25807:15)
at RenderPass.render (https://code.playcanvas.com/playcanvas-1.56.0.dbg.js:22604:11)
at FrameGraph.render (https://code.playcanvas.com/playcanvas-1.56.0.dbg.js:28382:24)
I have a fix which we should release as part of 1.57.1 sometimes next week most likely:
In the meantime, you could build the engine yourself with the fix and use that, that’s pretty easy.
1 Like
Fantastic, thank you so much!