Support for multiple UV channels

[heretique]

Hi all,

I’m new to PlayCanvas and I’m wondering if it is possible to use meshes with multiple UV channels in PlayCanvas. I stumbled on this old post that hasn’t been answered More uv channel so it not clear to me if this is possible or not.

I need to customize some stylized characters with different eyes and mouths that are painted on the face and although I could easily achieve that by using separate “eyes” and “mouth” meshes floating slightly above the face it is not optimal for my use case.

Instead, for my face mesh I would like to use two additional UV channels to achieve that, the issue is that if I inspect the .GLB file created upon importing it in PlayCanvas I see only two UV channels instead of my four UV channels. I made a comparison between the PlayCanvas generated .GLB file and another one that I exported directly from Blender and the Blender one has the right amount of UV channels.
I’ve also tried to upload directly the .GLB file exported by Blender but that one was not properly imported, the meshes are not visible in the Inspector.

I’m wondering why the additional channels are not imported because I see that the VertexFormat supports up to 8 channels for the texture coordinates semantic (SEMANTIC_TEXCOORD7).

As an alternative I was also thinking of using the vertex colors as UV coordinates, I even created a small Blender script that copies the two additional UV channels into the RGBA components of the vertex colors but in PlayCanvas the vertex colors are TYPE_UINT8 and that will greatly affect the precision when sampling I assume.

Is it possible to “hook” into the import task somehow, even though the Editor API is not open or any other way to have my imported meshes support multiple UV channels (except the obvious way of loading the asset myself and programmatically create the mesh myself)?

PS: I think that a lot of special effects could benefit from a scriptable import pipeline that allows customizable attributes for the vertex description

[mvaligursky]

We had some fixes done few months ago in our fbx->glb pipeline to support all UV channels. Perhaps there’s still some bug left there. Are you sure all of them were exported to your fbx file? If you could create an issue here https://github.com/playcanvas/editor/issues with a simple test fbx which does not convert to glb correctly, we’d investigate in when time allows.

You can use your glb that contains all channels. You won’t see the character in the editor, but it will work fine during runtime, so that’s probably the best way for you to go in the meantime. There are some tutorial projects on how this is done, forum search should find some info.

It’s not possible to hook into import on this level, this is a c++ project doing the conversion on our backend.

We also plan to import glb files directly into Editor, but this is not scheduled for few more months.

[heretique]

@mvaligursky thanks for clarifying things. I’m pretty sure the FBX has all four channels because I re-imported it back in Blender to check that (I couldn’t find a tool to inspect the FBX, the old FBX Review from Autodesk doesn’t show this kind of info unfortunately).

@mvaligursky

You can use your glb that contains all channels. You won’t see the character in the editor, but it will work fine during runtime, so that’s probably the best way for you to go in the meantime. There are some tutorial projects on how this is done, forum search should find some info.

When I used my glb, it disappeared from the instantiated template and I assumed it won’t work, it is good to know that there is this alternative, thanks.

I’ll create an issue for the import with a test FBX file and try using my own glb for now and see how that works.

[mvaligursky]

See this example

Added examples of loading GLB assets in Editor

GLB: https://developer.playcanvas.com/en/tutorials/loading-gltf-glbs/ Draco compressed GLB: https://developer.playcanvas.com/en/tutorials/loading-draco-compressed-glbs/

And also some engine examples

• one on how you can modify geometry on CPU PlayCanvas Examples

and some example on how to access materials

Loading glb file with KHR_materials_transmission extension

When you load glb at runtime and create an instance of it, for example this way: const house = asset.resource.instantiateRenderEntity(); You can then find and change all materials this way: const renders = house.findComponents("render"); renders.forEach((render) => { render.meshInstances.forEach((meshInstance) => { meshInstance.material.blendType = pc.BLEND_MULTIPLICATIVE; meshInstance.material.update(); }); }); is this what you need?

[heretique]

Hi all, I finally had some time to continue investigation on this subject.
Although the GitHub issue has been fixed and the PlayCanvas importer properly imports and creates multiple UV channels (above 2) the engine doesn’t seem to support this at all.
I logged the VertexFormat for the Mesh of an imported 3D model and the format shows only two UV channels and the vertex information for the additional UV channel is dropped. I’m seeing that the stride only accounts for the vertex buffer information inside the Mesh and not the whole info in the .GLB file.

Looking at the implementation of VertexFormat::init method I also noticed that the semantic texture coordinates for the UV channels from 2 to 5 are used to define the world transform matrices for hardware instancing in the standard material if I’ve figured this out correctly (as a float4 for each channel, I guess rows or columns of the matrix).
On the other hand I noticed that the collectAttribs utility function found inside src\graphics\program-lib\utils.js uses a dictionary to generate shader attribute declarations that include attributes for up to 8 texture channels (this seems to be used only for the particle shader though)

const attrib2Semantic = {
    vertex_position: SEMANTIC_POSITION,
    vertex_normal: SEMANTIC_NORMAL,
    vertex_tangent: SEMANTIC_TANGENT,
    vertex_texCoord0: SEMANTIC_TEXCOORD0,
    vertex_texCoord1: SEMANTIC_TEXCOORD1,
    vertex_texCoord2: SEMANTIC_TEXCOORD2,
    vertex_texCoord3: SEMANTIC_TEXCOORD3,
    vertex_texCoord4: SEMANTIC_TEXCOORD4,
    vertex_texCoord5: SEMANTIC_TEXCOORD5,
    vertex_texCoord6: SEMANTIC_TEXCOORD6,
    vertex_texCoord7: SEMANTIC_TEXCOORD7,
    vertex_color: SEMANTIC_COLOR,
    vertex_boneIndices: SEMANTIC_BLENDINDICES,
    vertex_boneWeights: SEMANTIC_BLENDWEIGHT
};

Given all this, should I expect the Mesh object to have all the UV channels from the .GLB model, it is not totally clear to me, or do I need to load the .GLB myself and manually create the Mesh definition?

[mvaligursky]

I think this might be the reason for those uv coordinates being ignored by the glb-loader

playcanvas/engine/blob/b78f62c1e68f867e8f85c7748db80ab7d30ec595/src/resources/parser/glb-parser.js#L139-L148

and there’s this that is relevant as well https://github.com/playcanvas/engine/issues/3485

[heretique]

Hey @mvaligursky is this something that will be fixed inside the glb parser? Should I log an issue on the github repo?

[mvaligursky]

I did the change: https://github.com/playcanvas/engine/pull/3591
It should be released perhaps next week, not completely sure yet.

If you want to use those in the customized standard shader, you will likely still need something along these lines:
https://github.com/playcanvas/engine/issues/3485

If you want completely custom shader, that should be ok to do.

[dadiaar]

Just tested this by importing a cube with 3 UVs into the editor, .fbx and .glb (I reimported both objects into Blender to make sure they had the 3 UVs)

The material only let’s me to choose between UV0 and UV1, so at least the editor doesn’t look like accepting that, not sure about the engine.

The glb importer seems to be able to accept the 8 UVs, but is the only time the “TEXCOORD_2” reference is in the repository (not being used?)