How to Properly Load Custom Skinned Mesh and Animation Data

Help & Support
,
1

Hello,

I’m working with my own custom model format and have already written a parser for it. However, I’m struggling to find documentation on how PlayCanvas expects certain data for loading and animating skinned meshes. Specifically, I’m unsure about:

  1. What coordinate space PlayCanvas uses for bone and animation matrices?
  2. How to properly create a skinned mesh and bind animations to it using code (not the editor)?
  3. Why my mesh gets deformed after binding the skeleton, which suggests an issue with the coordinate space or data being supplied.
  4. How to debug and visualize the bone structure to ensure my bone data is correct. Is there any helper function to display the bones and check how they look?

I learn faster through examples, so example code demonstrating how to create a mesh, skin it, and add animation would be very helpful for me to understand the process.

Here’s my current code for creating the skinned mesh and adding the skin, but when I add the skin to the mesh, it gets broken. Any advice would be greatly appreciated!

function createSkinnedMeshInstance(
    app: pc.Application,
    skin: pc.Skin,
    boneNodes: pc.GraphNode[],
    geo: any,
    name: string
): pc.MeshInstance {
    const mesh = new pc.Mesh(app.graphicsDevice);
    mesh.setPositions(geo.positions.flat());
    mesh.setNormals(geo.normals.flat());
    mesh.setUvs(0, geo.uvs.flat());
    mesh.setIndices(geo.indices);

    // Skinning data
    const boneIndices: number[] = [];
    const boneWeights: number[] = [];
    for (const w of geo.weights) {
        boneIndices.push(...w.bones);
        boneWeights.push(...w.weights);
    }

    mesh.setVertexStream(pc.SEMANTIC_BLENDINDICES, boneIndices, 4);
    mesh.setVertexStream(pc.SEMANTIC_BLENDWEIGHT, boneWeights, 4);
    mesh.skin = skin;
    mesh.update();

    const material = new pc.StandardMaterial();
    material.diffuse.set(1, 1, 1);
    material.update();

    const meshInstance = new pc.MeshInstance(mesh, material, boneNodes[0]);
    const skinInstance = new pc.SkinInstance(skin);
    skinInstance.bones = boneNodes;
    meshInstance.skinInstance = skinInstance;
    return meshInstance;
}

Thanks for your help!

2

So I’ve figured out the creation of the mesh, skinning of the mesh. It is all good now. But I don’t see documentation on how to create animation, add the animation to the skinnedmesh and play a clip. Animation contains multiple clips. Anyone know where I can find those information?

3

I think for this you might need to look at the source code of the engine. A good starting point would be glb parser, you can see how glb gets loaded:

4

Hello,

Thank you for the response. I actually already checked that, its one of the first place I tried to look into. But it’s a lot, is there a simpler example on just creating the animations and playing it after creating the skinnedmesh? I’m good until creating the skinnedmesh, but can’t find any example on what correct classes to use for creating animation. The data I already figured out through the glb parser. My animation data are local matrices per bone per frame. Just need to know how to add it to the skinnedMesh.

Thank you again.

5

@slimbuck is the person the most familiar with animation data - any suggestions here?

6

It’s more about setting up the animations after I already parsed the data. In another library I would:


function createAnimationClips(modelMesh, animData) {
    for (const anim of animData) {
        const tracks = [];
        const frameTime = 1 / 30;

        for (let frameIdx = 0; frameIdx < anim.numFrames; frameIdx++) {
            for (let boneIdx = 0; boneIdx < anim.numBones; boneIdx++) {
                const bone = bones[boneIdx];
                const frameMat = new Matrix4().set(anim.frames[boneIdx][frameIdx]);

                const pos = new Vector3();
                const rot = new Quaternion();
                const scl = new Vector3();
                frameMat.decompose(pos, rot, scl);

                const t = frameIdx * frameTime;
                const track = boneTracks[boneIdx];

                track.posTimes.push(t);
                track.posValues.push(pos.x, pos.y, pos.z);

                track.rotTimes.push(t);
                track.rotValues.push(rot.x, rot.y, rot.z, rot.w);

                track.scaleTimes.push(t);
                track.scaleValues.push(scl.x, scl.y, scl.z);
            }
        }

        for (const track of boneTracks) {
            tracks.push(new VectorKeyframeTrack(`${track.name}.position`, track.posTimes, track.posValues));
            tracks.push(new QuaternionKeyframeTrack(`${track.name}.quaternion`, track.rotTimes, track.rotValues));
            tracks.push(new VectorKeyframeTrack(`${track.name}.scale`, track.scaleTimes, track.scaleValues));
        }

        const clip = new AnimationClip(anim.name, -1, tracks);
        clips.push(clip);
    }

    return clips;
}

and

    const mixer = new AnimationMixer(skinnedMesh);
    const clips = createAnimationClips(skinnedMesh, anims);
    const clip = mixer.clipAction(clips[0]);
    clip.play();

And this is what I’m trying to figure out for playcanvas. Whats the equivalent. Thank you.