[SOLVED] Transfer animation between objects

[modelsbymike3d]

I am working on a project where I want to import a ReadyPlayerMe avatar and then animate it. So far I have the import portion working and I am able to add the avatar to the scene. I am doing this all via JavaScript.

I am stuck on the animation part though. I’ve imported a Mixamo animation and if I manually add a character to the scene it is animated. What I want to do is apply the animation to my ReadyPlayerMe avatar after it is imported.

Initially I thought renaming the bones to match the Mixamo hierarchy would work, but that does not seem to work. I’ve done some Googling, but this is my first time working with PlayCanvas so I don’t know if the answer isn’t out there or if I don’t know enough to know what to be searching for.

How do I apply the Mixamo animation to the imported avatar via JavaScript? The bone hierarchy is the same (except for the names) so I think the animation should be able to transfer pretty smoothly in theory.

Project link: PlayCanvas 3D HTML5 Game Engine

[yaustar]

@Saad_Haider Did you work with Ready Player Me avatars before?

[yaustar]

@modelsbymike3d , I see some of the problem. The code is creating a render component hierarchy and that is not compatible with the animation component.

Either use the older model component via the function: https://api.playcanvas.com/engine/classes/ContainerResource.html#instantiatemodelentity

Or use the newer anim component

AnimComponent | Engine API Reference - v2.7.7

eg

        renderRootEntity.addComponent('anim', { activate: true });
        renderRootEntity.anim.addLayer('Base');
        renderRootEntity.anim.baseLayer.assignAnimation('idle', self.animation.resource, 1, true);

Trying to work with your example project to see if I can get it working

[Saad_Haider]

Yes, I have integrated them in one of my projects, I was using the simple Anim component and just swapping the default model out and replacing it with the dynamically loaded model, just be careful about the caching issue, because the link will be the same(from readyplayerme) so the browser will return the cached model even if the user changes a cosmetic on the model, so to overcome it, add something in the link arguments like “id=2312321” some random number so the cached model is not returned.

[modelsbymike3d]

That might work better for my use case. So to make sure I understand, I add a placeholder avatar with the animation to my project, then at runtime I can read whichever one I want and swap out just the model?

[Saad_Haider]

Yes that is correct, you can also download the GLB from the readyplayerme and convert it to FBX and use it as a placeholder.

[modelsbymike3d]

I am still not able to get it to work. I wasn’t sure how to swap out the mesh and materials, so I tried going the route of copying the animation from the placeholder to the imported model. I created a super simple animation for the placeholder so all the bones should match between the two.

I tried both the anim and animation components, but I’m having a hard time figuring out what exactly I need to do. Same project, just edited the script.

[yaustar]

I’ve forked the project and had a closer look

TLDR There are differences between the two hierarchies of the mixamo model and the Ready Player Me

Mixmamo

Ready Player Me

The RootNode entity is important here as that is used for the path for animating the nodes.

There are also other differences where not all all the node names match and more importantly, some nodes are different scales.

I managed to get it somewhat working but as a very tiny person here: https://playcanvas.com/editor/scene/1239928

To be honest, your best bet here is to export animations from Mixamo using the Ready Player Me rig as detailed here: Mixamo Animations for UE4 - Ready Player Me

The two hierarchies between the default mixamo model and the Ready Player Me just don’t match up.

[modelsbymike3d]

Thank you so much for that info! I didn’t realize the bone structure was different; I misunderstood their documentation.

And thanks for sharing your fork! I got the animation applied to my own character now. The x rotation is off by 90 degrees, but that was easy enough to fix by rotating the parent entity.

I’m super excited to keep building with PlayCanvas!

[Saad_Haider]

Why not upload the default Readyplayerme on Mixamo, let Mixamo configure the animation to the bone structure of the avatar of Readyplayerme? AFAIK I did that for my integration to make the animations compatible.

[modelsbymike3d]

My idea is to create some sort of scene with an animation and let each user import their own ReadyPlayerMe avatar. I figure it would be easiest to not mess with the avatar bone structure at all.

[Saad_Haider]

I do the same thing: (The user can choose any kind of avatar, and the animations from Mixamo work fine on all of them.

Saad Haider (saadhaider2)

as I said, you can follow these steps to achieve it:

1. Upload default avatar to Mixamo.
2. Mixamo will output animation which is compatible with the bone structure of the avatar and since all readyplayerme have same bone structure, the animations will work perfectly fine for all avatars.
3. After loading the avatar as GLB from their link, just beware of the cache (since link is always same), so to bust it use some argument in the link.
4. Replace the loaded glb (model) with the model in your project and the animation would run perfectly.

I dont have a sample project for the above steps but will I make an example of readyplayerme integration with Playcanvas sometime soon, atm I have alot going on, maybe some time in the next month.

[modelsbymike3d]

Oh awesome! I didn’t realize Mixamo would keep the existing rigging. Just gave it a try and it worked great!

[modelsbymike3d]

Here’s my first proof of concept! Lots more to do, but so far working in PlayCanvas has been super enjoyable. Thanks again for all the help!

Michael Porter on LinkedIn: #augmentedreality #metaverse #playcanvas

[Jean-Philippe_Encaus]

Thanks for the answer, follow-up Twitter, can you add more details ?

1. I follow exactly this video but my avatar lost texture in Mixamo. In Windows 3D Viewer the texture seems messed-up
2. What format should I download from Mixamo after ? It seems I can only download FBX, should I move by Blender again ? or Play Canvas ?

Sorry for my n00b question, I already integrate and lipsync my ReadyPlayerMe avatar on my SmartMirror now I try to add animations like all of you

Thanks !

[Saad_Haider]

Hi @Jean-Philippe_Encaus,

1. When you export the character from blender as .fbx for Mixamo, do you embed the textures in the .fbx file?

2. You can download the .fbx

[Jean-Philippe_Encaus]

Yes I follow the video, check the include texture mark. On Windows 3D Viewer the fbx contains a kind of texture but it look like a patchwork. I assume Mixamo ignore them.

• I’m on fresh blender install
• I try with a random ReadyPlayer Me avatar
  I probably missed something.

So as a workaround In Blender I copy Material + UVMapping from ReadyPlayer Me avatar to Mixamo Fbx, it works but a little more slower

[Saad_Haider]

Right, so in blender, I make the path mode to copy and click the button on its right side.

[Jean-Philippe_Encaus]

Yes, me too, and in the video too, that really weird

[Jean-Philippe_Encaus]

• On the left the ReadyPlayerMe glt
• On the right, export then import back the fbx

There is some kind of texture but messed up. I don’t understand why