Mighty Morphing

Help & Support
5

In your project you’re doing
asset._resources[0].meshInstances[i].morphInstance.setWeight(0,1);

So you’re modifying base asset, not the scene model (also .resource can be used instead of ._resources[0] AFAIK). Each model instance has its own morphInstance.

This line seems to change character’s eyelashes:
app.root.findByName("Model").model.meshInstances[0].morphInstance.setWeight(0,1)
The only difference is that here I’m accessing scene model.

6

Its read only for me dont expect me to respond

1 Like
7

Thanks @Mr_F, I replaced _resources with resources, not much of a difference.

That script is also attached to the model, so this.entity and app.root.findByName("Model") should technically be the same thing.

Either way, I tried replacing that as well and didn’t get the results you’re mentioning.

If this did work for you, it would make sense that you saw a change in the eye, since _weights[0] references the “Blink_Left” weight, so setWeight(0,1) would ideally close the left eye.

I didn’t get the same result though…

Any thoughts? Thanks! :slight_smile:

8

It’s read only for everyone except for the team members which is the default for public projects. You can still look at all script, materials etc but you can’t make any changes. You can also fork the project if you want to make edits and test a few fixes etc.

9

yes ik, he tryed to communicate to me through the script and i didnt know how to reach him other wise

10

This must be a tough one… not getting much of a response.

Any thoughts?

11

I’ve got it ‘working’: https://playcanvas.com/editor/scene/579478

Stepping through the code, _weights doesn’t actually have any values (ie it’s an empty array) until the 2nd frame update. I don’t know why, perhaps someone with more engine knowledge can give an explanation.

Edit: Looking at the engine code some more, and it looks like it doesn’t get filled till the first update call, which may be after the scripts logic and therefore you can’t set any weights till the script’s 2nd frame update.

2 Likes
12

Nice @yaustar how did you discover this??

13

man… now it seems obvious, but I’ve been cracking my head over that.

I did notice that _weights was populated after initialization (because of the underscore)… but I didn’t think of going into update instead of initialize or postinitialize.

Very much appreciated!!

Next step… I want to animate a sequence of morph coefficients.

14

Did a bit more debugging and it looks like _weights get filled after the first render call hence the setWeight needs to be done in the next update call after the first render(aka the 2nd frame script update).

  1. Assume that Mr_F is unlikely to post something that doesn’t work :slight_smile:
  2. In the past, some values are set in initialize or postInitialize so I tried setting the value in postInitialize and update
  3. Found that setting the value in update worked so assumed that doing it just the once in the first update would be fine. It wasn’t.
  4. Added a breakpoint in the debugger for the setWeight function and looked at _weights and noticed it was empty in the first script update call but was populated in the second.
  5. Looked at the engine code where it was being populated and traced back through the callstack to see that it was called in the render function of the engine which is after the logic update call.
3 Likes
15

Ouch. That actually seems like a bug. Should work if changed before first render. Will look into it.

1 Like
16

Lmao… So you know how to work this morphing stuff? how does it work?

17

Mr_F is one of the developers contributing to the PlayCanvas engine. He has a lot of graphics/rendering knowledge :slight_smile:

18

Oh ok i was wondering why i understood none of it :smile:

19

Thanks @Mr_F and @yaustar !

A follow up question to this would be if the animation parser is looking into morphs?

I looked into src/resources/animation.js and found that v4 is creating animation keys for bones:

        _parseAnimationV4: function (data) {
            var animData = data.animation;

            var anim = new pc.Animation();
            anim.setName(animData.name);
            anim.duration = animData.duration;

            for (var i = 0; i < animData.nodes.length; i++) {
                var node = new pc.Node();

                var n = animData.nodes[i];
                node._name = n.name;

                var defPos = n.defaults.p;
                var defRot = n.defaults.r;
                var defScl = n.defaults.s;

                for (var j = 0; j < n.keys.length; j++) {
                    var k = n.keys[j];

                    var t = k.t;
                    var p = defPos ? defPos : k.p;
                    var r = defRot ? defRot : k.r;
                    var s = defScl ? defScl : k.s;
                    var pos = new pc.Vec3(p[0], p[1], p[2]);
                    var rot = new pc.Quat().setFromEulerAngles(r[0], r[1], r[2]);
                    var scl = new pc.Vec3(s[0], s[1], s[2]);

                    var key = new pc.Key(t, pos, rot, scl);

                    node._keys.push(key);
                }

                anim.addNode(node);
            }

return anim;

…but I don’t see it parsing morph information.

I was trying to follow the same format for bone animation in order to recreate it under the morphs node, but I’m not sure if this would be read by any function.

I was trying to avoid depending on .update in order to create a morph animation sequence.

[here’s a condensed version of what I’m doing]

CheckMorphs.prototype.initialize = function () {
    this.frames = { 
     0: {
          0 : / *coefficient between 0 and 1 */,
          1:  / *coefficient between 0 and 1 */, 
          n: {...} 
     },
     1: {
          0 : / *coefficient between 0 and 1 */,
          1:  / *coefficient between 0 and 1 */, 
          n: {...} 
     },
     n: {
          0 : / *coefficient between 0 and 1 */,
          1:  / *coefficient between 0 and 1 */, 
          n: {...} 
     },
   }
};
CheckMorphs.prototype.update = function () {
   self = this;   
   jQuery.each(this.frames, function(i, val){
     self.entity.model.meshInstances[0].morphInstance.setWeight(i, val);  
  }
};

I’ve managed to get the face to move, but it’s incredibly choppy. I can only assume that it’s not interpolating from morph to morph.

Concrete questions:

  • Is there a way to integrate a morphs weights keys array into an animation file (even if at this point I have to do it manually)?
  • Is there a better way to interpolate between a .seWeight and another .setWeight?

20

Animation for morphs isn’t loaded at the moment. Animation system isn’t good in terms of blending. So your best bet is to export morph animation to a custom file format or animate procedurally.

Try setWeight(pc.math.lerp(weight1, weight2, blend)), where blend is from 0 to 1

1 Like
21

Thanks again @Mr_F!

So this is where I’m at

  • The biggest deal here is the FPS… it drops from 60 to ~4
  • Also, .setWeight starts to slowly deform the mesh as the animation moves forward

things to note:

  • the morph data is being matched… meaning that some of the blendshape used to capture the data are not a direct match to the current model’s blendshapes. For example the capture data has BrowsUp_L, where the model has BrowOutterUp_Left, BrowsInnerOutter_Left, Brows_Up_Left., etc. So that can account for the unnatural expressions on the model. I’m still tweaking it to mirror them better.
  • The capture data includes things link LookUp_L, LookOut_L, LookDown_L, which I’ve converted to traditional bone animation by updating the rotation for children under this.entity.model.skinInstances.bones hierarchy (Right and Left eye). That doesn’t seem to be happening though…
  • Unfortunately the data was captured at 24fps, haven’t figured a way to speed it up to 60fps using the .update (that’s the least of my worries at this point)
  • I haven’t added the lerp. It just felt like overkill with the current FPS situation.

Any thoughts on how this could be improved?

22

Sorry for a late reply. 4 fps, really?! How many vertices are in the model and how many of them are morphing? How many blendshapes are active (weight > 0) at the same time? What device/browser?

That doesn’t seem to be happening though…

Bones are just regular scene nodes - you can try finding them via entity.findByName(“bone”) and rotate. Does it help?

23

I think this is the most relevant conversation regarding the issue I am running into. If I am reading this correctly, there is no way ‘out of the box’ to have the animations in a fbx drive the morph targets in real time. The animation would have to be parsed and updated on every frame. Is this correct?

@Mr_F can you elaborate a little more on:

Animation for morphs isn’t loaded at the moment. Animation system isn’t good in terms of blending. So your best bet is to export morph animation to a custom file format or animate procedurally.

24

As it happens, the glTF loader supports animated morphs just fine! For example:

https://cx20.github.io/gltf-test/examples/playcanvas/index.html?category=tutorialModels&model=AnimatedMorphCube&scale=1&type=glTF

We’ll be integrating this into the Editor over the next few months.