Recently I’m trying to display MMD (MikuMikuDance) model and animation on PlayCanvas. And I encountered seemingly PlayCanvas FBX import issue. Would PlayCanvas team investigate this?
(MMD is one of Japanese popular amateur 3D animation environment. Tons of free assets, motion data. Popular MVs and there is even community support for Unity Mechanim)
Unfortunately I’m not a 3D expert who can write importer/exporter. So I used Blender 2.76b (+FBX plugin 3.6.2) to convert PMX(model)+VMD(motion) to FBX with baked motion. And I now got seemingly OK “Dressed Mumin” FBX file that I can preview with Autodesk FBX Review (v22.214.171.124)
The FBX file: Google Drive: Sign-in
When I uploaded it to my project, JSON convert works and it actually animates. But I see some problems.
- This model have some rigid bodies for clothes and hair physics. And the rigid bodies are visible on PlayCanvas while they are invisible in Autodesk FBX Review.
- All mesh animations seem to be working but it seems some meshes are roteted or placed in unexpected manner. (Material seems to be good) As a result, it ended up with a somewhat scary screen like below.
I’m not building anything but just using converters and PlayCanvas feature. I’m hoping that I reproduce some popular MMD video on PlayCanvas.
my friend faced similar issue.
it should not be used Japanese for material name.
my friend used Japanese for texture file name, so his model become unpublishable.
it has no effect for solve to rename on PlayCanvas.
he remove the texture once, then uploaded renamed file again.
on your project, some parts of the model are named in Japanese.
Thank you for making this review and finding some problems.
It definitely seems like we doing something wrong on our end. We will look into that problem, and let you know once we are ok with it.
In meantime, you could try to export that model using a bit older FBX plugin for Blender, that might help.
OK I will try older version.
If you have some idea about against which FBX binary format version did PlayCanvas developed importer, please let me know. (I used FBX 7.4 binary format that requires FBX SDK 2014 or newer)
Japanese issue is interesting. I’ll keep it in my mind. But since Autodesk FBX Review can show the model and baked animation without problem, I think there is something in PlayCanvas importer.
After your post, I worked on Blender and the MMD importer and I can render the model and the animation as I expect in Blender. So I restarted to work on PlayCanvas FBX export/import issue. Let me update my progress so far.
My understanding was that PlayCanvas can only import certain version of FBX format. So I used Autodesk FBX converter to convert FBX from Blender to various FBX versions and compared the behavior on PlayCanvas. And I found that the same issue happens on PlayCanvas for all FBX 2009, 2010, 2011, 2012 and 2013.
The FBX 2013 on FBX converter’s viewer is below. (The face seems wrong but this is the expected rendering result. This model is specialized for shadeless + toon rendering. All FBX 2009 to FBX 2013 displayed as expected in this converter)
The import result on PlayCanvas is below. (https://playcanvas.com/project/366965/overview/model-test-2)
All FBX versions show the same issue. Also this issue is very similar to the one in my previous post thought the models are different. I changed the opacity of the material applied to rigid bodies to 0% so they are not visible anymore though.
And finally, the published animated model on PlayCanvas is below. (http://playcanv.as/b/koDLn5e8)
I uploaded the blender file and all FBX files here. (https://drive.google.com/folderview?id=0B08xnIVZXi7cY2ttU0pzN0RiR1k&usp=sharing)
I think one of the reason why the two models showed the very similar issue is that these two models shares the standard MMD bone names in Japanese. Current my guess is PlayCanvas somehow fails to handle the MMD standard bone name in Japanese for neck(首) or head(頭) in FBX file.
I will keep working on this. If your side have any progress, please let me know. Thank you!
Have you tried to rename bones to latin, and ensuring all of them are uniquely named?
Check if that helps?
I reviewed this issue with my friend visual designer.
He guess that the bone of head would be short. so the head of that model will be dropped by physical simulation.
The effect of bone may be a little different from Maya or MMD.
So he has changed the bone length (make longer) for PlayCanvas on our game that is developing now.
Thanks! I will check the bone / model behavior from bone length perspective as well.
Interesting thing is the rotation of the head is -90 degree around X axis after the import. And once the animation is applied, the bone head location seems to be set to (0, 0, 0) but the baked animation is properly played. As you described, I think the head bone is located or rotated unexpectedly due to some unknown reason.
I modified the name of the bone from “首” to “neck” and “頭” to “head” and the head model issue is resolved. My current guess is that PlayCanvas has a problem with handling “a bone with a single unicode character”.
https://playcanvas.com/project/367166/overview/model-test-3 (this project plays a music and it takes long time to load because of long animation data. Sometimes it seems the browser stopped)
My next step is adjusting the face/body shader so that it will render with toon shading like the original MMD environment.
I’m pretty new to shading though. Does anybody know a good document to start with?
After observing the behavior of the demo, I realized two things.
- PlayCanvas FBX importer doesn’t seem handling shape key animation. Which is applied to this model’s facial animation. Is this the case? If so, is there any plan to support it?
- It seems the animation doesn’t quite sync with the music if the frame rate dropped. There might be something in model animation frame skip.
Thanks a lot for testing this out, and reporting issues, we really appreciate the collaboration on that matter.
So regarding unicode, we need to have a look into that, it is probably our convertor messing it up.
Regarding freeze on load, we’ve noticed recently a problem with calculating AABB for heavily skinned and instanced models. We are working on on the fix to resolve that problem.
Regarding shape key animation, which is vertex-based animation. It is in our roadmap, but it is not high priority atm.
Toon shading is possible. Few options you can entertain:
- overriding materials chunks to modify lighting behaviour. This option will allow you to benefit from all material lighting mechanics, PBR, IBL, and other features. While look more toonish.
- making custom toon shading shader - this options would loose benefits of scene lighting. Making it with scene lighting would require an effort.
Cell-shading can be done as post effect, or as duplicated model with back faces, full-black color and small offset in vertex buffer for each vertex based on vertex normal.
Thanks again, for investigating into fbx issues.
Thank you for the shading information.
Since shading and FBX import are different topic, I will create a new thread and ask shader related question in there.
We’ve updated FBX SDK on assets pipeline, it should now accept latest FBX’s.
I imported the FBX 2013 model I shared in this thread and PlayCanvas still shows the same issue. My guess is that the issue is not in FBX SDK itself but in somewhere between FBX SDK output to JSON model/motion data.
Regarding two problems:
Naming - yes it is still a problem. We will look into that problem.
Weird bipet things, will have to look at too.
Update: I tried to import another model and I got additional information about erroneous bone import. So let me share here.
I feel there is some UTF-8 or UTF-16 handling issues in PlayCanvas FBX to JSON converter. Seemingly some is an issue in handling the tail of the bone name. Some is an issue in handling a character code in certain range. But still I didn’t narrow down to the exact cause.
Pattern 1-1 (Adding a character after the problematic bone name fixes the issue)
The bone named “頭” (UTF-8: E9A0AD, UTF-16: 982D) unexpectedly located at seemingly (0, 0, 0) of parent’s local coordinate. I could fix it by adding “です” (UTF-8: E381A7 E38199, UTF-16: 3067 3059) after “頭”
Pattern 1-2 (Adding a character after the problematic bone name fixes the issue)
The bone named “上半身” (UTF-8: E4B88A E58D8A E8BAAB, UTF-16: 4E0A 534A 8EAB) unexpectedly located at seemingly (0, 0, 0) of parent’s local coordinate. I could fix it by adding “.” (UTF-8L 2E, UTF-16: 002E) after “上半身”.
Pattern 2-1: (Adding a character doesn’t fix the issue. Has to replace entire problematic characters to fix the issue)
The bone named “ﾈｸﾀｲ1” (UTF-8: EFBE88 EFBDB8 EFBE80 EFBDB2 31, UTF-16: FF88 FF78 FF80 FF72 0031) unexpectedly located at seemingly (0, 0, 0) of parent’s local coordinate. I could NOT fix it by adding “.” (UTF-8L 2E, UTF-16: 002E) after “ﾈｸﾀｲ1”. I could fix it by replacing “ﾈｸﾀｲ1” (UTF-8: EFBE88 EFBDB8 EFBE80 EFBDB2 31, UTF-16: FF88 FF78 FF80 FF72 0031) with “ネクタイ1” (UTF-8: E3838D E382AF E382BF E382A4 31, UTF-16: 30CD 30AF 30BF 30A4 0031)
We use the FBX SDK to convert FBX to our JSON format. The FBX SDK processes all strings as UTF-8. I did some tests with various Kanji characters and it seems to process as expected. However, can you provide me with a simple object that breaks? Or maybe, if I create a skinned cylinder with 3 bones using the bone names you are having problems with, maybe I can reproduce the problem…
I created a simple repro model with Blender and the JSON works even with suspicious bone names.
I started to think the most straight forward way is I generate JSON file from the FBX file before the bone name change and from the FBX file after the bone name change. Then carefully compare those two focusing on the updated name.