The exact FPS-conversion from Blender to PlayCanvas?


#1

Without having any illusions to where it might end, I am (maybe naïvely) trying to make a lipsynced storytelling project (FBX-based with firing/adhering sound bites). Within that realm the notion of reaching the highest possible synchronization {the exact FPS-conversion between my used FBX-builder; Blender, and PlayCanvas} becomes pivotal [by the way, googled alternatives to FBX+sound heavily already].
As it stands right now I come to the sub-conclusion that;

  1. The consensus maximum execution framerate in PlayCanvas stands at: 60 FPS.
  2. Beneath that maximum I have found that a strip extent/action extent of 400 frames, result in a FPS-conversion of approximately 28 FPS:
    -> put this alongside the imported duration result in PlayCanvas: image
    Conversion result: 400 frames/14.1667 second = 28.235 FPS-as-functional-export
  3. Also beneath that maximum, I find to believe that there is an alternative export-setup conversion. Here the duration result of 14.1667 seconds rely on the actual extent of the keyframes in the Blender ‘Dope sheet summary’/‘NLA-strip’ only:
    Conversion result: 340 frames/14.1667 second = 23.999 FPS-as-functional-export

Conclusive question: Am I right to think that the conversion is based on 3) and not 2)?


Heavy bugs within the 'scene change' functionality
#2

Blender with have its own internal ‘FPS’ so I wouldn’t be use frames as a measure of time. I would have thought that the animation timeline in blender would have actual time units (seconds) to work with.


#3

Ok, I am currently doing some testing alongside, where to computers (PC and laptop) executes the same custom made script that uses the 0.4 conversion (FPS: 24/60). Maybe I will post some of the result - not so alerting an issue, so kind of solved… wanted some extra info from backend, but thx anyway :slight_smile:


#4

IIRC Blender defaults to 24 or 25 frames per second for animation


#5

Right. Animation systems like Blender typically work on units of “frames” and then let the user specify how many frames they want to play per second. I would assume that an FBX export would respect the FPS setting of the particular animation timeline in question. But a person would have to test that to be sure.

From observation, it seems like PlayCanvas tries to write a frame ever 1/60 of a second. The dt interval is typically around 0.01667. I think I would try to monitor dt in some way if I were trying to keep things in perfect sync.


#6

Now we are ‘nerding’ somewhere :slight_smile:

yaustar: consider your gangnam dancing robot, from the animation example was made in Blender. Then you would have to change the executed animation speed in PlayCanvas editor to 0.4 (conversion) in order to slow it down (perceived speed-correctness).

wturber: In the PlayCanvas FPS example, the top corner number can be changed from FPS to MS -> this ms is exactly the dt update time that changes in relation to ‘cost’ (system performance [Profiler: GPU, triangles, shaders etc.]).


#7

IIRC, PlayCanvas targets vsync rate and runs as fast as it can.

I still don’t understand why we are talking about frames and not time? If it’s understood that each frame in blender is 1/24 the of a second, then animate as such? Am I missing a workflow issue here?

Edit: to put it more clearly, PlayCanvas framerate is variable and frame counts should not be used as a measure of time. Use dt in the update loop to keep track of how much time as elapsed.


#8

On those areas I totally agree, yaustar … and will alter my knowledge, to what you are saying in regards to vsync (https://www.vsynctester.com/detect.html) - that is super info

The main focus of this conversion-post; Is getting to a future place where FBX better mimics a video-file (rendering follows sound in execution):

  1. Autodesk has only allowed sound to work in an FBX-file, in one their own MotionBuilder https://forums.autodesk.com/t5/fbx-forum/can-fbx-contain-sound-track/td-p/4258886
  2. One cannot stretch-variate the sound execution time in PlayCanvas (which is good and ok within this sync-issue), so all the fuzz about this post, concerns my own experiment of making a scripted structure that comes close to a workable (render-sound) synchronization.
  • hope to help myself along others, while 1) can’t be used inside PlayCanvas

->> furthermore in regards to vsync-area: I am already using dt in my scripting, to go beyond and above vsync rates of 60


#9

And in relation to yaustar’s last remark; this post should have been called: " A vsync FPS-conversion from Blender to PlayCanvas"


Heavy bugs within the 'scene change' functionality
#10

I think there is some misunderstanding. I think PlayCanvas targets the vsync/max possible refresh but it doesn’t mean it will always hit it. Depending on the performance of the PC and the app, it can run up to the max refresh rate. So this could mean 10fps, 5, 30, 43, etc. It’s completely variable hence frame counts should not be used.


#11

Based on the very simple and limited information that PlayCanvas gives on animation timing, it seems likely that they’ve engineered PlayCanvas to respect FBX frame rates. That should keep animations and audio in pretty good sync automagically since Playcanvas would then stuff or drop animation frames so that the animation plays back at the correct speed. If so, then keeping audio and animations in sync should be pretty straightforward simply using time intervals as yaustar suggests. But apparently the animations aren’t playing back at the right speed?

So here’s my question - have you actually made an animation and checked to see if it plays back at the correct speed? If not, then you may be making this more complicated than it needs to be. If PlayCanvas and Blender are doing their jobs correctly, then audio and animations should stay in good sync. Have you actually tested this? Your Blender screenshot only shows frames but you’ve told us nothing about the Blender frame rate.


#12

wturber: PlayCanvas plays my Blender animation 1-to-1 in regards to execution if Speed at the animation slot hasn’t been altered. As such I need to set this to 0.4167 (24/60), if all machines devices has a vsync rate of 60 (equal til 60 frames pr second; resulting in a way too fast playback of 24 FPS). From there the whole messy premise took off for me, as I now try to use performance-feedback dynamically to make sound+animation watchable … and yes I am actually quite/pretty close to finding an all-calibrating algorithm @yaustar (in the perfect world, and properly within the next 5-10 years there will hopefully be a Motionbuilder like sync-option, with a platform-penetration as PlayCanvas). But thx for the feedback - I will try an finish the project, that is disclosed for now. Will properly post a link of it when (if-ever :-/ ) fully finished


#13

So, by 1:1 you mean one Blender frame for each PlayCanvas dt interval?


#14

actually more on the front of passed milliseconds of app, in relation to entity.animation.currentTime (as in duration on the middle image above)


#15

I wanted to test my assumptions about how PlayCanvas deals with FBX animations. So I created two “clock” animations in Lightwave 2018 (sorry, I’m not adept with Blender). Each animation sweeps a clock hand 360 degrees in four seconds.

I brought both FBX files into Autodesk FBX Review (free app) and confirmed that they had 24 and 30 frame steps per each second after export from Lightwave. FBX Review reported that both had a time length of 4 seconds even though the 30 fps version has more frame steps.

PlayCanvas also reports each animation as having a four second length.

I put together a little project that starts both animations at the same and then pauses them at the same time. I let this app run for many minutes, and the two animations manage to stay in perfect sync despite the keyframe and frame rate differences.

I timed the sweeps for ten revolutions with the stopwatch on my phone and the animations appear be running very close to real time. I timed it at 39:88 while the clock hands on the animations seemed to be about one or two ticks past vertical. That seems within the range of error you’d expect when manually triggering the PlayCanvas and the stopwatch.

Neither of my LCD displays likes me trying to change the refresh rate to something other than 60hz. So I can’t really test how this scales with another refresh rate. But I suspect it will do fine.

I’m not at all sure that this will be useful for you, but it does seem to show that if the FBX file is created correctly with the right frame rate, that PlayCanvas will do a pretty good job of keeping the animations in sync with real time.

Here’s the project:

https://playcanvas.com/project/603238/overview/fbx-timing


#16

I was thinking about this and it occurred to me that perhaps these animations might stay in good sync by being restarted every four seconds which is the length of each animation. So I changed the animation length to one minute using a single keyframe at frames 1440 and 1800 respectively for the 24 fps and 30 fps animations. They seem to stay in sync just fine with the longer duration.

I’ve also played the animations on my Moto G5+ smart phone that is somewhat challenged by PlayCanvas and runs at about a 17 fps average rate. The animations stay in essentially perfect sync on the G5 as well.

Public build: https://playcanv.as/b/p4ZuDIKO/


#17

Ok, looks good wturber, and seems like other types of sw are exporting the duration much better.
Cf the title/headline of this post, though :slight_smile:


#18

Sure. Just figured it was worth understanding how PC behaves normally with FBX animations.