Adding lots of entities in bulk - performance considerations

Is there a way to defer the engine’s calculations whilst I add lots of entities?

I’m currently using the engine by itself after trying a few alternative 3D libraries and having performance/rendering issues. I’m attempting to add a frankly ludicrous number (10,000+!) of box entities to a scene and, unsurprisingly, the more that are added the slower the engine gets to add another.

I’m fine with being told that what I’m doing is crazy but I’m wondering if there’s anything I could be doing to help even a little. I’ve already tried distributing my entities in the node tree so that app.root doesn’t have to manipulate a huge children[] array but to little effect. Note that there’s no problem displaying the entities when they’re in the scene; the only issue is adding them in the first place.

I’m wondering if I can tell the engine to hold off on recalculating things whilst I add several entities and then get it to recalculate with all of them in the scene or if there’s something else I can do to assist the engine when I’m making unreasonable demands?

~ Bruce

Hey, do you have a simple project where you replicate an issue with some timings check to benchmark it?
In chrome dev tools there is a profiler, that can be used to have a look at performance of certain parts of engine. So that will help to identify what is a bottleneck and why it gets slower with bigger hierarchy.
Or share some code for replicating that.

Great idea!

The two things that immediately jump out are GraphNode.findOne() and Material.clearVariants() (both called during Entity.addChild()). They’re easily taking up 50% of the execution time.

I’ll go see what they do under the hood and see if I can’t make their lives easier.

~ Bruce

1 Like

Alright, that seems like it is checking if entity is already added to hierarchy.
So there is a solution to this - we need to create global index per app, of all entities in hierarchy, and just do check against it.

That might optimise cost of adding entity dramatically, and wont lead to increasing cost based on number of entities in hierarchy.
Worth of a ticket here: https://github.com/playcanvas/engine

Material.clearVariants() is an interesting case.

All of my entities were clones of one another and thus shared the same Material. If I create a new entity to clone (with a new Material) every 1000 copies all of the time spent in Material.clearVariants() goes away.

~ Bruce

Hmm, there might be unreasonable things going on indeed.
Thank you for looking into that.

It would be good it do profiling and iterate on it, to identify what can be improved.
Like building index of entities by their guid is one of those.

There’s very little I can do to mitigate the time spent in GraphNode.findOne().

The entities I’m adding don’t exist in the scene so findOne's tree search is always going to run through every node no matter how I organise the tree.

You’re probably right that the solution for an existence check is to have a hash of all the GUIDs for an immediate look-up (at the expense of having to maintain it).

~ Bruce

This is my first program using PlayCanvas’ engine; I think it’s a bit early for me to be raising bug reports. :grin:

All told I’m pretty happy with PlayCanvas. I tried three.js and Babylon.js as well and only PlayCanvas didn’t wholly buckle under the stupidity I threw at it.

If you’re interested in what I’m doing I’m generating geodesic constructions for Minecraft. I’ve been running my profiling against 12 points, radius 80 and transforms kt.

1 Like

This is my first program using PlayCanvas’ engine; I think it’s a bit early for me to be raising bug reports.

That’s not bug, but there is an improvement can be done, and is seems like easy one. So definitely worth a ticket.

That’s a nice demo indeed, and looking at performance - you are totally right, this shall not take that long.

Very cool project! :smiley:

Although, yeah, it did take quite some time to generate…

Do you use Twitter? If you do, and you mention playcanvas in relation to this, we’ll make sure we RT!

Added a ticket: Entity.addChild() slows dramatically when there are lots of entities

1 Like

Dave’s fix makes a massive difference. I patched a local copy of the engine and gave it a spin and now don’t hit a wall until around 16,000 entities (when renderShadows and renderForward start to creak).

1 Like

Good news, we can probably deploy that change today, barring any disagreements.

BTW, you are linking to playcanvas-latest.min.js in that application which is the master branch and gets all changes instantly and may break. You might want to use playcanvas-stable.min.js instead which is the stable branch.