Unloading and Reloading Assets

Help & Support
1

Hey guys

We are currently starting to work on a new project, which will feature a lot of levels and a lot of different assets. We already ran into some problems in the past with having too many assets being loaded in the VRAM. So this time we wanted to tackle this issue before it becomes a problem at all. This is the project where I’m currently testing stuff out: Test Project: Asset Loading

I noticed when setting preload on assets to false, the app will automatically load the asset, if it is needed in a scene. So I added an event listener to listen to pc.app.assets.on('load'), so i can remember which assets were loaded dynamically. The next step I figured would be to unload all of these assets, when you change the scene. When I inspect the VRAM via the PC Profiler, it looks good, the memory seems to be freed when i go back to the empty scene, except for Textures Other.

image>image>image

But I got problems when I returned to the scene with the dynamically loaded model, as it stayed black, with no texture whatsoever. When I tried unloading only materials and models, but kept the textures, it kinda works, but texture assets in the VRAM are not freed. Also in both cases, I got this warning, which I don’t quite understand:

image
image933×25 2.05 KB

I also found various other posts regarding unloading assets, but they all don’t seem to try to reload an asset, once it has been unloaded:

So here are my questions:

  • Can i rely on the auto loading the same way it happens the first time I load a scene?
  • Do I have to manually call load again, once I unloaded an asset?
  • Do I have to keep in mind the order in which assets need to be loaded? (E.g. materials referencing textures)
  • Is the timing important during a scene transition? Do I need to wait until the old root is destroyed, or the new scene hierarchy is fully loaded?
  • What is Textures Other? Is it the buttons, which I’m currently ignoring, or how can I free it?
  • What is the meaning of the warning?
  • Do you know anything else that might be important?

We also created an issue in your Git repository regarding this topic https://github.com/playcanvas/engine/issues/1751

I also found a pull request, which might be related to the ‘the model stayed black’ issue? https://github.com/playcanvas/engine/pull/763

Looking forward to your help and replies
Thanks

4 Likes
2

Hi @AliMoe,

This is quite a study you did, thanks for sharing. Optimal loading/unloading of assets is quite mandatory to get right if you have a huge assets bank and/or trying to optimise for low RAM usage (mobile).

To my knowledge:

I think not, this will happen only once when the asset is requested for the first time and you haven’t manually unloaded. As soon as you use the asset.unload() method you will have to manually reload the asset using:

this.app.load(asset);

Not sure, but to my experience I never had an issue with the order I am manually loading my assets (model, material vs texture). Only when I reference resources in custom shaders the order is important, otherwise the shader may fail to compile/run.

It depends on how you plan to showcase your transition. The scene hierarchy is a collection of entities, that load and are made available independently of any asset loading.

From the moment that you have the new hierarchy available and you parent it to the Root entity, it will be available much like any regular entity added as child.

Good question, I wasn’t able to find any relevant property in the engine source code with a quick look.

I am not sure about it, most likely one of the engine devs here or on the repo can answer about it.

1 Like
3

I suspect the “Textures Other” would be the framebuffer / depth buffer / shadow maps - so render targets used by the engine. Try changing the shadaw map resolution / rendering window resolution and see if the size changes.

3 Likes
4

Thanks guys for the reply, you already helped me quite a bit. I got a bit further in my journey to fully releasing the memory today:

  1. I split the process of reloading the assets into 2 parts. First, when starting to load a new scene, I unload all the assets that I remembered from when I entered the scene. Then after the new scene hierarchy was successfully loaded, I manually load all the assets which I already knew that need to be loaded. If I don’t know anything about this scene yet, I will keep listening for the pc.app.assets.on('load') event and add those assets to a reference object. This object is created dynamically at the moment, and gets expanded whenever you enter a new Scene. It looks something like this:
this.assetReferences = {
    scene_A: {
        model: [...],
        material: [...],
        texture: [...],
    },
    scene_B: {
        model: [...],
        material: [...],
        texture: [...],
    },
    ...
};
  1. Not relying on the auto loading and instead manually loading assets, which I already knew will be needed, made this work when transitioning between 2 scenes. The catch though: It only works without any problems, when those 2 scenes do not share any assets. If the scenes share a material or a model, both the model and the material get auto reloaded, but the texture (E.g. a diffuse map on the material) doesn’t. The model stays black, as if it didn’t have a diffuse map.
    My guess is, that the auto loading triggers for the model and the material, since they are both referenced in the scene graph, but not for assets referenced by other assets. It could also be related to the warning mentioned above. Materials seems to be validated only once and then a flag gets added to prevent redundant validations. Maybe the auto loading would be triggered there? @yaustar can you maybe provide some insight on that?
    If that is the case, I would need to know somehow which assets reference each other. If I unload a material and a texture, once I load this material again, I need to manually load the texture as well. Where would I find these references?

Once this issue is resolved, the next step will be finding reoccurring assets and unload only the ones that are necessary.

5

It’s possible it only handles this at the time the assets are created and not later. May have to dig a bit deeper on that one and perhaps get a reproducible / check if it’s expected behaviour.

My experience with handling assets is to use tags on the assets themselves for each scene that they are needed.

That way, I can find the assets by tag for the scene that they are needed for and load them ahead of time.

It’s manual but more controllable.

6

I was struggling with this the other day, trying to get a reference to texture assets as referenced by a material.

Though it seems impossible to do this since the material diffuse/emissive etc slots reference the texture resource, not the asset. So there is no way to get the asset IDs.

In my case I solved this in a not so clean way: grab the texture name and find the asset by its name (this breaks if asset names aren’t unique, which they can be).

I’d say this is a valid feature request to be able to get referenced assets by another asset.

7

Doing a quick test with a StandardMaterial and Texture asset, it seems like it only does the asset dependency load on the very first load. This could be a bug or some sort of intentional optimisation.

I add it to repo tomorrow morning as it doesn’t feel right.

2 Likes
8

We will have multiple world maps with 10+ levels each, similar to games like Candy Crush. So adding a tag to each asset for every level that it appears in doesn’t seem feasible. Especially since we plan to continuously release new content.

9

Can the assets be divided up in groups of:

  • Common to every world
  • Specific to each individual world
  • Specific to each level

We did this a lot during our console days, even duplicating assets to make the loads easier to manage in game. We had a common folder, a folder for each world and one for each zoom. There was also some special folders for things like faces, bodies etc that didn’t really fit the above structure.

(Side note: you can group select the assets and add/remove tags so if you have them in a folder structure, you can select all and add a tag in one go)

10

We don’t know yet, how we will organize our level assets. But it is more than likely, that we will have some assets, that are specific to a world at first, but later on we will reuse them in another world. So in the long run, most assets might end up in the Common pool. At that point we could just preload them all.

Manually setting those tags sounds like a pretty tedious task, which is likely very prone to human errors, especially since in the beginning we will iterate a lot, when it comes to building levels. So we would like to avoid it.
If we could write an editor-script (like in unity) or something similiar, to automatically set those tags, this approach would be more viable I think.

11

@AliMoe I’ve tried doing something similar in the past:

  • Given an entity hierarchy, find all assets used by various components

What I did was traverse the hierarchy and do checks for model/element/animation components for any used assets. It wasn’t an easy task since there are a lot of cases but at the end it worked.

We had a list of all used assets in a level, which we could load and signal at the end that the level is ready.

What was hard to fully nail was the model->materials->textures references, so we added a constraint that all asset names should be unique. From there we could easily find the relations.

12

I created a GitHub issue here: https://github.com/playcanvas/engine/issues/2260

1 Like
13

Something that may help is that the first time the material is loaded, there is a private property called _assetReferences that may be useful?

image
image2270×328 86.7 KB

1 Like
14

Aha! Is this new? Thanks @yaustar!

That’s great as a starting point, even as a private property it can be quite helpful in this case.

15

Don’t think it is new but only spotted it as I was digging into the engine. Not sure what it is used for though

1 Like
16

@Leonidas Your approach could be a good backup plan. I think it is a lot easier to put restraints on the naming of assets, and you could also put console warnings in place, in case some assets have the same name.

@yaustar Cool, that’s a nice lead to follow on. Thanks for creating the issue. I didn’t get to work on this yesterday, but I will keep you updated on my progress once I get back to it.

2 Likes
17

It works :partying_face: :partying_face: :partying_face:

I used _assetReferences to load linked assets like you proposed. A problem I faced was, that after loading all assets in _assetReferences, this property was emptied. So I can’t continuously rely on it to load all needed asset. Instead I stored all references in the previously mentioned data structure and check there, if any additional assets need to be loaded. It now looks a bit like this:

this.assetReferences = {
    scenes: {
        scene_A: {...}, // These look like before
        scene_B: {...},
    },
    references: {
        material: {
            12345678: [ asset_1, asset_2, ... ],
            23456789: [ asset_3, asset_4, ... ],
        }
    }
    ...
};

The key 12345678 is the id of the asset, so it’s unique and I don’t really need the additional material object inside references. But I thought especially for debugging purposes this might be helpful. I also don’t know if there are other asset types which have the _assetReferences property, so I built it as if, just in case.

The only thing that kind of bothers me is that I rely on this “private” property, so it might break in the future. It would be nice to find a solution without it to always be sure it works. But now I can go on to the next step of “intelligently” reloading only assets which are needed.

Ohh, and please take a look at the code if you want to, I will keep this project public and updated.

6 Likes