[Bug] Rotating lightmapped object leads to lighting artifacts

underlight · June 7, 2021, 4:53pm

Hello guys!

So basically when you try rotating a lightmapped object (color + direction) - lighting artefacts start appearing until rotation is reset to original one, at which lightmap was baked. Same thing was observed with animated objects - seems like it has something to do with how world normals change during rotation / animation, and current directional lightmap shader doesn’t process it correctly (?)

I’ve been observing this issue for quite a long time already (back in 2019), but only recently it started blocking one of my AR / 8th Walls project since I can’t bake any lighting for AR scene because of this rotation issue. Hope it can be resolved soon - or at least would be happy to hear out suggestions how I could fix it myself and make a pull request. Cheers!

Test build: https://playcanv.as/p/j2DJ4KSh/
Test project: https://playcanvas.com/project/802582/

yaustar · June 7, 2021, 5:26pm

@mvaligursky any insight on this one?

mvaligursky · June 7, 2021, 6:47pm

lightmapping as a technique is really only suitable for static (non-rotating objects), especially the one which bakes the direction as well.
To use some baked lighting on dynamic objects, you’d usually use Sh lighting using light probes or similar, which we have on the roadmap to work on later in the year.

In the meantime you can try and bake lightmap without direction - but even that won’t work for more complex setups … for example if your cube casts shadow on some plane, the shadow would not update as the cube rotates.

underlight · June 7, 2021, 7:46pm

Thanks, got it. I understand limitations - but for AR when you want to prebake lighting and then give the user an option to rotate the object / scene on his table - it would be awesome. Usually the entire scene is being rotated, so shadows won’t need an update anyways.

Guess I’ll stick with non-directional lightmaps for current prototype, and then will try to modify directional lightmap shader chunk to suit my needs. Any ideas how that could work? I understand why that happens - lightmap direction data is baked in the world space, and it relies on geometry normals being in the world space too, and then during rotation L dot N or something similar goes wrong. Quick fix would be providing vec3 uniform with scene rotation and modify L dot N factor to take that rotation into account. It should work, but doesn’t look that elegant though.

mvaligursky · June 7, 2021, 8:06pm

This is the chunk you’d need to modify

github.com

playcanvas/engine/blob/master/src/graphics/program-lib/chunks/lightmapDir.frag

uniform sampler2D texture_lightMap;
uniform sampler2D texture_dirLightMap;

void addLightMap() {
    vec3 color = $texture2DSAMPLE(texture_lightMap, $UV).$CH;
    vec3 dir = texture2D(texture_dirLightMap, $UV).xyz;
    if (dot(dir, vec3(1.0)) < 0.00001) {
        dDiffuseLight += color;
    } else {
        dLightDirNormW = normalize(dir * 2.0 - vec3(1.0));

        float vlight = saturate(dot(dLightDirNormW, -dVertexNormalW));
        float flight = saturate(dot(dLightDirNormW, -dNormalW));
        float nlight = (flight / max(vlight, 0.01)) * 0.5;

        dDiffuseLight += color * nlight * 2.0;
    }

    dSpecularLight += color * getLightSpecular();
}

but it’s not immediately obvious to me what needs to be done … I’d probably hope to provide a matrix4 uniform representing the rotation, and transform direction vectors by it. More details would need some research, as I have not implemented this.

underlight · June 7, 2021, 9:09pm

Thanks, that’s more than enough for starters, cheers!