How to get 16bit normal map in PlayCanvas (smooth low poly mesh)

Hi, I try to do realistic car body made from CAD files and I do there automatic decimation often.

When I have low poly meshes then I use a normal map to fake the smoothness of the surface.

I`ve prepared low poly test sphere for that:

This is how it performs with 16bit normal map in Marmoset Toolbag:

This is how it performs with 8bit normal map in Marmoset Toolbag:


On 8bit normal map you can notice glitches/banding errors.

I’ve tried to use both normal maps in PlayCanvas but both won`t look good (see below).

How to use 16bit normal map with PlayCanvas?

I did test project. You can download textures and model there to test.

https://playcanvas.com/project/1016973/overview/8bit-normal-map-vs-16bit

Gold is 8bit, silver 16bit.
https://launch.playcanvas.com/1608561?debug=true

I upload textures also here:


Model was prepared in Rhino. Normal map baked in Substance Painter. From high poly to low poly model.

1 Like

I spoke to the GFX team and was told that not all devices well support 16 bit and therefore it would need to check and convert down depending on the support.

The Area lights LUT does this: engine/webgl-graphics-device.js at b035de78c8e6fb7a34e560f83b3a55d697e173e6 · playcanvas/engine · GitHub

Another possibility is to use texture format PIXELFORMAT_111110F which is WebGL2 only but means you can upload a standard 8bit PNG but just change the texture format.

1 Like

@mdesign Can you host the images elsewhere (github/zip) as the forum software will convert the images to a different format.

I`ve put a zip folder as an asset to the project folder (public). May it be like this?

image

Yep, that works

There may be small errors in the normal maps but the overall difference is visible.

So please don’t look into those few errors here but the rest is almost perfect with 16 bits.


(this is 16bit normal map in a marmoset toolbag) - you won’t notice there are any glitches except those few errors which I pointed out and those are my fault).

Hmm, so both have banding on my screen. The only difference I see is that one is gold and the other is silver. What am I missing?

Yes, but in Marmoset Toolbag 16bit is without banding.
It’s the same now because after importing/uploading 16-bit normal map to PlayCanvas it was downgraded to 8-bit by PlayCanvas automatically. That is the problem and that`s why both look the same. 16bit should be without banding.

You need to use 16bit png, convert pixels to 11-11-10 format when loaded, and upload that data to texture. Using standard 8bit PNG won’t give you better precision.

You could likely do the 8888 to 11-11-10 conversion ahead of time and store that in 8888 PNG, but I’m not sure this would be without complications.

I`m not sure If I follow you correctly.
Is it enough only to convert that 16bit image to 11-11-10 format and change file extension to *.png?

Is that all or should I code something?

Where should I look for info about some software to convert png to 11-11-10? I hear first time in my life about that file format. Is it some RAW?

Should I convert it only inside the code of PlayCanvas or deliver converted?

Edit: In the documentation of PlayCanvas I’ve found that there is also PIXELFORMAT_RGB16P.

Why I can`t use that?
How should I convert those image? Could you provide some small example?

I doubt there would be any tools to help with GL_R11F_G11F_B10F format (google for it, that is the Open GL / WebGL format name)

Typically it’s used as a format for render targets, but not for textures. I cannot even guarantee all platforms will allow you to upload data to it at all.

Unless you are prepared to write bunch of code to test it, I’d just go with HalfFloat / Float format which I suggested to @yaustar here:

In this case you’d need to have some png loader library that can load 16bit pngs (maybe even browser can do it? No idea). And then similarly to the the Area lIghts LUT code does, you’d convert those 16 bit values to either 4xHalfFloat / 4xFloat or 8888 format, depending on what platforms supports, and upload it to texture.

But none of these options is super easy to do.

2 Likes

Thanks a lot for the clarification.

it is also hypothetic way to split 16bit png to two 8bit images with tool like this: GitHub - czero69/ImageSplitter: Splits 16 bit per channel image into two 8 bit per channel image (lower 8 bits and upper 8 bits). Useful wehen working staff like webgl and wants use 16 bit per channel.

It would split 16bit to lower 8bit and higher 8bit. So I would use both of them.
Could you provide me with some info on how to blend two 8-bit normal maps in PlayCanvas?

I’m not sure splitting it would be a good option, as you would not be able to use be-linear texture interpolation to sample those textures in the shader. You’d need to do manual interpolation, or have a strange errors / noise in the normal map. I don’t have a code for any of this handy, I’m sorry, it’s not just few lines or anything easy like that.

1 Like

Sure, understood thanks.

This could work:

  • use that tool to split 16bit png into 2 8bit pngs.
  • load those pngs as normal texture assets by PlayCanvas
  • when loaded, get their pixels, and combine them into a float array (so basically your data would be the same as original 16bit png)
  • use the already referenced code of Area lights LUT to load this float data into float / half-float / 8bit texture, depending on what is available on the device. On majority of devices you will get 16 or 32 bit per channel
  • use it as a normal map on your material
1 Like

Thanks a lot, I will try it.

for normal maps specifically, you could also write a splitter that splits 16bit normal map into a single RGBA tga texture for example. Normal map only needs to channels, R and B, the green cam be computed. So you just need to store 16bit R as RG in 8 bit, and 16bit B as BA. So you’d have a single 8bit texture. Then you’d continue from line 3 on my previous post (convert them into a float array)

1 Like

we have an example of loading png using upng library here

perhaps this could just be used to load 16bit png directly … get floats from it and just do the rest

1 Like

Work in progress here: https://playcanvas.com/project/1017976/overview/f-8bit-normal-map-vs-16bit

I believe I’ve loaded the 16 bit texture correctly but still looks the same. Still investigating