How to access pixels of material.diffuseMap

Help & Support
1

I have material which was loaded as an pc.Asset. This material has diffuseMap (which was loaded with the material) and I want to access pixels of this texture. I do it as follows:

mat.diffuseMap.lock({mode: pc.TEXTURELOCK_READ});

But instead of an array with pixels I get ImageBitmap object with properties { width: 512, height: 512 }

obrazek
obrazek1159Ɨ449 35.3 KB

2

Hi @NokFrt,

Check this post:

There is also support for this added in the engine recently. I think itā€™s merged, though Iā€™m not aware of a public example on how to use it. @slimbuck @mvaligursky any idea?

3

The async texture download is just an internal implementation weā€™re testing, itā€™s not ready for release. The API will most likely change as well as weā€™ll implement WebGPU version.

1 Like
4

Hi @Leonidas, thank for the reply. Your methid readPixelsFromTexture() works but I still have a problem with it.

I have a material with a diffuseMap texture. I load this material as an Asset and I donā€™t use this material on any MeshInstance.
I want to create a new texture based on the original texture from the material an then I want to use this new texture for another material.

The problem is that I canā€™t get pixels from the original texture if I donā€™t use the original material on a MeshInstance. If I try to use you readPixelsFromTexture() before using the original material, all pixels are zero.

Is there a way how to initialize the texture without applying the material to a MeshInstance?

5

Most likely the texture hasnā€™t been uploaded to the GPU before being used by a model. Can you try calling upload to your texture before accessing it?

https://developer.playcanvas.com/en/api/pc.Texture.html#upload

6 
const baseMat = assets.find(MonsterManager.BODY_ASSET_KEY, pc.eAssetType.material).resource as pc.StandardMaterial;
baseMat.diffuseMap.upload();

setTimeout((): void => { this.initBodyMaterial(); }, 1000);

initBodyMaterial() use the baseMat.diffuseMap texture for filling dynamic texture. It works if I use the baseMat on some visible MeshInstance but not without it :frowning: pc.Texture.upload() didnā€™t help.

7

One way to extract data from any texture (including compressed formats) is to render it to another texture. Iā€™m just adding code to handle this to usdz exporter, but this is the core of this:

const textureBlitVertexShader = `
    attribute vec2 vertex_position;
    varying vec2 uv0;
    void main(void) {
        gl_Position = vec4(vertex_position, 0.5, 1.0);
        uv0 = vertex_position.xy * 0.5 + 0.5;
    }`;

const textureBlitFragmentShader = `
    varying vec2 uv0;
    uniform sampler2D blitTexture;
    void main(void) {
        gl_FragColor = texture2D(blitTexture, uv0);
    }`;

const { width, height } = texture;

const dstTexture = new Texture(device, {
    name: 'ExtractedTexture',
    width,
    height,
    format: PIXELFORMAT_RGBA8,
    cubemap: false,
    mipmaps: false,
    minFilter: FILTER_LINEAR,
    magFilter: FILTER_LINEAR,
    addressU: ADDRESS_CLAMP_TO_EDGE,
    addressV: ADDRESS_CLAMP_TO_EDGE
});

const renderTarget = new RenderTarget({
    colorBuffer: dstTexture,
    depth: false
});

// render to a render target using a blit shader
const shader = createShaderFromCode(device, textureBlitVertexShader, textureBlitFragmentShader, 'ShaderCoreExporterBlit');
device.scope.resolve('blitTexture').setValue(texture);
device.setBlendState(BlendState.NOBLEND);
drawQuadWithShader(device, renderTarget, shader);

// read back the pixels
// TODO: use async API when ready
const pixels = new Uint8ClampedArray(width * height * 4);
device.readPixels(0, 0, width, height, pixels);

dstTexture.destroy();
renderTarget.destroy();
2 Likes