Capture a Screenshot iPhone Not working

Hello :wave:

I am trying to capture image in perspective camera mode with UI.

But it is not working on iPhone.
When I try to capture image on iPhone it just reloads the page completely.

https://playcanvas.com/project/997760/overview/screenshot-iphone-bug

Device info:

Any help will be appreciated :pray:

Works on my iPhone 8, iOS 15.

You will need to try to catch any errors that could be fired in the dev console and track down what it could be.

I would also try commenting out code in the screenshot code such as the part where it converts it to a PNG and try to isolate which code section is causing the issue.

Can you also try restarting your device and closing background browser tab and background apps to see if it’s memory related

1 Like

Hello @yaustar :wave:
Thanks for reply!

I went through different things…
The problem is I unable to track down deeper because its client device.

And I come to know that. the code right before creating download link is creating issue… It completely reloads the page… This happens in chrome and safari for that specific device.

How it can be fixed?

As I can’t reproduce it locally, I’m afraid I can’t really help or investigate.

If you can tell me the last executing line of code before it crashes, maybe I can advise?

Have you tried reducing the resolution of the screengrab/capture?

Nope…

But I try to capture screenshot on pc using this setting
image

The captured screenshot png size is more than 4MB and its resolution is 1440x2560…

In my project i have width 720 height 1280
image

So, where i can set force resolution for screenshot i capture to 720x1280 ?

=========================================================

EDIT1:

I created sample project contains light, models animation, tween, bg music , ui.

Project:
https://playcanvas.com/project/999322/overview/capturingscreenshot-bug-bkp-test

Play directly: https://playcanv.as/p/3qkmNXjf/

I shared this with client for testing on iPhone 11 pro max.
Result:

  • The capture system worked on safari but only for 2-3 clicks (took pause for few seconds before clicking again on btn) in one go… Then page reloads…
  • Does not work on chrome… Upon clicking on btn page reloads…
  • On safari the generated png is very big approx 8 MB in size.

I am not sure it is memory issue or what…
Any help will be appreciated :pray:

The Max models generally have had some odd memory related issues for some reason. Lower end iOS devices seem to be fine when Max models seem to run into issues more often than not

With the screenshot code:

var Screenshot = pc.createScript('screenshot');
Screenshot.attributes.add('cameraEntity', {
    type: 'entity',
    description: 'The camera entity to use for taking the screenshot with. Whatever, this camera renders will be in the screen capture.'
});


Screenshot.prototype.initialize = function () {
    this.createNewRenderTexture();

    this.app.graphicsDevice.on('resizecanvas', function (w, h) {
        this.secsSinceSameSize = 0;
    }, this);

    var device = this.app.graphicsDevice;
    this.lastWidth = device.width;
    this.lastHeight = device.height;

    this.secsSinceSameSize = 0;

    this.triggerScreenshot = false;

    var onTakeScreenshot = function () {
        this.triggerScreenshot = true;
        this.cameraEntity.enabled = true;
    };

    this.app.on('ui:takeScreenshot', onTakeScreenshot, this);
    this.app.on('postrender', this.postRender, this);

    // Disable the screenshot camera as we only want it enabled when we take the screenshot itself
    this.cameraEntity.enabled = false;

    // Ensure it gets rendered first so not to interfere with other cameras
    this.cameraEntity.camera.priority = -1;

    // Add a <a> to use to download an image file
    var linkElement = document.createElement('a');
    linkElement.id = 'link';
    window.document.body.appendChild(linkElement);

    // Clean up resources if script is destroyed
    this.on('destroy', function () {
        this.app.off('ui:takeScreenshot', onTakeScreenshot, this);
        this.app.off('postrender', this.postRender, this);

        window.document.body.removeChild(linkElement);

        if (this.renderTarget) {
            this.renderTarget.destroy();
            this.renderTarget = null;
        }

        if (this.colorTexture) {
            this.colorTexture.destroy();
            this.colorTexture = null;
        }

        if (this.depthTexture) {
            this.depthTexture.destroy();
            this.depthTexture = null;
        }

        this.canvas = null;
        this.context = null;

    }, this);
};


// update code called every frame
Screenshot.prototype.update = function (dt) {
    // We don't want to be constantly creating an new texture if the window is constantly
    // changing size (e.g a user that is dragging the corner of the browser over a period)
    // of time. 

    // We wait for the the canvas width and height to stay the same for short period of time
    // before creating a new texture to render against.

    var device = this.app.graphicsDevice;

    if (device.width == this.lastWidth && device.height == this.lastHeight) {
        this.secsSinceSameSize += dt;
    }

    if (this.secsSinceSameSize > 0.25) {
        if (this.unScaledTextureWidth != device.width || this.unScaledTextureHeight != device.height) {
            this.createNewRenderTexture();
        }
    }

    this.lastWidth = device.width;
    this.lastHeight = device.height;
};


Screenshot.prototype.postRender = function () {
    if (this.triggerScreenshot) {
        this.takeScreenshot('screenshot');
        this.triggerScreenshot = false;
        this.cameraEntity.enabled = false;
    }
};


Screenshot.prototype.createNewRenderTexture = function () {
    var device = this.app.graphicsDevice;

    // Make sure we clean up the old textures first and remove 
    // any references
    if (this.colorTexture && this.depthTexture && this.renderTarget) {
        var oldRenderTarget = this.renderTarget;
        var oldColorTexture = this.colorTexture;
        var oldDepthTexture = this.depthTexture;

        this.renderTarget = null;
        this.colorTexture = null;
        this.depthTexture = null;

        oldRenderTarget.destroy();
        oldColorTexture.destroy();
        oldDepthTexture.destroy();
    }

    // Create a new texture based on the current width and height
    var colorBuffer = new pc.Texture(device, {
        width: device.width,
        height: device.height,
        format: pc.PIXELFORMAT_R8_G8_B8_A8,
        autoMipmap: true
    });

    var depthBuffer = new pc.Texture(device, {
        format: pc.PIXELFORMAT_DEPTHSTENCIL,
        width: device.width,
        height: device.height,
        mipmaps: false,
        addressU: pc.ADDRESS_CLAMP_TO_EDGE,
        addressV: pc.ADDRESS_CLAMP_TO_EDGE
    });

    colorBuffer.minFilter = pc.FILTER_LINEAR;
    colorBuffer.magFilter = pc.FILTER_LINEAR;
    var renderTarget = new pc.RenderTarget({
        colorBuffer: colorBuffer,
        depthBuffer: depthBuffer,
        samples: 4 // Enable anti-alias 
    });

    this.cameraEntity.camera.renderTarget = renderTarget;

    this.unScaledTextureWidth = device.width;
    this.unScaledTextureHeight = device.height;

    this.colorTexture = colorBuffer;
    this.depthTexture = depthBuffer;
    this.renderTarget = renderTarget;

    var cb = this.renderTarget.colorBuffer;

    if (!this.canvas) {
        // Create a canvas context to render the screenshot to
        this.canvas = window.document.createElement('canvas');
        this.context = this.canvas.getContext('2d');
    }

    this.canvas.width = cb.width;
    this.canvas.height = cb.height;

    // The render is upside down and back to front so we need to correct it
    this.context.globalCompositeOperation = "copy";
    this.context.setTransform(1, 0, 0, 1, 0, 0);
    this.context.scale(1, -1);
    this.context.translate(0, -this.canvas.height);

    this.pixels = new Uint8Array(colorBuffer.width * colorBuffer.height * 4);
};


// From https://forum.playcanvas.com/t/save-specific-rendered-entities-to-image/2855/4
Screenshot.prototype.takeScreenshot = function (filename) {
    var colorBuffer = this.renderTarget.colorBuffer;
    var depthBuffer = this.renderTarget.depthBuffer;

    // Fix for WebKit: https://github.com/playcanvas/developer.playcanvas.com/issues/268
    // context must be cleared otherwise the first screenshot is always used

    // https://stackoverflow.com/a/6722031/8648403
    // Store the current transformation matrix
    this.context.save();

    // Use the identity matrix while clearing the canvas
    this.context.setTransform(1, 0, 0, 1, 0, 0);
    this.context.clearRect(0, 0, colorBuffer.width, colorBuffer.height);

    // Restore the transform
    this.context.restore();

    var gl = this.app.graphicsDevice.gl;
    var fb = this.app.graphicsDevice.gl.createFramebuffer();
    var pixels = this.pixels;

    // We are accessing a private property here that has changed between
    // Engine v1.51.7 and v1.52.2
    var colorGlTexture = colorBuffer.impl ? colorBuffer.impl._glTexture : colorBuffer._glTexture;
    var depthGlTexture = depthBuffer.impl ? depthBuffer.impl._glTexture : depthBuffer._glTexture;

    gl.bindFramebuffer(gl.FRAMEBUFFER, fb);
    gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, gl.TEXTURE_2D, colorGlTexture, 0);
    gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.DEPTH_STENCIL_ATTACHMENT, gl.TEXTURE_2D, depthGlTexture, 0);
    gl.readPixels(0, 0, colorBuffer.width, colorBuffer.height, gl.RGBA, gl.UNSIGNED_BYTE, pixels);

    gl.deleteFramebuffer(fb);

    // first, create a new ImageData to contain our pixels
    var imgData = this.context.createImageData(colorBuffer.width, colorBuffer.height); // width x height
    var data = imgData.data;

    // Get a pointer to the current location in the image.
    var palette = this.context.getImageData(0, 0, colorBuffer.width, colorBuffer.height); //x,y,w,h

    // Wrap your array as a Uint8ClampedArray
    palette.data.set(new Uint8ClampedArray(pixels)); // assuming values 0..255, RGBA, pre-mult.

    // Repost the data.
    this.context.putImageData(palette, 0, 0);
    this.context.drawImage(this.canvas, 0, 0);

    var image = this.canvas.toDataURL('image/png');
    var b64 = this.canvas.toDataURL("image/png").replace("image/png", "image/octet-stream");

    // Thanks https://stackoverflow.com/a/44487883
    var link = document.getElementById('link');
    link.setAttribute('download', filename + '.png');
    link.setAttribute('href', b64);
    link.click();
};

Function createNewRenderTexture creates the texture size and canvas size to make the screenshot.

You can see where device.width/height is used where the textures are created and the canvas.

For testing purposes, I would just make this a small fixed size such as 512 x 512 and see if that helps.

If it does, I would add some extra logic that ensures the sizes isn’t higher than a certain value in either dimension.

1 Like

@yaustar Thanks for explaining it.

As you mentioned i modify this.

Screenshot.prototype.createNewRenderTexture = function () {
    
	...
	
    // Create a new texture based on the current width and height
    var colorBuffer = new pc.Texture(device, {
        // width: device.width,
        // height: device.height,
        width: 512,
        height: 512,
        format: pc.PIXELFORMAT_R8_G8_B8_A8,
        autoMipmap: true
    });

    var depthBuffer = new pc.Texture(device, {
        format: pc.PIXELFORMAT_DEPTHSTENCIL,
        // width: device.width,
        // height: device.height,
        width: 512,
        height: 512,
        mipmaps: false,
        addressU: pc.ADDRESS_CLAMP_TO_EDGE,
        addressV: pc.ADDRESS_CLAMP_TO_EDGE
    });

    ...
};

Nice! It generated PNG with 512 x 512.

You can check the output here: https://playcanv.as/p/3qkmNXjf/

It will take some time to test on actual iPhone device… I will let you know the result…

Thank you

EDIT:

@yaustar Thank you very much tested on 11 pro max, iPhone 13 and 12 mini both browsers works fine
I am using 1280 x 720
https://playcanv.as/p/3qkmNXjf/

Thanks

Hello,

I am using screen capture thing in game and it is working fine on android, iOS normal browsers (chrome, safari, Firefox).

But now game URL is shared on Instagram. When people opens link from Instagram and try to capture it this is what error we are getting.

This is happening for Android device.
Also I come to know that link is opening inside in app browser.

image

This game also have share button but it is also not working…

image

Any help will be appreciated :pray:

Sounds like an issue with Instagram API and you will have to talk to their community to get help

1 Like