Taking photo/screenshot in AR

Help & Support
1

Hello,

I’ve been trying to figure out how to take a screenshot when using AR. The point is to include the virtual entities in the photo/screenshot. So for example, you can place a tree or a car on your desk and take a photo/screenshot with the said tree or car.

Unfortunately, it seems that I am stuck at the very beginning. I’ve created a small UI element, a 2d screen with a text that says “Take Screenshot”.

s1

s2
s2516×1196 32.2 KB

It seems that the UI Text element is considered a “pc.ElementComponent.” So looking at the docs pc.ElementComponent it should have a " touchstart" event.

Before worrying about taking a screenshot, I just want to display an alert when the UI Element with the text “Take Screenshot” is tapped but it doesn’t work and I’m not really sure what I’m doing wrong.

Here is a part of the code:

CreateObject.prototype.initialize = function () {
    this.app.mouse.on('mousedown', this.handleClick, this);

    const button = this.app.root.findByName('Text');
    button.on('touchstart', () => alert('hello'), this);
};

It doesn’t do anything, which makes me think that I’m using on touchstart event incorrectly but I don’t really know what it is that I am messing up.

Here is the project.

2

Hi @Kris,

To subscribe to mouse or touch events on a UI element you need to check the Enable Input property on your element component, and then use the following event handler:

    // touch events
    this.entity.element.on('touchstart', this.onPress, this);
    this.entity.element.on('touchend', this.onRelease, this);

Here is a helpful tutorial on the subject: https://developer.playcanvas.com/en/tutorials/ui-elements-buttons/

1 Like
3

Thank you @Leonidas I got it to working with your help.

I think what had me lost previously was the fact that I was calling an incorrect thing.

I was referring to “Entity” whereas I should have been referring to “Element component”. At least this is how I understand it.

btn
btn2195×518 83.5 KB

This is useful to know.

1 Like
4

So, continuing with this topic, my next step is to try capture a screenshot including both the virtual elements placed in the canvas and the camera feed.

I have taken the code from this project: capturing a screenshot and this is what I have:

const Button = pc.createScript('button');

Button.prototype.initialize = function() {
    const buttonElement =  this.app.root.findByName('Button').element;
    buttonElement.on('touchstart', this.takeScreenshot, this);
};

Button.prototype.takeScreenshot = function() {
    const newWindow = window.open('', '');
    newWindow.document.title = "Screenshot";
    newWindow.document.body.style.margin = "0";

    const canvas = this.app.graphicsDevice.canvas; 
    const img = new Image();
    img.src = canvas.toDataURL();
    
    newWindow.document.body.appendChild(img);
};

project

Now, I was probably a bit naïve thinking I could just do this, and sure enough, the screenshot only captures the virtual elements, but it does not capture the camera feed.

screen
screen360×512 12.1 KB

What would I need to look into to capture both the camera feed and the virtual elements? For example, I want to place some cubes and my desk and take a screenshot.

5

The camera feed is probably a separate DOM object so that is a whole of complexity. Does 8th Wall not provide an API for this?

1 Like
6

Seems like this is what I need to look into: https://www.8thwall.com/docs/web/#customize-video-recording, although the examples there include a-frame setup, so I guess, I’ll need to figure out how to use that in PC.

/edit on a second thought, maybe this would be more appropriate for me https://www.8thwall.com/docs/web/#xr8canvasscreenshot. Will try to experiment.

7

So this is indeed a bit complex to implement but definitely doable.

What essentially you need to implement is a separate DOM canvas element where you will paste:

  • the canvas output from the capturing a screenshot project
  • the current frame from the active video steam

Then you will save to image the output of this new canvas. Search online for examples on how to combine different canvas and stream elements together.

1 Like
8

Thank you Leonidas. Thankfully 8th Wall provides a camera pipeline module that can generate a screenshot (including camera feed and virtual objects) so I don’t have to result to implementing separate DOM elements.

In case anybody reads this thread, here is my code. It takes screenshot when user taps the UI button. Note, there is probably a better way to write/implement this, but this is what works for me at the moment.

// jshint esversion: 6, unused: true, varstmt: true
const Button = pc.createScript('button');

XR8.CanvasScreenshot.pipelineModule();
XR8.addCameraPipelineModule(XR8.canvasScreenshot().cameraPipelineModule());


Button.prototype.initialize = function() {
    const buttonElement =  this.app.root.findByName('Button').element;
    buttonElement.on('touchstart', this.takeScreenshot, this);
};

Button.prototype.takeScreenshot = function() {
    XR8.canvasScreenshot().takeScreenshot().then(
        data => { 
            const newWindow = window.open('', '');
            newWindow.document.title = "Screenshot";
            newWindow.document.body.style.margin = "0";
            
            const img = new Image();
            img.src = 'data:image/jpeg;base64,' + data;
            
            newWindow.document.body.appendChild(img);
        },
        error => {
            console.log(error);
        });

};
3 Likes
9

Thanks for sharing!

1 Like
10

Hi,
Im actually trying to detect a business card. The image of the card is not stored in the 8 th wall. Initially the 8th wall camera has to be turned on and when any random card is displayed, 8 th wall has to know that it detected a card , but not a toy or phone. Once it detects a card I want to display a message saying that card is detected. Can anyone suggest me how to achieve this. I am thinking of doing canny edge detection , but dont know how to do this in 8 th wall

11

Might be best asking on the 8th Wall Slack?

1 Like