How can I get the webcam feed?

Help & Support
1

I’d like the background of my game to be the webcam feed, and be able to switch from front to back, etc. Does PlayCanvas support this? I don’t need to do anything with the video texture. I just needs the webcam feed to show up. I did a quicksearch and found this tutorial, but really what I want is much simpler. I don’t want to use the webcam’s video texture, I just want the webcam feed on the background. Think of a selfie app.

EDIT

I will need to be able to take a screenshot, so I guess I’m going to need the video texture inside PlayCanvas after all and need to manipulate it? Can I make the game’s virtual camera become the phone’s webcam?

2

Hi @Marks,

If the only thing you need is to render a video feed from the web camera, then you as you said you don’t really need a video texture (you can grab a screenshot directly from the feed using a Canvas object).

This isn’t so much a PlayCanvas related question, you need to use the browser native methods to query the available cameras (front/rear), switch between one or the other, and then use a Canvas object to take a screenshot when required.

You can do a google search to find related JavaScript resources on the subject. Here is one:

3

@Leonidas

(you can grab a screenshot directly from the feed using a Canvas object).

This would be true if I only needed a screenshot of the webcam feed. But there will be other stuff on the screen from PlayCanvas. It’s a 2D game that happens on the screen, but with the webcam feed as the background.

Knowing this, I still don’t need the webcam feed streamed to PlayCanvas’ engine?

4

You can do it either way:

  1. Use a video texture to render it on a PlayCanvas model/element (you have code for that).

  2. Or render it on an HTML object, when taking a screen shot use JS Canvas operations to get:

  • A screenshot from the PlayCanvas canvas
  • A screenshot from the video feed
    Add them together on a canvas and save the resulting screenshot.
2 Likes
5

I will go with method 1. How can I make the videoTexture fill the whole screen?

6

You could play the video on a UI element that fills the whole screen, check this post, it has some ideas on how to approach that:

1 Like
7

@Leonidas there was a modified example showing how to use the webcam texture. I read it somewhere but can’t find it. You were the one who posted it. Could you link it here?

EDIT
Here it is PlayCanvas 3D HTML5 Game Engine

1 Like
8

@Leonidas sorry to bother, I followed this post, but when I clicked on the guy’s project, it said scene not found. Could you lend a hand here?

9

Most likely the project author removed or made private the original project. You can try asking on that post.

Otherwise try following the steps outlined to do the same in your project.

10

@Leonidas As far as I understood, I need to create a material, attach to the UI element, then pass the video feed to the material with

this.app.on(this.playEvent, function (videoTexture) {

    var material = this.screenMaterial.resource;

    material.emissiveMap = videoTexture;

    material.update();

}, this);

However I am trying to access the .resource but it doesn’t seem to exist

this.entity.element.material.resource

There is no autocomplete for the resource property.

This is my current set up

image
image987×840 32.4 KB

11

Solved using this.entity.element.texture = videoTexture;

1 Like
12

@Leonidas Another question guys. I’d like to mirror the webcam feed on the x axis. There is a flipY parameter, but no flipX T_T.

13

Yeah, that parameter was put in place to keep older compatible with engine changes.

For mirroring on X I think you will have to use a custom shader (GLSL, I don’t have an example in mind).

Otherwise another option may be to use a custom material on your UI element, and play with diffuseMap rotation/tiling/offset. You may be able to mirror it with some negative values. @mvaligursky any other trick in mind?

Edit: I see you get some quality help on Discord, plenty of tricks :wink:

2 Likes
14

@Leonidas Now the problem that I’m having is that the camera feed stretches and doesn’t maintain the aspect ratio of the video. I need it to cover the screen but maintaining the aspect ratio of the video, and overflow the boundaries of the screen if necessary. I thought having the anchor 0,0,1,1 would work, but it covers the screen without maintaining the aspect ratio of the vid. Something more needs to be done here.

15

Try using this code that works in a similar way to CSS cover: Responsive UI for all device in PlayCanvas - #6 by yaustar

2 Likes
16

I’m having an issue here. I managed to have the webcam feed on a video texture that covers the whole screen, but now I can’t see the particles that are on the world.

I’m not sure why, since the webcam feed is on a image element like all other images, but only the webcam feed blocks the particles

17

Are the particles rendering on a layer that is in front of the webcams UI layer?

18

The particles are rendering on the world layer. I tried changing to the UI layer as well. Didn’t work. I tried moving the particles above and below the webcam feed in the hierarchy, didn’t work. I tried using screen space, didn’t work.

19

@yaustar Ok I managed to do it. I created a new layer that is just like the UI layer, but above the UI layer, and assigned it to the webcam image. Not sure why this was necessary at all, because having the particles on the UI layer and below the webcam feed on the hierarchy should have rendered the partciles on top of the webcam feed, but it didn’t. Can you tell me why? Anyways, problem solved.

EDIT

In fact…the particles are always rendered behind other UI elements, even if the particles use the UI layer and is below other UI elements on the hierarchy(when talking about UI elements, elements below on the hierarchy are drawn on top of elements above). Is this behavior expected?

20

Mostly likely an order of render issue. The UI layer is set to manual order which the UI screen components control the elements that are their children

The particles aren’t controlled by the screen component so chances were that it rendered first, then the webcam feed element which would cover it.

Expected to be undefined/random, yes

1 Like