PlayCanvasAR - loading textures and hiding the camera feed

Hi there,

My name is Mick, I’m a trainee software developer. I’m making an AR art gallery webapp with Vuejs. To do this I stumbled upon playcanvas AR, which I’m no trying to implement.

Everything seems to work fine, but I can’t figure out how I can add a texture (jpg of the art I want to display in AR) to the box and how I can switch off the device camera (e.g. phone camera) when exiting the AR mode.

My current code looks like this:

To insert code type:

<template>
  <canvas style="overflow: hidden;">
  </canvas>
</template>

<script>
  export default {
    beforeDestroy () {
      console.log('hallo')
      var video = document.querySelector('video')
      var canvas = document.querySelector('canvas')
      console.log(video)
      document.body.removeChild(video)
      document.body.removeChild(canvas)
      console.log(this.constraints)
      // navigator.mediaDevices.getUserMedia(this.constraints)
    },
    mounted () {
      let vm = this
      /* eslint-disable */
      
      // Create the canvas for our application 
      var canvas = document.querySelector('canvas')
      canvas.style.position = "absolute"
      canvas.style.top = "0px"
      canvas.style.left = "0px"
      // document.body.appendChild(canvas)

      // Create the application and start the update loop
      var app = new pc.Application(canvas)
      app.setCanvasFillMode(pc.FILLMODE_FILL_WINDOW)
      app.setCanvasResolution(pc.RESOLUTION_AUTO)
      app.start()

      // Lift scene ambient lighting up from black
      app.scene.ambientLight = new pc.Color(0.2, 0.2, 0.2)

      // Load PlayCanvasAR library scripts
      app.assets.loadFromUrl("static/playcanvas-ar.js", "script", function (err, asset) {
        // Create the AR-enabled camera using generic camera calibration settings
        app.assets.loadFromUrl("static/camera/camera_para_iphone.dat", "binary", function (err, asset) {
          var camera = new pc.Entity("AR Camera")
          camera.addComponent("camera", {
            clearColor: new pc.Color(0, 0, 0, 0) 
          })
          camera.addComponent("script")
          camera.script.create("arCamera", {
            attributes: {
              cameraCalibration: 
              asset,
              detectionMode: 0, // Mono template
              labelingMode: 1, // Black Region
              processingMode: 0, // Frame
              thresholdMode: 0, // Manual
              threshold: 125,
              trackerResolution: 2, // Full 
              trackAlternateFrames: true,
              debugOverlay: true,
              videoTexture: false
            }
          })
          app.root.addChild(camera)
        })

        // Load the Hiro marker pattern and create a marker entity with it 
        app.assets.loadFromUrl("static/marker/patt.sbk3", "binary", function (err, asset) {
          var hiro = new pc.Entity("Hiro Marker")
          hiro.addComponent("script")
          hiro.script.create("arMarker", {
            attributes: {
              pattern: asset,
              width: 1,
              deactivationTime: 0.25,
              shadow: false,
              shadowStrength: 0.5
              }
          })
          app.root.addChild(hiro)
          // Create a box as a child of the Hiro marker 
          var box = new pc.Entity("box")
          box.addComponent("model", {
            type: "box",
          })
          box.setLocalPosition(0, 0, 0)
          box.setLocalScale(vm.scaleWidth, 0.1, vm.scaleHeight)
          hiro.addChild(box)

          // Create a directional light 
          var light = new pc.Entity()
          light.addComponent("light", {
            type: "directional",
            color: new pc.Color(1, 1, 1),
            castShadows: true,
            shadowBias: 0.2,
            normalOffsetBias: 0.1,
            shadowDistance: 10,
            shadowType: pc.SHADOW_PCF5 
          })
          light.setLocalEulerAngles(22, 0, -16)
          hiro.addChild(light

          // load texture from  url
          app.assets.loadFromUrl("static/material/painting.json", 'material', function (err, asset) {
            var material = new pc.Material()
            material.diffuseMap = asset.resource
            material.update()
            box.model.model.meshInstances[0].material = material
          })
        })
      })

      // Resize the canvas when the window is resized
      window.addEventListener("resize", function () {
        app.resizeCanvas(window.innerWidth, window.innerHeight)
      })
    }
  }
</script>

<style scoped>
</style>

I hope you guys can help me out.

Thnx,

Mick

To load a texture, do something like:

        app.assets.loadFromUrl("../assets/clouds.jpg", "texture", function (err, asset) {
            var material = new pc.PhongMaterial();
            material.diffuseMap = asset.resource;
            material.update();

            box.model.model.meshInstances[0].material = material;
        });

I got this from one of the engine examples:

To hide the camera video feed, I did add the ‘exitAr’ function:

You could try calling that:

camera.script.arCamera.exitAr();

Alternatively, you can just hide the video element with CSS. It’s created here:

To hide it, you’d do something like:

camera.script.arCamera.video.style.display = 'none';

Hi Will,

Thnx for the advice, I’ll try it out immediately.
I already managed to remove the video stream on destroy, but somehow the webcam of my MacBook stays on.
So I managed to remove the video stream like this:

 beforeDestroy () {
      var video = document.querySelector('video')
      var canvas = document.querySelector('canvas')
      document.body.removeChild(video)
      document.body.removeChild(canvas)

And about the material, I tried your example. But I’m confused about how to assign the material to the box, since the box is a child of the Hiro marker.
So should I use:

box.model.model.meshInstances[0].material = material

or
something like:

hiro.box.model.model.meshInstances[0].material = material

Thanks again :slight_smile:

EDIT:
The loading of the material works perfect with box.model.ect… Thank you so much.

loadFromUrl("this.painting.img", "texture", function (err, asset ) {}

But this, of course, won’t work, because load from url looks for a map and not a dynamic prop.

Another problem I encounter… The box is not printer straight on the marker, it is turned a bit. And when I move the marker the box moves away from the marker instead of being fixed on the marker.

EDIT 2:
This screenshot illustrates how my object currently gets printed on the marker, as you can see the object is placed with an angle instead of flat

It’s really hard for me to figure out why the rotation is happening without being able to run the app and debug. Can you provide a URL?

If the rotation is constant, i.e. it does not matter how you rotate the device, it’s always offset, then you need to rotate the model compared to the marker reference I guess.