Video Chromakey Shader

Hi, i need a working example of a video being chromakeyed to transparency (from green screen video). While i found a handful of examples, none of them work. Not on mac, not on windows. Planes just stay black. May be they are to old.

Anybody having a working example?

i solved it myself. for anybody interested here is what you need to do:

  1. create a scene with one entity you want the image to apear on. Typically this is a plane.

  2. create a shader file. Name it “veChromakey”
    and paste this code inside the file:

attribute vec3 aPosition;
attribute vec2 aUv0;

uniform mat4 matrix_model;
uniform mat4 matrix_viewProjection;

varying vec2 vUv0;

void main(void)
{
    vUv0 = aUv0;
    gl_Position = matrix_viewProjection * matrix_model * vec4(aPosition, 1.0);
}

Thats the vertex shader. Nothing special.

  1. create another shader and name it “frChromakey” which is the fragment shader.
    Paste this inside:
varying vec2 vUv0;

uniform sampler2D uDiffuseMap;
uniform float uTime;


void main(void)
{

    vec3 tColor = texture2D(uDiffuseMap, vUv0).rgb;
   float a = (length(tColor - vec3(0.1,0.9,0.2)) - 0.5) * 7.0;

    gl_FragColor = vec4(tColor, a);
}

the color value to kay out is hardened here, because i could not manage to do this with attributes/parameters.

Lastly create a script file and paste this:

var ChromaKey = pc.createScript('chromaKey');


ChromaKey.attributes.add('vs', {
    type: 'asset',
    assetType: 'shader',
    title: 'Vertex Shader'
});

ChromaKey.attributes.add('fs', {
    type: 'asset',
    assetType: 'shader',
    title: 'Fragment Shader'
});

ChromaKey.attributes.add('diffuseMap', {
    type: 'asset',
    assetType: 'texture',
    title: 'Diffuse Map'
});


// initialize code called once per entity
ChromaKey.prototype.initialize = function() {

    var app = this.app;
    var model = this.entity.model.model;
    var gd = app.graphicsDevice;

    var diffuseTexture = this.diffuseMap.resource;

    var vertexShader = this.vs.resource;
    var fragmentShader = "precision " + gd.precision + " float;\n";
    fragmentShader = fragmentShader + this.fs.resource;

    // A shader definition used to create a new shader.
    var shaderDefinition = {
        attributes: {
            aPosition: pc.SEMANTIC_POSITION,
            aUv0: pc.SEMANTIC_TEXCOORD0
        },
        vshader: vertexShader,
        fshader: fragmentShader
    };

    // Create the shader from the definition
    this.shader = new pc.Shader(gd, shaderDefinition);

    // Create a new material and set the shader
    this.material = new pc.Material();
    this.material.setShader(this.shader);

    // Set the initial time parameter
    this.material.setParameter('uTime', 0);

    // Set the diffuse texture
    this.material.setParameter('uDiffuseMap', diffuseTexture);

    this.material.blendType = pc.BLEND_NORMAL;
    // Replace the material on the model with our new material
    model.meshInstances[0].material = this.material;
};

// update code called every frame
ChromaKey.prototype.update = function(dt) {

};

Now attach this script to your object you wish to have a chromakey texture on. Then populate the slots with the vertex and fragment shader. lastly you should have a dummy green screen texture. Means an image. Green background - which will be keyed out - and then some other colours in there you wish to see.

And the main trick to get this to work is this line:
this.material.blendType = pc.BLEND_NORMAL;

Hope it helps.

2 Likes

This has way better quality in keying than any example i could find for play canvas. By the way, it is just a port of a shader from A-Frame.

and it also works with video but then you need to rewrite the script with the video part

Thanks for posting the solution! Much appreciated!