[SOLVED] Toon Shading Techniques

dave · December 2, 2015, 2:50pm

Continuing the discussion from FBX import issue?:

There is a simple example of Toon Shading here using the technique @max describes of duplicating the model.

nakata0705 · December 2, 2015, 5:12pm

Thank you very much for the shader example. It’s very helpful and I’ll check the code later.

I was working on the similar approach based off of PlayCanvas custom shader example. My first objective is rendering a model without a shade.

What I trying to do was find the material imported from FBX and applied to the model’s face. Then replace a shader associated with the material with a new “shadeless” shader. My goal is being able to render any imported FBX material without shadow by replacing a shader of the existing material.

It seems my code successfully created a shader instance but it didn’t change how the model is rendered. (https://playcanvas.com/editor/code/367166/shadeless_shader.js)

So my first question is, is it possible to replace a shader of the existing material and render? Both of your toon sample and custom shader sample creates a new material in a code. So I want to confirm my approach is valid.

Just for your information, the FBX model’s face I’m trying to render is optimized for “shadeless” rendering while other part of body is expected to be rendered with toon shading. (I’m not a model creator and still trying to understand the model)

(Below is a Blender render image with the default shading + back face culling. The floating eyes and mouth has shape key animation for its facial animation. Notice nose is also a floating polygon)

(Below is a Blender render image with the default shading + no back face culling. As you see, the model already has a black mesh for nose and cheek edge. Notice the static nose+mouth mesh is now visible. Its normal is facing inside of the model and used to render the skin color when the face is rendered from side)

(Below is a Blender render image with shadeless + back face culling. This is the closest to the desired result. The face skin color comes from a flat polygon behind eyes and mouth. The shade on forehead is actually a colored polygon)

nakata0705 · December 3, 2015, 8:09am

Hello Dave,

I created a “shadeless” shader and applied to some meshinstances in my project. And I realized that the meshinstances stopped animating just like a screenshot below. (Though the shading itself became “shadeless” as expected)

It seems that I have to do something more than your example to apply a custom shader while keeping the skeltal animation. Would you let me know how? Would it be the case that PlayCanvas is using vertex shader to apply skeltal animation?

By the way, I tried to apply the custom shader to the FBX imported material and it didn’t work. (line 12 and line 35 of https://playcanvas.com/editor/code/367166/shadeless_shader.js) Would you also let me know why it didn’t work? Thanks a lot!

Link to the project: https://playcanvas.com/project/367166/overview/model-test-3

dave · December 3, 2015, 9:37am

Yes, that is exactly it.

Skinned animation is done in the vertex shader, so you probably don’t want to replace that with your own.

I’ll have to defer to @max or @Mr_F to let you know the best way to replace the fragment shader of an existing material.

We’re working on another way of doing this which Max eluded to before which is the shader chunk system. Unfortunately the API for that isn’t finished completely so I’m hesitant to recommend you use that at the moment.

max · December 3, 2015, 8:33pm

I’ve tried replacing some global shaderChunks that is used for materials.
It basically does pasteurisation of light, making it graduated.
Here is the project: https://playcanvas.com/project/367538/overview/toon-shading

nakata0705 · December 4, 2015, 6:47am

Thank you Max.

With your code and Shader Chunk API in “Plasma Shader” example, I finally got the desired rendering result.
I applied the “shadeless” shader to the face, the eyes and the nose materials. And applied your toon shader to all other materials. (with 3.0 toonSteps and minimum diffuse value 0.5)

Link to the project

I will start working on the particle effect next. Thank you very much for you continuous help. I hope the facial animation becomes available soon!

alonerone · June 19, 2017, 1:34pm

Hello guys, is there any reason why the code doesn’t work anymore ? I’ve tried to compile both max and nakata0705 projects and I get an error … I’m an interactive designer with a (small) technical background and I’m looking for a way to get a nice toon shader . I need it for some animated meshes so the shaderChunk alternative looked perfect to me … Anyway thanks for what you’ve already done.

will · June 21, 2017, 5:41pm

It looks like the shader chunk system has changed since @nakata0705 created that app. Hey, @Mr_F - what needs to be changed to get it working again?

Mr_F · June 22, 2017, 4:40pm

It seems, @nakata0705 had a somewhat unusual toon shading method using special flipped geometry. Here you go, I made an example that works with skinning: https://playcanvas.com/project/489836/overview/toon-shader-with-skinning

devMidgard · March 3, 2019, 4:44pm

Hey, in case anybody wants to use @Mr_F’s shader, I noticed it was broken for playcanvas rev 1.0+ (it was originally made for 0.201-something), so I fixed it.

Here’s the public fixed project compiled with today’s latest revision: https://playcanvas.com/project/605855/overview/toon-shader-skinned

eproasim · October 7, 2020, 8:47pm

@devMidgard

Good afternoon,

Your skinning solution no longer works with PlayCanvas 1.35, but I was able to get it working again by downgrading to 1.28. I have no idea why.

It looks like it has something to do with the getBoneMatrix function in the vertex shader, but my lack of experience there:


ERROR: 0:83: 'getBoneMatrix' : no matching overloaded function found
ERROR: 0:84: 'getBoneMatrix' : no matching overloaded function found
ERROR: 0:85: 'getBoneMatrix' : no matching overloaded function found
ERROR: 0:86: 'getBoneMatrix' : no matching overloaded function found
ERROR: 0:83: 'return' : function return is not matching type:

and

ERROR: 0:106: 'getBoneMatrix' : no matching overloaded function found
ERROR: 0:107: 'getBoneMatrix' : no matching overloaded function found
ERROR: 0:108: 'getBoneMatrix' : no matching overloaded function found
ERROR: 0:109: 'getBoneMatrix' : no matching overloaded function found
ERROR: 0:106: 'return' : function return is not matching type:

I’m sure it isn’t right, but I was able to clear the return errors by explicitly declaring the value as mat4. Unfortunately no amount of my inexperienced fumbling with the getModelMatrix() function was able to get it working again.

I hope any of the inforation I provided was helpful. I’d really like to be able to use a toon shader for some skinned assets.

pixelpros · October 27, 2020, 5:52am

They had updated the functionality of getModelMatrix()

To fix this, replace the skinned Vertex Shader (transformToonEdgeSkinnedVS) with this:

uniform float edgeSize;

mat4 getModelMatrix() {
	#ifdef DYNAMICBATCH
	return getBoneMatrix(vertex_boneIndices);
	#elif defined(SKIN)
	return matrix_model * getSkinMatrix(vertex_boneIndices, vertex_boneWeights);
	#elif defined(INSTANCING)
	return mat4(instance_line1, instance_line2, instance_line3, instance_line4);
	#else
	return matrix_model;
	#endif
}

vec4 getPosition() {
    dModelMatrix = getModelMatrix();
    vec4 posW = dModelMatrix * vec4(vertex_position, 1.0);
    //posW.xyz += skinPosOffset;
    dNormalMatrix = mat3(dModelMatrix[0].xyz, dModelMatrix[1].xyz, dModelMatrix[2].xyz);
    vec3 normal = normalize(dNormalMatrix * vertex_normal);
    posW.xyz += (normal / 100.0) * edgeSize;
    dPositionW = posW.xyz;
    return matrix_viewProjection * posW;
}

vec3 getWorldPosition() {
    return dPositionW;
}

Notice that the ‘dModelMatrix’ is directly calling getModelMatrix() now, it does not need to be multiplied as that is already handled in the matrix function.

Edit: Credit to @kprimo for exposing the PC shaders Standard Material Shaders Share

eproasim · October 27, 2020, 10:48am

Thank you so much for the insight and the exposed PC shaders! It’s much appreciated!

Juanix123 · March 2, 2021, 10:02pm

Hi, I’m trying to use this scripts in my new project but them don’t seems to work now or I’m doing something wrong, here is an attach screen of the error

any help or idea?
thnks for your time!

devMidgard · April 2, 2021, 7:10pm

If you notice in the initialize method, the script needs all these resources to be inserted in the PlayCanvas Editor:

    var transformToonEdgeVS = this.app.assets.find("transformToonEdgeVS").resource;
    var transformToonEdgeSkinnedVS = this.app.assets.find("transformToonEdgeSkinnedVS").resource;
    var lightDiffuseToonPS = this.app.assets.find("lightDiffuseToonPS").resource;
    var edgePS = this.app.assets.find("edgeToonPS").resource;
    var ramp = this.ramp.resource;

By looking at your screenshot it seems that the ramp resource is not set. Please look at the script in the Editor and assign a color ramp texture, in the example project I have public there’s one, feel free to use that!

Charlie_Van_Norman · September 29, 2023, 3:05pm

Hi everyone, it appears these solutions are outdated and no longer work. @Playcanvas mods should we post something about how toon shading can work now? For example the entire API for “model” vs “render” is now different … Here is the Github.io version: PlayCanvas Examples