Blocking ui input (virtual joystick and screen gesture input)

Help & Support
, , ,
1

Hi there!

I have a virtual joystick in my application that controls the movement of a character, and it works pretty well.

Additionally, I’ve programmed a screen area used to look around (orbit the camera). The problem is that when using the virtual joystick, it’s interpreted as screen space, which causes screen orbiting while moving the player.

Do you have any ideas or suggestions on how to combine both and make them work at the same time? It seems I would have to somehow exclude the virtual joystick from the screen calculation or detection for this.

I know there’s e.stopPropagation(), but it causes the screen rotation to stop working while using the virtual joystick. Is there a simple workaround I haven’t thought about, or would I need to define areas for screen detection and exclude the virtual joystick area?

Thanks for your input!

2

Hi @Question2u ! Can I see your current code for movement?

3

Hi @Codeknight999

Thanks for your help!

Basically, I’m using this joystick script as base and modified it for my needs, which I found in an example from PlayCanvas:

var TouchJoystick = pc.createScript('touchJoystick');
TouchJoystick.attributes.add('identifier', { 
    type: 'string', 
    default: 'joystick0',
    title: 'Idenitifier',
    description: 'A unique name for the joystick to refer to it by in the API. Joysticks are also buttons so this will also be the name of button in the API. It will give a warning in browser tools if the name is not unique.'
});

TouchJoystick.attributes.add('type', { 
    type: 'string',
    default: 'fixed', 
    enum:[
        {'Fixed in place': 'fixed'},
        {'Move to first touch and fixed': 'relative'},
        {'Move to first touch and drags': 'drag'}
    ],
    title: 'Type',
    description: 'Set type of behavior for the joystick.'
});

TouchJoystick.attributes.add('baseEntity', { 
    type: 'entity',
    title: 'Base Entity',
    description: 'Image Element Entity that shows the base of the joystick.'
});

TouchJoystick.attributes.add('nubEntity', { 
    type: 'entity',
    title: 'Nub Entity',
    description: 'Image Element Entity that shows the nub (top) of the joystick.'
});

TouchJoystick.attributes.add('axisDeadZone', { 
    type: 'number', 
    default: 10,
    title: 'Axis Dead Zone',
    description: 'The number of UI units from the position of the Base Entity where input is not registered.' 
});

TouchJoystick.attributes.add('axisRange', { 
    type: 'number', 
    default: 50,
    title: 'Axis Range',
    description: 'The number of UI units from the position of the Base Entity that the Nub Entity can move to and is the maximum range'
});

TouchJoystick.attributes.add('hideOnRelease', { 
    type: 'boolean', 
    default: false,
    title: 'Hide on Release',
    description: 'Will only show the joystick when the user is using it and will hide it on touch end. This is commonly used if you don\'t want the joystick to block what\'s being shown on screen.'
});

TouchJoystick.attributes.add('positionOnRelease', { 
    type: 'string', 
    default: 'stay',
    enum:[
        {'Stay': 'stay'},
        {'Original': 'original'},
        {'Last start': 'lastStart'}
    ],
    title: 'Position on Release',
    description: 'Where to move the joystick on release and can help keep the screen tidy so that there are clear areas to show the game and arrange controls.'
});

TouchJoystick.attributes.add('vibrationPress', { 
    type: 'number', 
    default: 0,
    title: 'Vibration duration (ms)',
    description: 'If the device supports vibration with \'Navigator.vibrate\', it will vibrate for the duration set here on touch down.Set to 0 to disable.'});

TouchJoystick.attributes.add('debugText', { 
type: 'entity', 
default: 0,
});

// initialize code called once per entity
TouchJoystick.prototype.initialize = function() {
    if (window.touchJoypad && window.touchJoypad.sticks[this.identifier] !== undefined) {
        console.warn('Touch joystick identifier already used, please use another for Entity: ' + this.entity.name);
        return;
    }

    this._originalLocalPosition = this.baseEntity.getLocalPosition().clone();
    this._lastPointerDownPosition  = new pc.Vec3();

    this._setAxisValues(0, 0);
    this._inputDown = false;
    this._pointerId = -1;

    //this._canVibrate = !!navigator.vibrate;

    this._setButtonState(false);

    this.on('state', (state) => {
        this._setEvents(state ? 'on' : 'off');
    });

    this.on('destroy', () => {
        if (window.touchJoypad) {
            window.touchJoypad.sticks[this.identifier] = undefined;
        }
    });

    this._setEvents('on');
};

TouchJoystick.prototype._setEvents = function (offOn) {
    this._setAxisValues(0, 0);
    this._pointerDown = false;
    this._pointerId = -1;

    this.baseEntity.enabled = !this.hideOnRelease;

    this.entity.element[offOn]('mousedown', this._onMouseDown, this);
    this.entity.element[offOn]('mousemove', this._onMouseMove, this);
    this.entity.element[offOn]('mouseup', this._onMouseUp, this);

    if (this.app.touch) {
        this.entity.element[offOn]('touchstart', this._onTouchDown, this);
        this.entity.element[offOn]('touchmove', this._onTouchMove, this);
        this.entity.element[offOn]('touchend', this._onTouchUp, this);
        this.entity.element[offOn]('touchcancel', this._onTouchUp, this);
    }
};

TouchJoystick.__uiPos = new pc.Vec2();
TouchJoystick.prototype.screenToUi = function (screenPosition) {
    /** @type {pc.Vec2} */
    const uiPos = TouchJoystick.__uiPos;

    // Convert to a normalised value of -1 to 1 on both axis
    const canvasWidth = this.app.graphicsDevice.canvas.clientWidth;
    const canvasHeight = this.app.graphicsDevice.canvas.clientHeight;  

    uiPos.x = screenPosition.x / canvasWidth;
    uiPos.y = screenPosition.y / canvasHeight;

    uiPos.mulScalar(2).subScalar(1);
    uiPos.y *= -1;

    return uiPos;
};

TouchJoystick.prototype._onMouseDown = function (e) {
    // Give mouse events an id
    e.id = 0;
    this._onPointerDown(e);
    if (this._pointerDown) {
        e.stopPropagation();
    }
};

TouchJoystick.prototype._onMouseMove = function (e) {
    e.id = 0;
    this._onPointerMove(e);
    if (this._pointerDown) {
        e.stopPropagation();
    }
};

TouchJoystick.prototype._onMouseUp = function (e) {
    e.id = 0;
    if (this._pointerDown) {
        e.stopPropagation();
    }
    
    this._onPointerUp(e);
};


TouchJoystick.prototype._onTouchDown = function (e) {
    if (this._pointerDown) {
        return;
    }

    const wasPointerDown = this._pointerDown;
    e.id = e.touch.identifier;
    this._onPointerDown(e);

    if (!wasPointerDown && this._pointerDown && e.id === 0) {
        e.stopPropagation();
    }
};

TouchJoystick.prototype._onTouchMove = function (e) {
    e.id = e.touch.identifier;
    this._onPointerMove(e);
    this.debugText.element.text = e.id; 

    if (this._pointerDown && e.id === 0) {
        e.stopPropagation();
    }
    e.event.preventDefault();
};

TouchJoystick.prototype._onTouchUp = function (e) {
    if (this._pointerDown) {
        e.id = e.touch.identifier;
        this._onPointerUp(e);
        e.stopPropagation();
    }

    e.event.preventDefault();
};



TouchJoystick.prototype._onPointerDown = function (pointer) {
    const uiPos = this.screenToUi(pointer);
    switch (this.type) {
        case 'drag':
        case 'relative': {
            this.baseEntity.setPosition(uiPos.x, uiPos.y, 0);
            this.nubEntity.setLocalPosition(0, 0, 0);
            this._pointerDown = true;
        } break;
        case 'fixed': {
            this.nubEntity.setPosition(uiPos.x, uiPos.y, 0);
            this._updateAxisValuesFromNub();
            this._pointerDown = true;
        } break;
    }

    if (this._pointerDown) {
        //if (this._canVibrate && this.vibrationPress !== 0) {
        //    navigator.vibrate(this.vibrationPress);
        //}
        
        // If it's a mouse event, we don't have an id so lets make one up
        this._pointerId = pointer.id ? pointer.id : 0;
        this._setButtonState(true);
        this._lastPointerDownPosition.copy(this.baseEntity.getLocalPosition());
        this.baseEntity.enabled = true;

        // Set the values for the joystick immediately
        this._onPointerMove(pointer);
    }
};

TouchJoystick.__tempNubPos = new pc.Vec3();
TouchJoystick.__tempBasePos = new pc.Vec3();

TouchJoystick.prototype._onPointerMove = function (pointer) {
    if (this._pointerDown && this._pointerId == pointer.id) {
        const uiPos = this.screenToUi(pointer);
        const axisRangeSq = this.axisRange * this.axisRange;
        this.nubEntity.setPosition(uiPos.x, uiPos.y, 0);

        /** @type {pc.Vec3} */
        const nubPos = TouchJoystick.__tempNubPos;
        nubPos.copy(this.nubEntity.getLocalPosition());

        const nubLengthSq = nubPos.lengthSq();

        if (nubLengthSq >= axisRangeSq) {
            if (this.type === 'drag') {
                // Work out how much we need to move the base entity by so that
                // it looks like it is being dragged along with the nub
                const distanceDiff = nubPos.length() - this.axisRange;
                const basePos = TouchJoystick.__tempBasePos;
                basePos.copy(nubPos);
                basePos.normalize().mulScalar(distanceDiff);
                basePos.add(this.baseEntity.getLocalPosition());
                this.baseEntity.setLocalPosition(basePos);
            }

            nubPos.normalize().mulScalar(this.axisRange);
            this.nubEntity.setLocalPosition(nubPos);
        } 
        
        this._updateAxisValuesFromNub();
    }
};

TouchJoystick.prototype._onPointerUp = function (pointer) {
    if (this._pointerDown && this._pointerId == pointer.id) {
        this.nubEntity.setLocalPosition(0, 0, 0);
        if (this.hideOnRelease) {
            this.baseEntity.enabled = false;
        }

        switch(this.positionOnRelease) {
            case 'original': {
                this.baseEntity.setLocalPosition(this._originalLocalPosition);
            } break;
            case 'lastStart': {
                this.baseEntity.setLocalPosition(this._lastPointerDownPosition);
            } break;
        }

        this._pointerId = -1;
        this._updateAxisValuesFromNub();
        this._setButtonState(false);
        this._pointerDown = false;
    }
};

TouchJoystick.prototype._updateAxisValuesFromNub = function() {
    const axisRange = this.axisRange - this.axisDeadZone;

    const nubPos = this.nubEntity.getLocalPosition();
    const signX = Math.sign(nubPos.x);
    const signY = Math.sign(nubPos.y);

    const axisX = pc.math.clamp(Math.abs(nubPos.x) - this.axisDeadZone, 0, axisRange) * signX;
    const axisY = pc.math.clamp(Math.abs(nubPos.y) - this.axisDeadZone, 0, axisRange) * signY;

    this._setAxisValues(axisX/axisRange, axisY/axisRange);
};

TouchJoystick.prototype._setAxisValues = function (x, y) {
    if (window.touchJoypad) {
        window.touchJoypad.sticks[this.identifier] = { x: x, y: y };
    }

    this.axisX = x;
    this.axisY = y;
};

TouchJoystick.prototype._setButtonState = function (state) {
    if (window.touchJoypad) {
        window.touchJoypad.buttonStates[this.identifier] = state ? Date.now() : null;
    }

    this._state = state;
};

the relevant areas are:

TouchJoystick.prototype._onTouchDown = function (e) {
    if (this._pointerDown) {
        return;
    }

    const wasPointerDown = this._pointerDown;
    e.id = e.touch.identifier;
    this._onPointerDown(e);

    if (!wasPointerDown && this._pointerDown && e.id === 0) {
        e.stopPropagation();
    }
};

TouchJoystick.prototype._onTouchMove = function (e) {
    e.id = e.touch.identifier;
    this._onPointerMove(e);
    this.debugText.element.text = e.id; 

    if (this._pointerDown && e.id === 0) {
        e.stopPropagation();
    }
    e.event.preventDefault();
};

TouchJoystick.prototype._onTouchUp = function (e) {
    if (this._pointerDown) {
        e.id = e.touch.identifier;
        this._onPointerUp(e);
        e.stopPropagation();
    }

    e.event.preventDefault();
};

If I comment out e.stopPropagation() for the touch inputs inside this script, using the joystick also causes the camera to orbit.

This is my camera controller (just the touch input section).

CameraController.prototype.initialize = function () {
    

    // set input for mobile (screen input) / standalone 
    if (_platform === "mobile") {
        // note: this section is added to make the complete screen recognize touch inputs for look functionality (check mobile input section below)
        var self = this;

        // Handle touch start
        app.touch.on(pc.EVENT_TOUCHSTART, function (e) {
            self.onTouchStart(e);
        }, this);

        // Handle touch move
        app.touch.on(pc.EVENT_TOUCHMOVE, function (e) {
            self.onTouchMove(e);
        }, this);

        // Handle touch end
        app.touch.on(pc.EVENT_TOUCHEND, function (e) {
            self.onTouchEnd(e);
        }, this);

        this.on('destroy', function () {
            app.touch.off(pc.EVENT_TOUCHSTART, this.onTouchStart, this);
            app.touch.off(pc.EVENT_TOUCHMOVE, this.onTouchMove, this);
            app.touch.off(pc.EVENT_TOUCHEND, this.onTouchEnd, this);
        }, this);

};



CameraController.prototype.onTouchStart = function (e) {
    for (let i = 0; i < e.touches.length; i++) {
        const touch = e.touches[i];

        const target = touch.target;
        if (target.tags === "ui_element") {
            continue;
        }

        // Start tracking touch
        if (!this.activeTouch) {
            this.activeTouch = true;
            this.lastTouchPosition = new pc.Vec2(touch.x, touch.y);
            break;
        }
    }
};


CameraController.prototype.onTouchMove = function (e) {
    if (!this.activeTouch) return;

    for (let i = 0; i < e.touches.length; i++) {
        const touch = e.touches[i];
        const currentTouchPosition = new pc.Vec2(touch.x, touch.y);

        
        const dx = currentTouchPosition.x - this.lastTouchPosition.x;
        const dy = currentTouchPosition.y - this.lastTouchPosition.y;

        
        this.eulers.x -= (this.inputSensivity* dx) / 20;
        this.eulers.y -= (this.inputSensivity* dy) / 20;

        
        this.eulers.y = pc.math.clamp(this.eulers.y, this.pitchAngleMin, this.pitchAngleMax);

        this.lastTouchPosition.copy(currentTouchPosition);
        break;
    }
};


CameraController.prototype.onTouchEnd = function (e) {
    this.activeTouch= false;
};

Both methods work fine when used “alone,” but when I use the virtual joystick first, the camera controller inputs aren’t recognized (due to e.stopPropagation()). However, if I comment out e.stopPropagation(), using the virtual joystick nub also causes the screen to orbit. I can’t figure out how to combine both so that I can use the movement joystick and orbit the camera via screen “movement/swiping” at the same time.

To clarify: using the virtual joystick shouldn’t affect the camera orbit, and if the joystick is in use, the camera rotation with the second touch input shouldn’t be blocked by e.stopPropagation(). If I understand correctly, e.stopPropagation() blocks all other events. Is there an option to create some kind of exception?

Note: I initially thought checking for the UI element’s tag could solve the issue, but it seems it doesn’t detect tags on the target object. Otherwise, I could exclude screen rotation when the touch input is recognized due to the joystick UI.

4

I think I understand.
Can you make a boolean variable that is for joystick controls, when an input from the joystick input is received, you can turn it to true, and when a mobile input is received just turn it to false.

If the variable is true, run the joystick controls, if it is false, run the input controls.

5

I need to check this out. It would be so easy if you could just set tags for UI elements and exclude those elements from touch input calculations, but somehow this doesn’t seem possible within a DOM structure—or at least, I couldn’t figure out how to set it up.

6

Ok, I found a solution that works for me:

I excluded the joystick UI element from the camera orbit script’s recognition when checking if the first touch is inside the bounds of the UI element.

It works pretty well, but I’m wondering if this is the best practice, or if there is a simpler way. As I mentioned, if we could just check the tags of the UI elements when interacting (I mean when the input mode is on), it would be much easier to exclude those elements from touch detection.

So, if anyone reads this:
Feel free to share your suggestions, ideas, or improvements. Maybe I missed a simpler solution, because this was quite time-consuming. :smile:

Cheers!

1 Like
7

Im glad you found an answer!

8

Hi @Codeknight999

Thank you very much for your help! :slight_smile:

1 Like