UI Mask Pixelated

Sadly I think you can’t do anything about that with the current masking implementation. Masking elements will use the alpha of the texture provided but only using 1-bit values (masked or not), fading isn’t supported.

Masks and Masking
Image Elements can be used to mask or hide elements that are descendents. To enable this feature set the mask property of an image Element to true.

If there is no texture asset assigned to an image Element used for masking the mask will be a rectangle defined by the width and height. If the image Element has a texture assigned the alpha-channel of the texture is used as the mask. Note, an image mask is 1-bit, i.e. there is no fading out of the mask using the alpha-channel of the texture.

https://developer.playcanvas.com/en/user-manual/user-interface/image-elements/

You could disable masking and try using multiple overlapping elements in place with both textures having alpha. Much like the UI progress bar example does:

https://developer.playcanvas.com/en/tutorials/ui-elements-progress/