[SOLVED] Render UI to texture

LeXXik · March 7, 2022, 7:31pm

Looking for an advice or a recommendation on how to approach rendering a long list of UI elements in a scroll view. The list could have a few items per row and could span a few screen heights (depending on device).

The issue is that each item from the scroll view is a multi-element item, which may contain shared sprites/images among all of items, as well as unique ones only belonging to that item. Consider a long list of menu dishes in a restaurant, where you could have decoration elements on each item that all of them share, while the dish image is unique to that item.

One option I consider is that the shared images/sprites could go all together in single atlas map, while the unique ones, not available on runtime would be requested and rendered when ready. This will tank the draw calls, as each dish would then add a draw call (or more if there are more unique elements).

To mitigate it perhaps an option is to use an offscreen camera that would render these items into a tall texture. Then that texture would be used as a background for the scrollview content and would only take a single draw call. However, I am not sure how to render a tall texture of screen elements, if they can go over the screen height. Can I change some properties of the offscreen camera so that it would be able to see the whole list? Or do I need to somehow break the stack into smaller ones and then stitch them?

@mvaligursky @slimbuck @yaustar if you have any ideas, feel free to chip in

Leonidas · March 7, 2022, 8:46pm

One idea that may work is to set the render camera to a custom aspect ratio and fill mode that matches the render target. And after the render is complete you set it back to the original settings.

That should work given you are able to calculate the right render target resolution for the UI elements (I haven’t tried it though and potentially if the render is slow it may glitch momentarily).

Something like this:

renderCamera.camera.aspectRatioMode = pc.ASPECT_MANUAL;
renderCamera.camera.aspectRatio = this.renderTexture.width / this.renderTexture.height;

this.app.setCanvasFillMode(pc.FILLMODE_NONE, this.renderTexture.width, this.renderTexture.height);

yaustar · March 7, 2022, 9:53pm

If the issue is drawcalls, can you disable/enable list item entities if they are in view or not? Or won’t that be enough?

mvaligursky · March 8, 2022, 8:48am

aspect ration is by default automatically calculated based on the render target dimensions … so that part should just work. Ideally you create a render target with texture, create a camera for rendering to it, and just render it it (by using a new layer on geometry as well as the camera. Something along this script:

github.com

playcanvas/engine/blob/dev/scripts/utils/cubemap-renderer.js

// How to use in the Editor:
// - create entity with Camera component - position of the entity defines where the cubemap is rendered from
//   and properties of the Camera are used to render cubemap (adjust near / far distance, clearing, layers and other properties)
//   Note: the layers should contain all layers visible by cubemap camera.
// - to use generated cube map, you can access it like this using script:
//   material.cubeMap = entity.script.cubemapRenderer.cubeMap;

var CubemapRenderer = pc.createScript('cubemapRenderer');

CubemapRenderer.attributes.add('resolution', {
    title: 'Resolution',
    description: 'Resolution of one side of a cubemap. Use power of 2 resolution if you wish to use Mipmaps.',
    type: 'number',
    default: 64
});

CubemapRenderer.attributes.add('mipmaps', {
    title: 'Mipmaps',
    description: 'If set to true, mipmaps will be allocated and autogenerated.',
    type: 'boolean',

This file has been truncated. show original

used here

github.com

playcanvas/engine/blob/dev/examples/src/examples/graphics/render-cubemap.tsx

import React from 'react';
import * as pc from '../../../../';

import { AssetLoader } from '../../app/helpers/loader';

class RenderCubemapExample {
    static CATEGORY = 'Graphics';
    static NAME = 'Render Cubemap';

    load() {
        return <>
            <AssetLoader name='helipad.dds' type='cubemap' url='/static/assets/cubemaps/helipad.dds' data={{ type: pc.TEXTURETYPE_RGBM }}/>
            <AssetLoader name='script' type='script' url='/static/scripts/utils/cubemap-renderer.js' />
        </>;
    }

    example(canvas: HTMLCanvasElement, assets: { 'helipad.dds': pc.Asset, script: pc.Asset }): void {

        // Create the app
        const app = new pc.Application(canvas, {});

This file has been truncated. show original

LeXXik · March 8, 2022, 10:54pm

Thank you, everyone for the inputs!

@yaustar yeah, in some cases there can be like 10-30 extra elements that are hard to batch, unfortunately.

@mvaligursky, right, so as @Leonidas mentioned I had to adjust the ratio. As you mentioned camera is fine, it grabs whatever texture resolution is set there, so it just works. The issue comes afterwards, when that texture is added to some existing UI element, skewing it, depending on the ratio differences. I had to adjust the element’s ratio, so there would be no distortion. I also had to flip on Y the UI element, as the render is upside down (and flipY flag on texture in this case doesn’t work for some reason).

For anyone interested, here is a result with an example of 20 UI images packing into a single draw call:
https://playcanvas.com/project/893690/overview/texture-composition

yaustar · March 8, 2022, 11:11pm

The flipY option is on the RenderTarget, not the texture

LeXXik · March 8, 2022, 11:16pm

Hmm, are you sure? I think it is on texture:
https://developer.playcanvas.com/api/pc.Texture.html#flipY

yaustar · March 9, 2022, 12:04am

Hmm… It’s also on the render target RenderTarget | PlayCanvas API Reference

LeXXik · March 9, 2022, 9:57am

Interesting… Anyhow, I just tried to use the flag on render target, but it doesn’t seem to change anything. My guess is that the texture is first flipped, then we render to it, which cancels any flipping? Not sure.

yaustar · March 9, 2022, 10:02am

Very odd May have to have a poke at it myself

yaustar · March 9, 2022, 12:45pm

Found the issue, the UI layer doesn’t render on a flipY render target.

I will report it as a bug

Edit: Bug report: https://github.com/playcanvas/engine/issues/4097

LeXXik · March 9, 2022, 4:58pm

Thank you! I remember I had that issue with not being able to render it. Thought I was doing something wrong.