Screen Coverage

Grimmy · May 12, 2023, 2:21pm

Hi, does anyone know how I would go about calculating a 3d model’s screen coverage/percentage of screen coverage based on its bounding box or the amount of pixels it is using on screen?

I had Chat GPT have a go, but although it speaks with confidence it failed miserably even after about 20 tweaks to my responses.

Thanks

Leonidas · May 12, 2023, 2:27pm

Hi @Grimmy,

You could use the camera worldToScreen method on the corners of the total bounding box of your model.

That would be 8 points in 2D coordinates, from these points you could calculate either another bounding box or to be more precise a polygon. And get its pixel volume (=screen coverage). ChatGPT may be of help here to give you an algorithm for this last bit.

Another way would be to use a camera to get a screenshot (on a render target) just of this model (add it to a layer to control rendering). From there read the pixels of the texture buffer and count how many pixels are colored from model.

From there you can get the screen coverage if you divide that number from the total pixels of your screen. This method would be quite precise but slightly slower given reading the pixels from the texture can introduce a performance hit.

mvaligursky · May 12, 2023, 2:39pm

There’s a private function which does something similar

github.com

playcanvas/engine/blob/95d9e538f5806217c2be70d1f2812d6a3d144fdd/src/scene/camera.js#L545-L575


      
          // returns estimated size of the sphere on the screen in range of [0..1]
          // 0 - infinitely small, 1 - full screen or larger
          getScreenSize(sphere) {
          
          
    if (this._projection === PROJECTION_PERSPECTIVE) {
          
          
        // camera to sphere distance
                  const distance = this._node.getPosition().distance(sphere.center);
          
          
        // if we're inside the sphere
                  if (distance < sphere.radius) {
                      return 1;
                  }
          
          
        // The view-angle of the bounding sphere rendered on screen
                  const viewAngle = Math.asin(sphere.radius / distance);
          
          
        // This assumes the near clipping plane is at a distance of 1
                  const sphereViewHeight = Math.tan(viewAngle);

This file has been truncated. show original

Grimmy · May 13, 2023, 2:14pm

Any pointer on how I would go about this? Is it possible to do the calculation (render to texture) every frame without the game dying?
I figured I would have a different solid colour for each model and then count the number of those coloured pixels for each model. Then from there I can get a screen coverage for each model by dividing by the full screen area.

Leonidas · May 13, 2023, 3:21pm

Hi @Grimmy,

If you’re looking to do this per frame I’d not recommend that way.

What @mvaligursky shared above would be the most performant option in this case.