How can I access the pixel color on screen, if I know x and y of the pixel? I only want to keep my calculation based on what is shown on screen

I just want to read every pixel’s color for some further calculations, but I am unable to find any reliable way to do that.