When entering WebXR the rendering to the canvas gets paused.
Is there a simple way to mirror one eye to the canvas, so other persons can see what the user is doing in VR?
I’m also ok with adding a second camera and parenting it to the HMD entity and let it render to the canvas. This will introduce some performance costs, but I can live with that.
My Setup is SteamVR + HTC Vive and WebXR with Chrome
In theory this is possible, but not something that is supported by the engine at the moment - so you’d need to implement it yourself. Currently engine renders to a single backbuffer, that comes either from the canvas, or from the XR.
Just had a look at the docs and I guess using the colorbuffer of the cameras rendertarget and drawing that to the canvas should work.
Gonna try that next week.
this.app.graphicsDevice.renderTarget also returns null.
I’ve also tried the creating my own rendertarget but calling getSource() on the colorbuffer also returns null.
Same is true when I use the ‘uSceneColorMap’ of one of the graphicsDevice targets. Also returning null.
I currently don’t have a VR headset connected to my pc. So I’m just trying to mirror the camera on postrender to a secondary canvas.
This is what I have so far:
I’m getting the error bindFramebuffer: object does not belong to this context
So it seems like it’s not possible to blit from one context to another.
Also I’m not quite sure how to reset the framebuffers. In your example it’s reseting to the framebuffer of the renderTarget of the graphicsDevice. But the rendertarget always is null for me. At least in postrender callback. Edit: Binding null to reset the framebuffers should work. I’ve added it to the script above.
This gives the following error: Framebuffer is incomplete: No attachments and default size is zero.
How can I specify the size and what exactly do I have to do with attachments?
Sorry I’m new to all the low level gl calls
To be honest, I don’t any easy answer here … framebuffers are not straighforward to work on in WebGL, and I didn’t expect this to be easy. EIther way, I would not expect you to call gl.createFramebuffer as you should not need to create a new framebuffer I think, but I’m not sure what happens under the hood in WebGL with the default framebuffer when XR is active, I have not touched that area.
I assumed that I have to draw to some extra buffer, which I then can show on the canvas. As the backbuffer isn’t shown.
I’ll try around a bit more and post a solution if I find one.
var ScreenMirroring = pc.createScript('screenMirroring');
ScreenMirroring.prototype.initialize = function () {
this.app.on('postrender', this.postRender, this);
};
ScreenMirroring.prototype.postRender = function () {
if (this.app.xr.active) {
let gd = this.app.graphicsDevice;
gd.gl.bindFramebuffer(gd.gl.READ_FRAMEBUFFER, this.app.xr.session.renderState.baseLayer.framebuffer);
gd.gl.bindFramebuffer(gd.gl.DRAW_FRAMEBUFFER, null);
gd.gl.blitFramebuffer(0, 0, gd.width, gd.height, 0, 0, gd.width, gd.height, gd.gl.COLOR_BUFFER_BIT, gd.gl.NEAREST);
gd.gl.bindFramebuffer(gd.gl.FRAMEBUFFER, null);
}
};
I’m now trying to use this.app.xr.session.renderState.baseLayer.framebuffer as read buffer. However when usin this framebuffer, the XR Session seems to crash/not start silently.
I’d say that this XR context is on a different device (HTC Vive) and so blitting this to desktop won’t work.
It might work using some XR emulator in the browser, as that’d run on the same device.
var ScreenMirroring = pc.createScript('screenMirroring');
ScreenMirroring.prototype.initialize = function () {
this.app.on('postrender', this.postRender, this);
this.app.xr.on('start', function () {
this.isActive = true;
}, this);
this.app.xr.on('end', function () {
this.isActive = false;
}, this);
this.isActive = false;
};
ScreenMirroring.prototype.postRender = function () {
if (this.isActive) {
let gd = this.app.graphicsDevice;
gd.gl.bindFramebuffer(gd.gl.READ_FRAMEBUFFER, this.app.xr.session.renderState.baseLayer.framebuffer);
gd.gl.bindFramebuffer(gd.gl.DRAW_FRAMEBUFFER, null);
gd.gl.blitFramebuffer(0, 0, gd.width, gd.height, 0, 0, gd.width, gd.height, gd.gl.COLOR_BUFFER_BIT, gd.gl.NEAREST);
gd.gl.bindFramebuffer(gd.gl.FRAMEBUFFER, null);
}
};
When using the start and end events off the XRManager it does correctly blit to the canvas! My guess is when using XRManager.active the framebuffer isn’t properly set up yet.
However this introduces another problem. I can now see the rendered image on the canvas but not in the headset anymore.
Also following warnings are printed: [.WebGL-0000625000A3FF00] GL_INVALID_OPERATION: Invalid operation on multisampled framebuffer Note: The XRSession has completed multiple animation frames without drawing anything to the baseLayer's framebuffer, resulting in no visible output.
Disabling anti aliasing gets rid of the firrst warning.
Binding the framebuffer back to the buffer of the xrsession baselayer, restores rendering to the headset!
Here is a working example which displays both eyes:
var ScreenMirroring = pc.createScript('screenMirroring');
ScreenMirroring.prototype.initialize = function () {
this.app.on('postrender', this.postRender, this);
this.app.xr.on('start', function () {
this.isActive = true;
}, this);
this.app.xr.on('end', function () {
this.isActive = false;
}, this);
this.isActive = false;
};
ScreenMirroring.prototype.postRender = function () {
if (this.isActive) {
let gd = this.app.graphicsDevice;
let webglLayer = this.app.xr.session.renderState.baseLayer;
gd.gl.bindFramebuffer(gd.gl.READ_FRAMEBUFFER, webglLayer.framebuffer);
gd.gl.bindFramebuffer(gd.gl.DRAW_FRAMEBUFFER, null);
gd.gl.blitFramebuffer(0, 0, gd.width, gd.height, 0, 0, gd.width, gd.height, gd.gl.COLOR_BUFFER_BIT, gd.gl.NEAREST);
gd.gl.bindFramebuffer(gd.gl.FRAMEBUFFER, webglLayer.framebuffer);
}
};
Is there any way to get this working with antialiasing enabled?
In this stackoverflow link the problem seems to be the different resolutions of the HMD and the canvas.
Which somehow prevents a successful blit when antialiasing is enabled?
If aa is really undoable I probably try to get some kind of aa with a post processing shader,