Gaussian Splatting + PlayCanvas

In regards to the firefox issue. I believe it can be resolved by checking oldData.byteLength != 0 before doing the postMessage and the setData.
Firefox does not yet implement ArrayBuffer.detached to indicate when the buffer has been transferred, but my breakpoint debug shows the length is zero:

Oh that’s interesting. I wonder why firefox has a detached buffer there…

It seems to be called repeatedly (I stepped it 5 times through before I just let it run to the error). I haven’t hunted back further through that code.

Ah yes, I forked it this morning but I’ve been using the ā€˜clean’ read-only version since I found the first error. Great stuff, I think that’s enough to get me moving again on Monday. Cheers!

1 Like

We’ll be also promoting the new engine (1.67.1) to the Editor early next week (probably Monday) and so the splats will work with the default engine too.

1 Like

This has been released, see the test project here: PlayCanvas 3D HTML5 Game Engine

2 Likes

Nice, you fixed the FF problem with arraybuffer access too!
EDIT: I spoke too soon, it’s fine if you run from the ā€˜play’ button but if you ā€˜launch’ from editor it still does this.

There seems to a be a problem with the depth sorting for the splats.
I haven’t dug into it yet, but I’ll take a look this afternoon.
Here’s a video of the issue (I added a lego r2d2 splat to the first person demo level):

File is 17MB which exceeds the 8MB cap here. I’ll see if I can cut the video down some more…

It works great with the depth sorting vs 3D geometry though! Perfect occlusion coming through the doorway.

EDIT: confirmed, it does the same with the guitar example. So it’s not just my ply file

Project is public:
https://playcanvas.com/project/1171413/overview/firstpersonsplat

I didn’t get as far into the code as I’d hoped to, but I wonder if the problem might be the camera angle to the splat? It works fine in the simple demo where you see it from a fixed camera and it’s rotating, so that seems to rule out a problem with the gaussian sort order… unless it’s assuming a fixed camera somewhere ā€˜in front’ of the object. When the camera is moving like the first person demo, any global assumptions about ā€˜forwards’ will fail and I’d expect to see something like this (an inside out R2D2, poor thing!)

the gs script you have there has a camera property … that is the camera that is used for the splat sorting. You need to attach the camera on the character, as that’s the one that is moving around, not the other one.

1 Like

OMG I can’t believe I didn’t spot that, thank you.
No idea why a first person controller has two cameras… but yes, that definitely explains it :smiley:

1 Like

Hello! Using the (amazing) rough port of antimatter15 for displaying splats I notice that there is a jitterintg vertical line in the background:

It seems persistant no matter what splat I load.

Anybody have a clue what could be causing this and how to fix it?

Thanks!

Is this your implementation of splat rendering?
Do you get that issue if you drop your ply file to our viewer? PlayCanvas glTF Viewer

No, I just forked this and changed the logo. Is there a project around that I can fork that uses the official implementation of 3DGS?

I tried using this but when I upload my own ply (cleaned with supersplat), I cannot drag it onto the the field where the ply is loaded… I am noob =\

Thanks!!

This one mentioned above:
https://playcanvas.com/project/1165904/overview/gaussiansplatdemo

Note that we’ll be releasing a new engine which will change this a bit in a week or so … as this project is a pre-release.

Thanks, and I tried that project. But I cannot figure out why I can’t load my own ply file. It displays fine in supersplat and the splatviewer. I upload my ply file into the assets, but as opposed to the guitar file, I cannot drag and drop my ply file into the GS entity modelAsset field.

I just checked this and it looks like we’ve accidentally released half a PLY import pipeline. The file imports as ā€œGsplatā€ asset instead of binary asset. We’ll get this fixed asap.

I just realised that I noticed that too, forgot to mention it. Looking forward to the fix!

Thanks again!

Exporting .splat works well, but it seems that loading them as gsModel is currently not possible. Is there any plan to support the loading of raw .splat files?

Advantages:

  • .splat files can be streamed in and are sometimes much smaller.
  • Public Gaussian splat repositories usually store .splat files (e.g., polycam, hugginface, etc.) that can be easily referenced.

Thanks!

Hi @TRT,

Sorry for the late response, I missed your message somehow.

I haven’t seen .splat files being made available from public repositories. Can you provide any links to these? I’ve only seen models offered in uncompressed PLY format.

We support loading uncompressed PLY and a compressed variant. The compressed variant does a better job of compressing the data than .splat. You can read more in the blog post.

To give you an idea of sizes, for a guitar.ply scene I created:
Original Uncompressed .ply: 132.8MB
Compressed .ply: 1.5MB
.splat: : 2.9MB

Thanks!

1 Like