Gaussian Splatting + PlayCanvas

Hi

I made a rough port of
https://github.com/antimatter15/splat

It supports .PLY and .splat streaming.

In order to correctly render with the scene, I reversed the splat rendering and changed the blending. The results look identical but correctly merge with the scene geometry:

Project:
https://playcanvas.com/project/1150775/overview/gaussian-splatting

Known issues:
I query the gl device directly so it does change the blend/vertex/vbo attributes, so stuff after the splat may create artifacts. So next step would be to correctly using Playcanvas vertex buffers and shaders.

PS: You can just add any splat or ply Url to view scene from from the original paper, like
https://playcanv.as/p/22UuSvsW?url=https://huggingface.co/cakewalk/splat-data/resolve/main/train.splat

6 Likes

Thank you for sharing! @slimbuck is also working on splats recently.

Yes. Check out the latest:

1 Like

@will looks like yours is based on the same shaders and sorting idea :slight_smile: and it runs very smooth!

I have some blue splats in the your implementation:

I had this once when I was not copying all vbo data after sorting, not sure if the same case or just my GPU/OS/Browser.

I guess you sort by distance and re-arrange the VBO by shuffling all vertex data around. I recently started to experiment with texelFetch in the vertex shader, suppliying all data as texture, just sort the particle indices (@LeXXik yes indices again :slight_smile: ) and fetch vertex attributes, which means much less memory copying from the worker and less cpu/gpu transfer.

The speedup is not great for large scenes, maybe its now vertex shader bound, I have to do some benchmarks.

Have you tried something in this regard?

1 Like

Yep, we store colors in the texture, and the rest of the data fill follow shortly.

@TRT Rather than work on separate implementations, it’d be cool to join forces! If you feel that you would like to contribute, check out the Model Viewer codebase:

Things are developing very rapidly, but it’d be great to build on the current momentum around this. :rocket:

BTW, I noticed the blue splats on iOS too yesterday. Doesn’t happen on my PC. Not entirely sure why that happens. Your suspicions could be correct…

Looks like you guys are already making great progress with indexed sorting. I need some time to dig into the viewer source.

1 Like

Yes, we’re moving quite quickly. But if you spot any way to contribute, don’t hesitate to get involved. :smile:

Hi @will
How did you remove the points around your captured PLY file, just remaining the main element itself?

We have a tool that you can use to delete unwanted splats. We’ll hopefully release a first version of it in the 24 hours or so. :smile:

Great news!!!
Thanks a lot!!!
:clap: :clap: :clap:

Will this be supported in editor? I’d love to do walkthroughs with audio…

Our gaussian splat tool has been published to Supersplat

2 Likes

Great work!
I saw a review of it last week and it is an amazing tool!

1 Like

I am looking for some example / documentation on how render a ply (GS model) in my own viewer (playcanvas npm module). I can load my model into playcanvas using the load= query parameter. But I want to know how to do it in javascript on my own app.

I have the simple example here running in my application, GitHub - playcanvas/engine: Fast and lightweight JavaScript game engine built on WebGL and glTF. But I want to extend this to load a ply by url and render in the viewer.

I have been trying to reverse engineer the model viewer code, but its getting a bit too complicated. So, before I implement the whole entire model viewer in my app I am looking for some help with exactly what methods I need without extra stuff like the sidebar and icons.

We are very busy adding splat support directly to the engine. We initially added support to model-viewer so we could iterate quickly, but the next minor engine release will have the support instead.

4 Likes

Here’s a little update on where we are at with 3D Gaussian Splatting:

https://twitter.com/willeastcott/status/1724029460263940320

3DGS support is now merged into the engine codebase:

Exciting times! :smile:

4 Likes

Hi,

I’m new here. came in searching for a way to present my splatts via the web.
Any news or resources I should check out?
I tried making a PlayCanvas modelviewer project but my .PLY was too big.

Thanks for any help.

Lolo

@slimbuck

Hi @Lodewijk_Luijt , what do you mean your PLY was too big?