Postshot to SuperSplat

Just double checking a documentary dataset, I don’t immediately see these differences as apparent:

Postshot:


Supersplat:

Opacity histogram:

I’ve been trying a newly implemented Radiance field profile called Splat3 in postshot, compared with MDMC which I used previously and there is quite some difference actually.

MDMC, same as in previous posts:

Splat3:

Opaque surfaces look much better, the distribution of gaussians compared on their opacity value is more even - however it struggles a lot with the gradient appearance of the ground shadow.

1 Like

@bjorn.syse - to try the float framebuffer, you can easily do this in the Editor. Add you splat to the scene, add this script on your camera, and configure its format to be RGBA16 or 32.

I can’t help but feel there might be a bug in the camera array tool which results in these wildly semi-transparent models to be produced.

Looking at the docs (Camera Array Tool for Blender), the plugin outputs the camera poses, colored sparse 3d points and the rendered frames. The resulting exported data should be mathematically perfect, the ideal training set. The trainer (brush, polycam et al) needs no colmap stage at all.

Are you using all the exported data for training or do you use just the rendered frames?

If you’re using everything, could you try using just the rendered frames? (and visa-versa, if you’re using just the rendered frames could you try training with all the sfm data?).

Thanks!

Also, we should check if all the instances of this issue we’ve seen were generated using the camera array tool.

Hi

I was using the full exported camera positions and sparse point cloud. Not just the rendered frames. But I could try that and see if there is any difference.

However, we did se some difference on the output depending on what algorithm was used in Postshot as i mentioned in the last post I did.

1 Like

Which version of PostShot has Splat3 please?

That would be any of the latest prerelease beta versions

I tried the same dataset in the latest version of Postshot splat3 and it seems to have solved some issues with the shadow.



However, distribution of opacity heavily weighted towards transparent now again. Still some “Scratchy” surfacing.

I have not yet tried this with RGBA16 or higher.

Trying with RGB16 here but I’m not convinced, It looks pretty far from how it looks in postshot I must say. More like a pastel drawing.

RGBA16

Regular:

I also tried training with only the rendered frames - but postshot could not solve those cameras (200 of them), so no model was created in that test. not sure if that tells us something here or not.

Hi @bjorn.syse,

What is RGBA16, the backbuffer in SuperSplat?

Any chance you can send me the PLY?

For the tricky scenes I’ve found RGBA32F works better than 16F.

Thanks!

I will confirm with the team today, but will likely add an option to SuperSplat today for rendering Float32.

I was referring to the renderFormat I set in the CameraFrame script on the camera:

btw, my playcanvas interface is slow like walking in water working with this model. It’s ply is 78MB, or 19MB if I compress it in supersplat. IT has 323K splats. Would you say this represents a particularly high resolution guassian splat? If I launch the project, it’s also very slow. I feel it’s somewhat faster in Supersplat however.

Long time no see!
Just to let everyone know, the combination of PostShot 0.6 (using Splat3) and SuperSplat 2.5 fixed the problem. It is now a 1:1 between both apps. Let the fun begin!

3 Likes

Oh awesome - thanks for letting us know! :raised_hands:

1 Like

Hi! I also have rendering differences between Postshot 0.6 & SuperSplat, but this comes from the PLY export, here is a rendering of the generation, and a rendering of the reimported PLY export! I reported the bug to Jascha, he managed to reproduce it, he is on it! Thanks! :pray:

@slimbuck

Hi @SA3D ,

Did you train this scene with antialiasing enabled by any chance? If so, then I would guess postshot is not rendering with antialiasing upon re-import.

I actually opened an issue on the upstream Inria repo asking to write a PLY comment into the header of scenes trained with antialiasing for this very reason. The issue I created is here. I might open a similar issue on postshot requests.

Thanks for letting us know about this reimport issue! (And cool car btw :slight_smile: )

1 Like

Yes! That’s right!!! @slimbuck I just tested without AA, and the reimport is much better, and the import into SuperSplat too! Bravo! I’ll pass the message on to Jascha! Afterwards, it would indeed be great to be able to add a little precision in the transparencies on SuperSplat, but it’s already much better without AA! And thanks @mvaligursky for the PlayCanvas script, can we have a little detail on how it works? Best. :pray:

1 Like

You’re welcome.
But what script are you talking about?