Rendering gaussian splats

[soundform]

I'm looking for a high-quality renderer of gaussian splats that supports global illumination (GI). Each scene is made of 1-10 millions of relatively small gaussians that overlap heavily, and often the same spot in 3d belongs to ~100 gaussians. I wonder if that's something already supported by this project.

A raymarching / raytracing type renderer equipped with a decent octree should be able to handle this. The only catch is that gaussians don't have solid surfaces with normals, so the traditional Phong-type lighting models don't apply. The scene is essentially a dense fog, or a cloud. GI would need to take into account that gaussians may be illuminated on one side, but remain relatively dark on the other. In other words, light accumulation cannot just assign each gaussian a single 0..1 illumination level. GI would need to do 5-10 bounces of light to achieve a good render quality.

Below is an example of such scene:

[gkjohnson]

This project generally only supports traditional mesh representations of geometry. I'm familiar with Gaussian Splats but as far as I'm aware path tracing or approaches for lighting theses types of models is not super well defined and is still fairly cutting edge. Are there any other tools that support what you're suggesting?

[soundform]

No, I haven't found any raytracers that can do this. However I cannot agree that it's not well defined, because it's the same as raytracing a cloud defined by its density function (e.g. https://www.shadertoy.com/view/XslGRr). It's just in this case the density function is a sum of many gaussians and it needs some clever techniques to evaluate it efficiently. For example: https://research.nvidia.com/labs/toronto-ai/3DGRT. I'll probably resort to writing my own: a one pass raytracing + basic shadows shouldn't be that difficult. It's GI that's the problem, although there are tricks to make it somewhat simple: https://iquilezles.org/articles/simplegi.

[gkjohnson]

Rendering a GS as similar to a cloud volume may be fine for your use case but it won't be right for splats that are representing hard or reflective surfaces, unfortunately. That second nvidia paper doesn't look like it's doing anything all that robust for lighting beyond basic shadows but may be a good place to start for what you're looking to do.