After an email exchange with the lead author, I just rendered one of their demo datasets using my Datoviz GPU rendering library [1]. It looks nice and it's quite fast. I'm just rendering uniform triangles using standard 3D rasterization.
https://github.com/user-attachments/assets/6008d5ee-c539-451...
(or https://github.com/datoviz/data/blob/main/gallery/showcase/s...)
I'll add it to the official gallery soon, with proper credits.
This looks really nice, but I cant help to think this is a stop gap solution like other splatting techniques. It's certainly better than NERFs, where the whole scene is contained in a black box, but reality is not made up of a triangle soup or gaussian blobs. Most of the real world is made up of volumes, but can often be thought of as surfaces. It makes sense to represent the ground, a table, walls, etc with planes, not a cloud of semi translucent triangles. This is like pouring LEGO on the floor and moving them around until you get something that looks ok from a distance, instead of putting them together. Obviously looking good is often all that's needed, but it doesn't feel very elegant.
Although, the normals look pretty good in their example images, maybe you can get good geometry from this using some post processing? But then is a triangle soup really the best way of doing that? My impression is that this is chosen specifically to get a final representation that is efficient to render on GPUs. I haven't done any graphics programming in years, but I thought you'd want to keep the number of draw calls down, do you need to cluster these triangles into fewer draw calls?
Is there any work being done to optimize a volumetric representation of scenes and from that create a set of surfaces with realistic looking shaders or similar? I know one of the big benefits of these splatting techniques is that it captures reflections, opacity, anisotropicity, etc, so "old school" photogrammetry with marching cubes and textured meshes have a hard time competing with the visual quality.
This seems like the natural next step after Gaussian splatting. After all, triangles are pretty much the most "native" rendering that GPUs can do. And as long as you figure out a way to make it differentiable (e.g. with their windowing function), it should be possible to just throw your triangles into a big optimizer.
Can anyone detail the use case for gaussian splatting to me? What are we trying to solve, or, where direction are we trying to head towards?
I'm more familiar with traditional 3D graphics, so this new wave of papers around gaussian splatting lies outside my wheelhouse.
Autoencoders should output these kinds of splats instead of pixel outputs and likely obtain better representations of the world at the bottleneck. These features can be used for downstream tasks.
Very exciting! Neural rendering finally coming to standard rasterizer engines.
Can someone explain what a splat is? I did graphics programming 25 years ago, but haven't touched it since. I don't think I've ever heard this word before.
I wonder how hard it would be to implement an extra processing step, to turn this to a more 'stylized low poly' look. Basically, the triangle count would be drastically smaller, but the topology would have to be crisp.
What is we use triangular pyramids instead of triangles?
Wouldn't this lead to the full 3D representation?
> In this paper, we argue for a triangle come back
Go team triangles!
2400+ fps (mindblown)
trianglebros we're so back
Make triangles great again
I'm so confused about the citation:
author = {Held, Jan and Vandeghen, Renaud and Deliege, Adrien and Hamdi, Abdullah and Cioppa, Anthony and Giancola, Silvio and Vedaldi, Andrea and Ghanem, Bernard and Tagliasacchi, Andrea and Van Droogenbroeck, Marc},
while on arxiv and the top of the page
Jan Held, Renaud Vandeghen, Adrien Deliege, Abdullah Hamdi, Silvio Giancola, Anthony Cioppa, Andrea Vedaldi, Bernard Ghanem, Andrea Tagliasacchi, Marc Van Droogenbroeck
> The triangles are well aligned with the underlying geometry. All triangles share a consistent orientation and lie flat on the surface.
When I first read "triangle splatting," I assumed Gaussians were just replaced by triangles, but triangles being aligned with geometry changes everything. Looking forward to seeing this in action in traditional rendering pipelines.