What Exactly Is Gaussian Splatting?

I’ve just discovered Gaussian Splatting, and I genuinely fear it might become my entire personality. At its core, this rendering technique delivers stunning photo-real graphics at a fraction of the computational cost required by traditional game development pipelines. This massive efficiency leap is exactly why I decided to dive deep into the subject with an expert who has lived and breathed photogrammetry for years.

First, let’s get the pronunciation right. It is pronounced GOW-see-an, not GAWE-shun, because the technique is derived from the work of German mathematician Johann Carl Friedrich Gauss. Don’t worry, I will keep the heavy mathematics to a bare minimum. My obsession actually started about a week ago when I stumbled upon a photorealistic browser-based FPS running seamlessly thanks to this exact method.

The game logic for that straightforward shooter was built by Iakov Sumygin, a Software Engineer at Snap Inc. Sumygin kindly connected me with the artist behind the game’s breathtaking environment, Christoph Schindelar. Schindelar is a self-described "Scan-Specialist" who previously worked at Quixel, the Epic-owned studio that boasts the world’s largest photogrammetry asset library.

The Science Behind the Splats

To understand how this works, Schindelar compares the method to an "extremely advanced sprite-based particle system," noting that early Unreal Engine implementations actually rendered these splats through Niagara, the engine’s internal particle tool. In this context, a Gaussian does not behave like the standard polygons you would find on a 3D tree model. Instead, imagine individual splats as dandelion tufts caught in the breeze, opaque at the center and fuzzier toward the edges. Just as a single seed does little on its own, a soft, cohesive shape emerges when thousands are layered together.

If you revisit that browser-based shooter, you will notice an almost pointillist or impressionistic fuzz in certain areas, hinting at the 3D Gaussians combining to form high-resolution, real-time assets. But how do they actually function under the hood? Schindelar explains, "Each splat is a 3D Gaussian that projects to an elliptical footprint on screen. It can be stretched differently in different directions, has opacity, and can change its color depending on the viewing angle using spherical harmonics."

The real resource-saving magic happens during the rendering phase. Because the process mostly involves projecting and blending these splats directly on the GPU, it runs at an exceptionally fast frame rate without frying your hardware.

Streamlining the Production Pipeline

Beyond raw performance, this technique is fundamentally reshaping how developers handle scanned assets. Historically, turning a scan into a game-ready asset required a grueling sequence of manual and semi-automatic tasks. Schindelar lists the traditional bottlenecks:

  • Optimization and mesh cleanup
  • Retopology and UV mapping
  • Texture projection and baking
  • Complex material setup

Schindelar has spent years navigating these workflows, creating everything from game-ready scanned assets to scan-based displacement textures. To illustrate the difference, he shared two models of a child’s hiking boot: a traditional RAW version and a splat-based version. The RAW model relied on laser-scanning for clean geometry, photogrammetry for sharp textures, photometric stereo for detail normals, and a polarization-based scan to eliminate glare, with the laces added manually later. While you could not rotate the RAW boot in a browser window, the splat version renders it instantly and interactively.

While optimized traditional mesh-and-texture workflows can still be incredibly fast, a pipeline centered around this method completely bypasses the heavy lifting. "In a Gaussian Splatting pipeline, many of these steps can be almost completely bypassed," Schindelar notes. "Instead of rebuilding the captured reality into clean geometry and textures, the training process directly turns the captured image data into a real-time representation."

This efficiency is further amplified by open-source tools like PlayCanvas, which Sumygin used to power the browser FPS. The engine recently integrated SplatTransform 2.0, a utility that delivers "fully automated, lightning-fast generation of high-quality collision for your splats." Without it, players would simply phase through walls, making this collision tool a vital bridge between raw scans and interactive gameplay. Ultimately, the training process behind this technique can reproduce real-world appearance in ways traditional methods struggle to match, capturing intricate details like hair, semi-transparency, translucency, and subsurface scattering with unprecedented accuracy.