Photogrammetry and 3D representation
Gaussian Splatting
How photographs of a place become a scene you can explore from new perspectives.
A scene made of Gaussians.
3D Gaussian Splatting (3DGS) represents a scene with oriented three-dimensional Gaussians. Each has a position, shape, opacity and appearance. Projecting and combining them produces an image from the chosen camera.
The method by Kerbl, Kopanas, Leimkühler and Drettakis, presented at SIGGRAPH 2023, combines an explicit representation with differentiable rendering. Original project ↗
From capture to exploration.
Multiview capture
Sharp photographs with overlap and changes in viewpoint help establish correspondences. Avoid blur, abrupt lighting changes and moving elements.
Calibration and structure
Structure-from-Motion estimates camera poses and a sparse point cloud. COLMAP can extract features, match them and perform this reconstruction. Calibration and coverage influence the result.
Training and export
The reference implementation optimizes the model with PyTorch and CUDA extensions. Training and viewing are different tasks, with different hardware and memory requirements.
What a splat stores and how it is rendered.
The center of the Gaussian in space.
Controls extent and orientation; it can be parameterized using scale and rotation.
Modulates its contribution to the image.
Spherical harmonics model color variations with viewing direction.
G(x) = exp[ −½ (x − μ)ᵀ Σ⁻¹ (x − μ) ]
This expression describes the spatial weight without a normalization factor. Gaussians are projected as elliptical footprints and composited with transparency in visibility order. Optimization adjusts their parameters and density. Technical paper, sections 4–6 ↗
Training and evaluation
The official repository can separate training and test images using --eval. Comparing views not used to fit the model helps assess generalization. The implementation includes PSNR, SSIM and LPIPS metrics; these do not replace visual inspection for artifacts. Reference evaluation ↗
Explore appearance and understand its limits.
A visual reconstruction does not guarantee metric accuracy or automatically create a watertight mesh. Areas without photographic coverage may remain incomplete; reflective surfaces and moving objects may produce artifacts.
Performance depends on the Gaussian count, resolution, memory and viewer. A PLY file can store different kinds of data: displaying its centers as points is not equivalent to rendering full Gaussians.
Explore TinyPlanet’s reconstruction collection in Observatory Explorer and discover how the view changes as you visit each observatory.
Further reading