SA-GS: Scale-Adaptive Gaussian Splatting for Training-Free Anti-Aliasing

Given a pixel $P_{t}$, when traversing Gaussian primitives that have been order-sorted within a tile, we compute the distance between the centers of the $S\times S$ sub-pixels and the center of the Gaussian primitive separately, as shown in Fig. 5(a). These sub-pixels have independent $\alpha$-blending processes and cumulative transparency $T_{s}$.The color of a pixel $C_{t}$ is determined by averaging the color of these sampled sub-pixels:

where $G$ is the number of Gaussian primitives on the z-buffer, $\alpha_{s}^{i}$ is the opacity calculated based on the distance between $s_{th}$ sub-pixel and $i_{th}$ Gaussian, and $SH_{i}$ is the spherical harmonic coefficient of the $i_{th}$ Gaussian. The function $\mathcal{F}(\cdot)$ is used for the spherical harmonic coefficient to color conversion. For fast convergence, we set $S=3$ in all experimental settings.

When the super-sampling hyperpameter of $S$ goes to infinity, it becomes integration. Consider a single Gaussian’s projection on the 2D camera plane, a 2D Gaussian whose PDF we can represent as $f(x,y)$, where $x$ and $y$ are coordinates on the camera plane, and we take axes of the 2D Gaussian Projection as the coordinate system axes, thus making the correlation of the projected Gaussians zero. As the correlation is zero we have that $f(x,y)$ can further be factored into the product $g_{x}(x)g_{y}(y)$, where $g(t)=\frac{\exp(-\frac{1}{2}(\frac{t}{\sigma})^{2})}{\sqrt{2\pi}\sigma}$. Let $\Phi(t)=\int_{-\infty}^{t}g(t)dt$ (subscripts $x$ or $y$ omitted) be the Gaussian integral. Let the region inside the pixel be $P$. Hence when calculating $\alpha$ during the traversal of the Gaussian z-buffer, we need to find the following double integral:

Axis-aligned Case. When the axes of the pixel are parallel to the 2D Gaussian Projection’s axes, the evaluation of (5) is simple because it can be calculated as the product of two Gaussian single integrals:

where $P_{x\min}$ and $P_{x\max}$ are marginal Gaussian intervals on the x-axis, $P_{y\min}$ and $P_{y\max}$ are the same on the y-axis. However, when the pixel’s sides is not aligned with the axes, $x$ and $y$ will be related. In the evaluation of (5) this is reflected as the inner integral containing variable instead of constant limits, and therefore the factorization carried out in (3.2.2) will no longer be applicable.

Pixel Rotation. In our implementation, we solve this problem by rotating the pixel such that it aligns with the Gaussian’s axes, as shown in Fig. 5(b). Thus, the integral can be easily computed using this approach as described in (3.2.2). However, rotating the pixel causes a deviation between the integral region and the original pixel region. We prove that for any pixel close enough to the center of the Gaussian to be affected during $\alpha$-blending, there exists a theoretical upper bound for the error. We also verify through numerical experimentation that this is empirically a good approximation. Detailed proofs and experimental results are provided in the supplementary material.

We simulated the effect of zoom-out and zoom-in on this dataset and retrained all the baseline models for comparison. The same setup of Mip-Splatting[24] is used. Specifically, for zoom-out, we trained 3DGS[10] using full resolution images and then plug in our method to test on progressively lower resolutions (1×, $\nicefrac{{1}}{{2}}$×, $\nicefrac{{1}}{{4}}$×, $\nicefrac{{1}}{{8}}$×). For zoom-in, we trained 3DGS[10] using $\nicefrac{{1}}{{8}}$× resolution images and then plug in our method to test on progressively higher resolutions (1×, 2×, 4×, 8×). Table 1 and Table 2 show the quantitative results for these two experiments.

When zooming out, our method gives comparable results at the training resolution and better performance at lower resolutions. The (heuristic and fixed) dilation operation of 3DGS[10] leads to severe degradation at low resolutions. Mip-Splatting[24] replaces 3DGS’s dilation operation with 3D smoothing and a 2D Mip filter, but still suffers from scale ambiguity at different resolutions. As depicted in Figure 6, our 2D scale-adaptive filter ensures that the Gaussian distribution is consistent with the training settings. Our integration and super-sampling modules further enhance the anti-aliasing effect. The super-sampling version of our method gives the best results for this setting, which exceeds Mip-Splatting[24] by 1.1dB in PSNR (see Tab. 1).

When zooming in, 3DGS[10] shows severe erosion artefacts at high resolution. Mip-Splatting[24] uses the 3D smoothing filter to reduce the effects of scale ambiguity of 2D filtering. As depicted in Figure 7, our method keeps the Gaussian scales consistent and greatly reduces erosion artefacts. Note that integration and super sampling are only designed to address the decrease in sampling frequency (zoom-out). The most significant contribution is made by 2D scale-adaptive filter, which produces results comparable to Mip-Splatting[24].

Following Mip-splatting, all baseline models were trained using multi-scale data from the $train+test$ section of the dataset and evaluated with multiscale data from the $val$ section. We follow the image sampling ratio in Mip-Splatting[24] to train the 3DGS[10]. Our quantitative evaluation is shown in Table 3. Our approach yields comparable results with Mip-Splatting[24], and we use vanilla 3DGS[10] on multi-scale training only and do not need to modify the training procedure. Meanwhile, our approach significantly outperforms 3DGS[10], demonstrating stable performance at different resolutions. 3DGS[10] performance degrades as resolution decreases, even in the case that it is trained on multi-scale.

We maintain a experiment protocol consistent with the Mip-NeRF 360[2] experiment mentioned above (Section 4.2) to evaluate the zoom-out and zoom-in effects. We also keep the same data split as in the multi-scale training experiment described above. Table 4 and Table 5 show the quantitative results for zoom-out and zoom-in effects. The qualitative results are shown in Fig. 8.

For zoom-out, our method achieves performance close to 3DGS[10] at training resolution and a steady increase in performance at lower resolutions. The super-sampling version of our method $SA\text{-}GS_{sup}$ significantly outperforms Mip-Splatting[24] to achieve performance with a gain of 3.61dB in PSNR in this setting (see Tab. 4). For zoom-in, Our 2D scale-adaptive filter achieves comparable results to Mip-Splatting[24], and our method is training-free.

Let the center of the pixel be at coordinates $(x_{c},y_{c})$, and the side length of the pixel be $l$. Let the pixel have a counterclockwise tilt of $\theta$ with respect to the standard x-axis. The first problem we need to solve would be to eliminate the tilt in order to simplify the double integral. In order to do so, we would need to apply a rotation transformation to the Bivariate Gaussian Distribution without changing its main form. Hence we first normalize it into a Bivariate Normal Distribution via scaling.

We construct $x^{*},y^{*}=\frac{x}{\sigma_{x}},\frac{y}{\sigma_{y}}$ such that we have the Normal Distribution $g_{x}^{*}(x^{*})=\frac{\exp(-\frac{1}{2}x^{*2})}{\sqrt{2\pi}}=\sigma_{x}g_{x}(x)$ ($g_{y}^{*}$ is analogous). Hence we have:

The rotational symmetry of the Bivariate Normal Distribution now allows us to rotate the pixel clockwise by $\theta$ with respect to the origin without changing the integral (this does not affect the integral). Now the new pixel region, $P^{*}$, is a parallelogram with top and bottom edges parallel to the x-axis. Let the four corners (labeled in order analogously to quadrants) be labeled respectively $P_{1}(x_{1},y_{1})$, $P_{2}(x_{2},y_{1})$, $P_{3}(x_{3},y_{3})$, $P_{4}(x_{4},y_{3})$, and the slope of the edges $P_{1}P_{4}$ and $P_{2}P_{3}$ be $k$. WLOG let $y_{1}\geq 0$ (otherwise conduct a reflection across the x-axis).

In order to find bounds for $\iint\limits_{P^{*}}f^{*}(x^{*},y^{*})dx^{*}dy^{*}$, if we keep the current boundary for $P^{*}$, we will need to integrate $\Phi g$, which would be unrepresentable. In order to find the theoretical range of the error, we instead try to fix the sliced Gaussian distribution by fixing $y^{*}$. Combining with (8), we readily have the following bounds:

Fig. 11 presents a visualisation of the theoretical analysis. Obviously, when the coordinates on the pixel are bounded by a constant times the respective standard deviations, coordinates of $P_{i}$ will also be bounded by the constant. Further, $k$ is only affected by $\theta$, $\sigma_{x}$, and $\sigma_{y}$. Hence the original double integral is bounded in our approximation, and so is its error.