RaySplats: Ray Tracing based Gaussian Splatting

Krzysztof Byrski 1 Marcin Mazur 1 Jacek Tabor 1 Tadeusz Dziarmaga 1 Marcin Kadziołka
˛ 1
Dawid Baran 1
1
Przemysław Spurek

Abstract RaySplats for Glass Objects with Shadows

3D Gaussian Splatting (3DGS) is a process that
arXiv:2501.19196v1 [[Link]] 31 Jan 2025

enables the direct creation of 3D objects from 2D

images. This representation offers numerous ad-
vantages, including rapid training and rendering.
However, a significant limitation of 3DGS is the
challenge of incorporating light and shadow re-
flections, primarily due to the utilization of raster-
ization rather than ray tracing for rendering. This
paper introduces RaySplats, a model that employs
ray-tracing based Gaussian Splatting. Rather than
utilizing the projection of Gaussians, our method
employs a ray-tracing mechanism, operating di-
rectly on Gaussian primitives represented by con-
fidence ellipses with RGB colors. In practice, RaySplats for Mirror Reflections
we compute the intersection between ellipses and
rays to construct ray-tracing algorithms, facili-
tating the incorporation of meshes with Gaus-
sian Splatting models and the addition of lights,
shadows, and other related effects. https://
[Link]/KByrski/RaySplatting

1. Introduction
3D Gaussian Splatting (3DGS) (Kerbl et al., 2023) is a
leading neural rendering technique capable of transforming
2D images into coherent 3D scenes with remarkably sharp
reconstruction. 3DGS uses a collection of Gaussians charac-
terized by color and opacity to represent scenes, increasing
training efficiency and enabling real-time rendering of high- Figure 1. RaySplats (our) incorporates ray tracing into the
quality meshes. The rendering process involves projecting 3D Gaussian Splatting framework. This allows us to integrate
these 3D components onto a 2D plane. While numerically meshes with lighting conditions and mirror effects.
efficient, it has certain limitations, particularly in terms of
integrating 3DGS with objects based on meshes and lighting
of 3D Gaussian Splatting with the ray-tracing model, lever-
effects, a process that poses significant challenges, primarily
aging bounding primitives (polytopes) for each Gaussian.
due to the complexities associated with ray tracing (Glass-
This approach utilizes iterations and is efficient, producing
ner, 1989).
high-quality renders thanks to the NVIDIA OptiX program-
3D Gaussian Ray Tracing (3DGRT) integrates the principles ming interface (Parker et al., 2010). However, 3DGRT
*
involves the approximation of primitives. This technique
Equal contribution 1 Jagiellonian University, Faculty of Mathe- can be challenging, especially for flat Gaussians, which are
matics and Computer Science, Cracow, Poland. Correspondence
to: Przemysław Spurek <[Link]@[Link]>. often used to generate meshes from 3DGS. Inter-Reflective
Gaussian Splatting (IRGS) (Gu et al., 2024) integrates flat
Gaussians from 2DGS (Huang et al., 2024) to effectively

1
 RaySplats: Ray Tracing based Gaussian Splatting

capture inter-reflections. Utilizing the rendering equation,

IRGS proposes a differentiable 2D Gaussian ray-tracing
approach within the 2DGS framework, thereby facilitating
re-lighting modeling within 2DGS.
As demonstrated above, although both 3DGRT and 2DGS
are capable of producing high-quality renders, they are sub-
ject to certain limitations. Specifically, 3DGRT utilizes
bounding primitives (polytopes), while IRGS is exclusively
dedicated to relighting 2DGS. In order to address this is-
sue, we introduce Ray Tracing based Gaussian Splatting
(RaySplats)1 , a model that utilizes ray tracing during both
training and inference. RaySplats eschews the use of the
polytope approximation and is compatible with 2D cases
involving flat Gaussians. In practice, we calculate the dis- Figure 2. RaySplats (our) uses ray-tracing based solutions. In prac-
tance between Gaussians represented by ellipses with rays. tice, we need two important points on rays passing through Gaus-
This approach facilitates the integration of our model with sian distributions. Then, the maximum response point is utilized
for aggregating colors along each ray. On the other hand, the
light conditions and the merging of scenes with mesh-based
intersection of Gaussian confidence ellipses is used to efficiently
objects.
detect Gaussians with non-empty intersection with the ray.
The following is a concise summary of our contribution:

• we propose RaySplats, a novel differential rendering This technique utilizes a tile-based rasterizer to project 3D
procedure for 3D Gaussian Splatting, which utilizes ray Gaussians onto a 2D plane, achieving state-of-the-art results
tracing during both the training and inference phases, in terms of visual fidelity and rendering efficiency. The
yet does not necessitate the use of polytope approxima- efficacy of 3DGS has been demonstrated in a variety of
tion, applications, including geometry reconstruction (Kerbl et al.,
2023), dynamic scene reconstruction (Yang et al., 2023;
• RaySplats enables the processing of lighting effects, Waczyńska et al., 2024), inverse rendering (Jiang et al.,
including reflections, shadows, and transparency, 2024; Liang et al., 2024), and street scene visualization
(Chen et al., 2023). However, relying on rasterization makes
• RaySplats facilitates the integration of 3D Gaussian it harder for Gaussian Splatting to accurately replicate ray-
Splatting within mesh-based models, thereby enabling based effects (Moenne-Loccoz et al., 2024).
the application of 2D Gaussians.
Ray-Tracing Methods To address the aforementioned
2. Related Works limitation, 3D Gaussian Ray Tracing (3DGRT) (Moenne-
Loccoz et al., 2024) introduced a differentiable ray tracer for
This section is divided into two parts. First, we describe
3D Gaussian primitives. This method efficiently performs
novel-view synthesis in general. Then, we discuss ray-
ray tracing across 3D Gaussian primitives, allowing radiance
tracing methods.
computation along arbitrary rays. Utilizing the NVIDIA
OptiX programming interface (Parker et al., 2010), 3DGRT
Novel-View Synthesis Neural Radiance Fields (NeRFs)
achieves high-quality renders with improved accuracy over
(Mildenhall et al., 2021) have significantly advanced the
rasterization-based approaches. However, its scope remains
field of novel-view synthesis by representing scenes as a
constrained to Gaussian primitives, limiting its compatibility
volumetric radiance field encoded in a coordinate-based
with generalized distributions (Condor et al., 2025; Hamdi
neural network. NeRFs allow the network to be queried at
et al., 2024; Kasymov et al., 2024).
any point to retrieve volumetric density and view-dependent
color, resulting in photorealistic scene renderings. Due to Subsequent advancements have focused on particular chal-
their high-quality output, NeRFs have become a founda- lenges inherent to Gaussian-based rendering. For exam-
tional approach for novel view synthesis. ple, Inter-Reflective Gaussian Splatting (IRGS) (Gu et al.,
2024) extends the 2D Gaussian Splatting (2DGS) framework
Building on NeRF’s success, 3D Gaussian Splatting (3DGS)
(Huang et al., 2024) to model inter-reflections by incorpo-
(Kerbl et al., 2023) introduced a novel representation of
rating a differentiable ray-tracing approach. By integrat-
scenes using fuzzy, anisotropic 3D Gaussian point clouds.
ing flat Gaussians from 2DGS with the rendering equation,
1
The source code is available at [Link] IRGS facilitates relighting modeling. While IRGS achieves
KByrski/RaySplatting. high-quality results, its emphasis on 2D Gaussian primitives

2
 RaySplats: Ray Tracing based Gaussian Splatting

RaySplats for Glass Objects with Shadows

Figure 3. RaySplats (our) allows us to combine 3D Gaussian splatting with a mesh-based rendering using lighting effects such as shadows
and transparency.

RaySplats for Mirror Reflections

Figure 4. RaySplats (our) allows us to combine 3D Gaussian splatting with a mesh-based rendering using lighting effects such as mirror
reflections.

restricts its applicability to 3D scene reconstruction or inte- a ray with a Gaussian distribution is outlined. It is important
gration with mesh-based models. Another approach, EnvGS to note that identifying Gaussians that intersect a given ray
(Xie et al., 2024), introduces a ray tracing-based approach is crucial for aggregating colors along that ray. This concept
to model complex reflections in real world scenes. By em- is subsequently elaborated upon in great detail. Finally, the
ploying environment Gaussian primitives for near-field and loss function of the RaySplats model is presented.
high-frequency reflections, it overcomes the limitations of
environment maps, which struggle with accurate reflection 3D Gaussian Splatting 3D Gaussian Splatting (3DGS)
modeling due to distant lighting assumptions. EnvGS in- (Kerbl et al., 2023) is a method of representing 3D scenes
tegrates these environment Gaussians with base Gaussian as a set of Gaussians
primitives, enabling detailed, real-time rendering. Unfortu-
nately, this method involves a two-step optimization process, G = {(N (mi , Σi ), α̂i , ci )}ni=1 , (1)
where, in practice, ray tracing is applied solely to the previ-
where trainable constants mi , Σi , α̂i , and ci denote the
ously trained 3DGS.
position (mean), covariance, opacity, and color of the i-
Despite advancements, challenges persist in integrating th component. The representation of color employs the
Gaussian representations with meshes and simulating light- Spherical Harmonics (SH) approach (Fridovich-Keil et al.,
ing effects like reflections, shadows, and transparency. Our 2022; Müller et al., 2022). 3DGS utilizes the factorization
RaySplats method overcomes these difficulties by extend- of the covariance matrix
ing the capabilities of 3DGS to include lighting effects and
allowing seamless integration with mesh-based models. Σ = RSSRT , (2)

where R is the rotation matrix and S is a diagonal matrix

3. RaySplats – Ray Tracing based Gaussian containing the scaling parameters.
Splatting The 3DGS algorithm entails the projection of Gaussians
This section is intended to provide an overview of our model. onto the image plane, wherein the color for a specific pixel
First, the classical 3D Gaussian Splatting approach is dis- is determined by the blending of a sequence of overlapping
cussed. Next, the process of determining the intersection of points, which are sampled from the respective Gaussian
distributions N (mi , Σi ) (we assume that these are indicated

3
 RaySplats: Ray Tracing based Gaussian Splatting

RaySplats for Glass Objects with Shadows Intersection of the Gaussian Distribution with the Ray
We use approaches similar to DSS (Yifan et al., 2019), which
describes a high-fidelity differentiable renderer for point
clouds. The expected color C(r) of the camera ray, defined
as r(t) = o + td (where o represents the origin and d
the direction), is determined by combining the colors and
opacities of Gaussians intersected by the ray. Therefore,
it is important to define how the Gaussians and the ray
intersect. The Gaussian component can be thought of as an
ellipse. Suppose a random vector x follows a 3D Gaussian
distribution. The Mahalanobis distance of the vector x
from the mean vector µ is denoted by the scalar quantity
(x − µ)T Σ−1 (x − µ). The surface on which the value of the
random variable is constant forms an ellipse (or an ellipsoid
in a multivariate context) with its center at µ. This ellipse,
also known as a probability contour, describes the minimum
region (or volume in a multivariate situation) that contains
Figure 5. RaySplats (our) is capable of modeling glass elements a given probability under the assumption of a Gaussian
in the 3D Gaussian Splatting environment, thereby facilitating the distribution. For the confidence level α ∈ [0, 1], we can
accurate visualization of glass reflections and the distortion of light
compute the confidence ellipsoid
due to refraction.

Eµ,Σ,α = {x ∈ R3 : (x − µ)T Σ−1 (x − µ) = Q}. (6)

by indices i ∈ {1, . . . , N }). This procedure is analogous to
that described (Kopanas et al., 2021; 2022). The color of a
pixel is thus determined as follows: where Q = Fχ−1 2 (3) (α) is the quantile of order α of the

N i−1
χ2 (3) distribution (i.e., with three degrees of freedom). Our
C=
X
ci αi
Y
(1 − αj ) (3) rendering procedure uses a “2.5D” approach using a point
i=1 j=1
from a Gaussian confidence ellipse. In practice, we take
the point from the intersection of the ellipses and the ray
where αi is given by evaluating a 2D Gaussian with covari- closest to the camera position. If the line does not hit the
ance matrix Σ multiplied by a learned per-Gaussian opacity Gaussian at all, or if the closest intersection point (i.e., the
α̂i (Yifan et al., 2019). one with the smaller value of the t parameter) is negative
and does not belong to the ray, we treat that ray as the ray
The aforementioned formula is a consequence of point- missing the Gaussian. In our implementation, instead of the
based alpha blending (Kerbl et al., 2023), wherein the color confidence level α, for flexibility we use the configurable
of a ray is defined as parameter Q, which is the desired quantile of order α of the
N χ2 (3) distribution, where α is the confidence level of the
X
C= Ti (1 − exp(−σi δi ))ci (4) Gaussian distribution. In practice, we compute the intersec-
i=1 tion between the line parallel to the ray and the ellipsoid
formed by the points whose Mahalanobis distance to the
with   mean of the Gaussian is equal to Q.
i−1
X
Ti = exp− σj δj , (5) The computation of such an intersection is a well-defined
j=1 problem that can be effectively solved using the following
proposition (for the proof, see Appendix A.1).
and samples of density σi , transmittance Ti , and color ci are
collected along the ray with intervals δi . It is imperative to Proposition 3.1 (Following (Hearn et al., 2010)). Consider
acknowledge the primary challenge in the parametrization of the ray defined as r(t) = o + td, where o is the origin and
each Gaussian, which encompasses its position, a covariance d is the direction, and the Gaussian component N (µ, Σ).
matrix (3 × 3 matrix), color, and opacity. On the other The first (closest to the origin o) intersection between the
hand, neural rendering is substituted by projecting Gaussian ray r(t) and the confidence ellipse Eµ,Σ,q is given by
distributions onto a 2D plane. This process is efficient
in training and inference; however, issues emerge in the
if ⟨o′ , d′ ⟩ ≥ 0
(
context of light conditioning. To address this challenge, we t∗ ,
t1 = ⟨o′ ,o′ ⟩−Q , (7)
propose a solution that integrates 3DGS with ray tracing. ⟨d′ ,d′ ⟩t∗ , if ⟨o′ d′ ⟩ < 0

4
 RaySplats: Ray Tracing based Gaussian Splatting

where
s
D E 2
d′ ′
′ ′ sgn⟨o′ ,d′ ⟩ Q− o′ − d
o′ , ∥d′∥
⟨o ,d ⟩ ∥d′ ∥
t∗ = − ⟨d′ ,d′ ⟩ − ⟨d′ ,d′ ⟩ .
(8)

The application of the NVIDIA OptiX programming inter-

face (Parker et al., 2010) and Proposition 3.1 facilitates the
efficient implementation of the detection of all ellipses pos-
sessing non-empty intersection with a ray in a numerically
stable manner. Consequently, the Gaussian components
aligned with the ray can be effectively determined in this
phase.

Color Aggregation Along the Ray Suppose we have

collected all the Gaussians along the ray defined as r(t) =
o + td (where o is the origin and d is the direction) in the
following family:

Gr(t) = {(N (mi , Σi ), α̂i , ci )}N

i=1 . (9)

Our methodology is similar to 3DGRT (Moenne-Loccoz

et al., 2024) in that we compute αi by taking the product of Figure 6. RaySplats (our) combines meshes with 3DGS-based rep-
the learned opacity for each Gaussian, denoted as α̂i , and resentations with different material structures.
the peak of the 3D Gaussian probability density function
scalar field normalized over the specific ray, i.e,

3DGRT, which do not utilize the theoretical maximum for

the number of intersectable Gaussians, imposing a threshold
n 3p
o
αi = α̂i max (2π) 2 |Σi |fN (mi ,Σi ) (r(t)) . (10)
t≥0 on the transmittance value Ti , RaySplats applies a config-
urable upper bound on the Gaussians intersected by the ray
We can then calculate the Gaussian maximum response ac- in the forward phase. The enforcement of this limit enables
cording to the 3DGRT (Moenne-Loccoz et al., 2024), using the storage of the indices of all Gaussians intersected by
the following proposition (for the proof, see Appendix A.2). the ray in an indices buffer, thereby preventing buffer over-
flow. This stored information is subsequently utilized in the
Proposition 3.2 (Following (Moenne-Loccoz et al., 2024)).
backward phase for gradient computation, eliminating the
Consider the ray defined as r(t) = o + td, where o is the
need for re-traversing the rays. As ray traversal constitutes
origin and d is the direction, and a family of Gaussians
the most resource-intensive operation in ray tracing-based
Gr(t) = {(N (mi , Σi ), α̂i , ci )}N
i=1 that possess a non-empty Multi-View Stereo tasks, this strategy reduces the number
intersection with the ray. Then we have
of ray-Gaussian intersections calculated by the algorithm by
n 3p
o half.
αi = α̂i max (2π) 2 |Σi |fN (mi ,Σi ) (r(t))
t≥0
′

d′
 , (11) In addition, RaySplats introduces the second additional
− 21 o′ − d′ ,o′
= α̂i · e
′
∥d ∥ ∥d ∥ threshold for computing the gradient of the last “meaningful”
Gaussian. If Ti falls below the first threshold, we do not kill
where Σi = Ri Si SiT RiT , o′ = Si−1 RiT (o − mi ), and d′ = the ray immediately, but allow it to traverse the scene and
Si−1 RiT d. hit some more Gaussians, since they carry the information
necessary to compute the gradient for the last “meaningful”
We emphasize that RaySplats models αi as the product of Gaussian, i.e., the Gaussian with index i for which Ti falls
the trainable parameter α̂i and the value of the probabil- below the first threshold.
ity density function of the 3D Gaussian. Therefore, we
To illustrate the RaySplats concept, consider the situation
use Eq. 3 and the Gaussian maximum response for each
where the first Gaussian hit by the ray is close to fully
component, as in Proposition 3.2.
opaque (i.e., α1 ≈ 1). Hence T1 = (1−α1 ) ≈ 0. Obviously,
In RaySplats, a distinct color aggregation method is em- the subsequent Gaussians along the ray have no effect on
ployed during the forward phase. In contrast to 3DGS and the final aggregated color. Note, however, that the second

5
 RaySplats: Ray Tracing based Gaussian Splatting

component of the gradient for α1 of the aggregated color, is a structural similarity loss that preserves fine details and
which is given by the following formula: texture consistency in the rendered image, and λ controls
  the balance between these two terms, allowing a trade-off
N j−1
dC X Y between pixel-wise accuracy and perceptual similarity.
= c1 − cj αj  (1 − αk ) , (12)
dα1 j=2 k̸=1

can be arbitrarily large, since the factor 1 − α1 has been

reduced. Therefore, we use:
j−1
Y
(1 − αk ) < ε2 (13)
k=i+1

as a ray termination criterion before intersecting the j-th

Gaussian in the final phase where the Gaussians are col-
lected for the last “meaningful” Gaussian gradient calcula-
tion.
The algorithm below provides more details for the color
aggregation operation in the forward phase.
Algorithm 1 provides more details for the color aggregation
operation in the forward phase. As long as the actual number
of Gaussians (the value of the variable k) does not exceed
the configurable maximum allowed number of Gaussians
(line 5), we traverse the ray (line 6), and if the intersection
is not empty (line 7), we aggregate the output image color I
(lines 11– 19). Otherwise, we set the sentinel value in the
indices buffer to −1 for the sake of gradient computation
in the backward phase (line 8) and terminate the algorithm
(line 9). If, after the color aggregation, the transmittance T1
falls below the first threshold, i.e., ε1 (line 20), we set the
variable second_phase (line 22), indicating that it is time
to proceed to the second phase of the algorithm, where the
Gaussians are collected for the sake of the last “meaningful”
gradient computation, if it was not yet set, or update the
transmittance T2 otherwise (line 24). Finally, we check
(line 27) if the transmittance T2 falls below the second
threshold, i.e. ε2 . If so, we terminate the algorithm after
setting the sentinel value in the indices buffer (lines 28– 29)
or “move” the ray origin o a bit so that the ray does not
hit the Gaussian with the index given by the index variable
again during the next run of the for loop (line 33).

Loss Function of the RaySplats Model Given that the

RGB color space is employed, the loss function is defined
as the average of loss functions calculated independently for 4. Experiments
the specific RGB components. We employ the analogical In this section, we present the findings from our experimen-
loss as in the 3DGS; however, we replace the L1 norm with tal study, encompassing both quantitative and qualitative
the L2 norm and the spherical harmonics with the RGB results. In addition, we have conducted an ablation study
color components. This results in the following formula for to further explore the impact of selected key parameters of
the loss function of each color component: RaySplats.
L = (1 − λ) L2 + λLD-SSIM , (14)
where L2 is the squared error between the predicted and tar- Datasets and Metrics Following (Moenne-Loccoz et al.,
get color component values, ensuring color fidelity, LD-SSIM 2024) we conduct experiments on three widely used datasets:

6
 RaySplats: Ray Tracing based Gaussian Splatting

Figure 7. Ablation study investigating the effect of three key parameters of the RaySplats model (our): the upper limit of Gaussians that
can be hit by the ray, the ray termination threshold ε1 used throughout the forward phase, and the quantile Q of order α of the χ2 (3)
distribution. The results are presented in terms of the PSNR metric (greater is better).

Table 1. Quantitative evaluation of RaySplats (our) on the Mip-NeRF360 (Barron et al., 2022), Tanks and Temples (Knapitsch et al., 2017),
and Deep Blending (Hedman et al., 2018) datasets. Comparison is made with the following baselines: Plenoxels (Fridovich-Keil et al.,
2022), INGP (Müller et al., 2022), M-NeRF360 (Barron et al., 2021), 3DGS (Kerbl et al., 2023), and 3DGRT (Moenne-Loccoz et al., 2024).
Mip-NeRF360 Tanks&Temples Deep Blending
SSIM ↑ PSNR ↑ LPIPS ↓ SSIM ↑ PSNR ↑ LPIPS ↓ SSIM ↑ PSNR ↑ LPIPS ↓
Plenoxels 0.626 23.08 0.719 0.379 21.08 0.795 0.510 23.06 0.510
Spherical Harmonics

INGP-Base 0.671 25.30 0.371 0.723 21.72 0.330 0.797 23.62 0.423
INGP-Big 0.699 25.59 0.331 0.745 21.92 0.305 0.817 24.96 0.390
M-NeRF360 0.792 27.69 0.237 0.759 22.22 0.257 0.901 29.40 0.245
3DGS-7K 0.770 25.60 0.279 0.767 21.20 0.280 0.875 27.78 0.317
3DGS-30K 0.815 27.21 0.214 0.841 23.14 0.183 0.903 29.41 0.243
3DGRT - - - 0.830 23.20 0.222 0.900 29.23 0.315
RGB RaySplats 0.846 27.31 0.237 0.829 22.20 0.202 0.900 29.57 0.320

Mip-NeRF 360 (Barron et al., 2022), Tanks and Tem- facilitates the incorporation of light effects, transparency,
ples (Knapitsch et al., 2017), and Deep Blending (Hed- and shadows through the employment of ray tracing tech-
man et al., 2018). For consistency, we evaluate the same niques. This approach eliminates the necessity of relying on
scenes as in (Moenne-Loccoz et al., 2024). Specifically, for formats contingent on viewing directions, thereby ensuring
Mip-NeRF360, we use four indoor scenes: room, counter, a more robust and consistent results.
kitchen and bonsai, as well as three outdoor scenes: bicycle,
garden and stump. On the Tanks and Temples dataset, we Qualitative Results By leveraging ray tracing, which en-
evaluate two large outdoor scenes: train and truck. For the ables the simulation of lighting conditions, RaySplats incor-
Deep Blending dataset, we use two indoor scenes: playroom porates RGB colors. Nevertheless, in almost every practical
and drjohnson. In line with prior work, all evaluations use scenario, our model delivers outcomes that are on par with
images downsampled by a factor of two for indoor scenes traditional 3D Gaussian Splatting employing spherical har-
and by a factor of four for outdoor scenes. monics. Specifically, as illustrated in Fig. 8, RaySplats
We maintain consistent train/test splits across all datasets. consistently delivers high-quality reconstructions. Addi-
For evaluation, we use three widely used metrics: PSNR, tionally, it is compatible with mesh-based models that can
SSIM (Wang et al., 2004), and LPIPS (Zhang et al., 2018). undergo modifications through the incorporation of ray-
tracing effects. As illustrated in Fig. 3, the integration of
glass properties within the model facilitates its connection
Quantitative Results In most cases, ourRaySplats model to meshes. The incorporation of shadows, transparency, and
demonstrates the capacity to attain outcomes that are anal- light reflections serves to enhance the visual fidelity of the
ogous to those of classical 3D Gaussian Splatting, as illus- model. Fig. 4 demonstrates model’s capacity to mirror Gaus-
trated in Table 1. It is noteworthy that our approach employs sian reflections, while Fig. 5 showcases the glass dragon on
RGB colors instead of spherical harmonics, a choice that a Lego structure from the NeRF synthetic dataset. Addi-
aligns with the prevalent utilization of RGB color representa- tionally, Fig. 6 illustrates the versatility of our technique by
tion in conjunction with ray tracing. Furthermore, our model showcasing it on a variety of colored meshes. It is evident

7
 RaySplats: Ray Tracing based Gaussian Splatting

ent computation process was implemented manually from

scratch, without recourse to the automatic differentiation
systems, based on the formulas derived in Appendices A.3,
A.4, A.5, A.6, and A.7. To compute the gradient of the loss
function, we swept the successive indices of the Gaussians
dI
stored in the buffer and, after computing the value of d□ i
,
we accumulated the derivative with respect to the □ param-
eter of the i-th Gaussian in the traversal order using the
global memory atomicAdd() operation. Thanks to the reuse
dI
of the previously computed values of dα i
, we were able to
compute each of the derivatives in O (1) time with respect
to the number of stored Gaussian indices N , without having
dI
to compute the value of dα i
from scratch in O (N ) time. If
dI
the value of dαi was not finite or the transmittance Ti fell
Figure 8. Examples of renderings of different types: the first col- below the threshold ε1 , we treated such a Gaussian as the
umn shows the ground truth image, the second column shows the
last “meaningful” Gaussian (from the perspective of color
rendering of the optimized RaySplats, and the third column shows
the rendering of 3D Gaussian splatting with RGB colors. The first
aggregation) and computed the derivatives with respect to
two rows consist of data from the Mip-NeRF360 dataset, while the its parameters using the regular naive formula based on the
last row consists of the Truck example from the Tanks&Templates Gaussians with indices stored in the successive remaining
dataset. Note that RaySplats gives comparable results to the classic entries of the index buffer.
3DGS.
We optimized the parameters of our model using the ADAM
optimizer (also implemented by hand) with the following
that our approach is capable of representing both reflections values of the hyperparameters: β1 = 0.9, β2 = 0.999, and
and shadows with precision. ε = 0.00000001. For the Gaussian parameters α̂, sx , sy ,
and sz (according to (Kerbl et al., 2023)) we used the inverse
Ablation Study RaySplats includes several key param- sigmoid function. Our technique is based on the similar den-
eters that determine the number of Gaussians combined sification strategy as in (Kerbl et al., 2023) except for the
along the ray and have the impact on quality of the recon- parameters mx , my , and mz , since we do not use the norm
structed 3D scene. In Fig. 7 we present an ablation study of of the positional gradient but the “regular” geometric gradi-
three such parameters: the parameter Q being the quantile ent in the densification criterion. Note that the former does
of order α of the χ2 (3)-distribution (with three degrees of not make sense in the more advanced ray tracing settings
freedom) where α is the Gaussian distribution confidence that we intend to explore in our future work. In our tech-
level, the upper limit of Gaussians that can be hit by the ray, nique, we also do not periodically set the Gaussian trainable
and the ray termination threshold ε1 used throughout the opacities α̂ to the value close to 0, as this may lead to a
forward phase. decrease in the precision of the loss function gradient. This
is due to the configurable upper limit of Gaussians the ray
Based on the results, while it is always profitable (from the can hit during the traversing phase.
perspective of the loss function) to increase the upper limit
of Gaussians that can be hit by the ray (which seems to
be in line with related techniques), increasing the value of 6. Conclusions
Q and decreasing ε1 seems to pay off only up to a certain In this paper, we introduced RaySplats, a novel ap-
point where the value of the PSNR statistic begins to slowly proach that integrates ray tracing into 3D Gaussian Splat-
decrease. During our experiments, we set these parameters ting (3DGS) to overcome the limitations of traditional
(individually for each of the datasets) to values that represent rasterization-based methods. By directly operating on Gaus-
a trade-off between reconstruction quality (in terms of PSNR sian primitives represented by confidence ellipses with RGB
value) and performance. colors, our method enables more accurate light and shadow
interactions. The intersection of ellipses and rays is com-
5. Implementation Details puted to construct a ray-tracing framework that enhances
3DGS with realistic lighting effects. In consequence, the
We developed our technique in C/C++ using the NVIDIA incorporation of meshes, lights, and shadows is facilitated,
OptiX 8.0.0 SDK. We implemented separate CUDA kernels resulting in a significant enhancement of the visual fidelity
for the forward color aggregation phase and the backward of 3D Gaussian Splatting models.
phase, where the gradient is computed based on the Gaus-
sian indices stored in the index buffer. The entire gradi-

8
 RaySplats: Ray Tracing based Gaussian Splatting

Limitations Our method requires a specialized renderer Hedman, P., Philip, J., Price, T., Frahm, J.-M., Drettakis,
that supports both ray tracing and 3D Gaussian Splatting. G., and Brostow, G. Deep blending for free-viewpoint
image-based rendering. ACM Transactions on Graphics
Impact Statement (ToG), 37(6):1–15, 2018.
Huang, B., Yu, Z., Chen, A., Geiger, A., and Gao, S. 2d
This paper presents work that aims to advance the field
gaussian splatting for geometrically accurate radiance
of Machine Learning. There are many potential societal
fields. In ACM SIGGRAPH 2024 conference papers, pp.
consequences of our work, none which we feel must be
1–11, 2024.
specifically highlighted here.
Jiang, Y., Tu, J., Liu, Y., Gao, X., Long, X., Wang, W.,
References and Ma, Y. Gaussianshader: 3d gaussian splatting with
shading functions for reflective surfaces. In Proceedings
Barron, J. T., Mildenhall, B., Tancik, M., Hedman, P., of the IEEE/CVF Conference on Computer Vision and
Martin-Brualla, R., and Srinivasan, P. P. Mip-nerf: A Pattern Recognition, pp. 5322–5332, 2024.
multiscale representation for anti-aliasing neural radiance
fields. In Proceedings of the IEEE/CVF International Kasymov, A., Czekaj, B., Mazur, M., Tabor, J., and Spurek,
Conference on Computer Vision, pp. 5855–5864, 2021. P. Neggs: Negative gaussian splatting. arXiv preprint
arXiv:2405.18163, 2024.
Barron, J. T., Mildenhall, B., Verbin, D., Srinivasan, P. P.,
Kerbl, B., Kopanas, G., Leimkühler, T., and Drettakis, G. 3d
and Hedman, P. Mip-nerf 360: Unbounded anti-aliased
gaussian splatting for real-time radiance field rendering.
neural radiance fields. In Proceedings of the IEEE/CVF
ACM Trans. Graph., 42(4):139–1, 2023.
conference on computer vision and pattern recognition,
pp. 5470–5479, 2022. Knapitsch, A., Park, J., Zhou, Q.-Y., and Koltun, V. Tanks
and temples: Benchmarking large-scale scene reconstruc-
Chen, Y., Gu, C., Jiang, J., Zhu, X., and Zhang, L. Pe- tion. ACM Transactions on Graphics (ToG), 36(4):1–13,
riodic vibration gaussian: Dynamic urban scene re- 2017.
construction and real-time rendering. arXiv preprint
arXiv:2311.18561, 2023. Kopanas, G., Philip, J., Leimkühler, T., and Drettakis, G.
Point-based neural rendering with per-view optimization.
Condor, J., Speierer, S., Bode, L., Bozic, A., Green, S., In Computer Graphics Forum, volume 40, pp. 29–43.
Didyk, P., and Jarabo, A. Don’t splat your gaussians: Vol- Wiley Online Library, 2021.
umetric ray-traced primitives for modeling and rendering Kopanas, G., Leimkühler, T., Rainer, G., Jambon, C., and
scattering and emissive media. ACM Transactions on Drettakis, G. Neural point catacaustics for novel-view
Graphics, 2025. synthesis of reflections. ACM Transactions on Graphics
(TOG), 41(6):1–15, 2022.
Fridovich-Keil, S., Yu, A., Tancik, M., Chen, Q., Recht, B.,
and Kanazawa, A. Plenoxels: Radiance fields without Liang, Z., Zhang, Q., Feng, Y., Shan, Y., and Jia, K. Gs-ir:
neural networks. In CVPR, pp. 5501–5510, 2022. 3d gaussian splatting for inverse rendering. In Proceed-
ings of the IEEE/CVF Conference on Computer Vision
Glassner, A. S. An introduction to ray tracing. Morgan and Pattern Recognition, pp. 21644–21653, 2024.
Kaufmann, 1989.
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T.,
Gu, C., Wei, X., Zeng, Z., Yao, Y., and Zhang, L. Irgs: Ramamoorthi, R., and Ng, R. Nerf: Representing scenes
Inter-reflective gaussian splatting with 2d gaussian ray as neural radiance fields for view synthesis. Communica-
tracing. arXiv preprint arXiv:2412.15867, 2024. tions of the ACM, 65(1):99–106, 2021.
Moenne-Loccoz, N., Mirzaei, A., Perel, O., de Lutio, R.,
Hamdi, A., Melas-Kyriazi, L., Mai, J., Qian, G., Liu, R., Martinez Esturo, J., State, G., Fidler, S., Sharp, N., and
Vondrick, C., Ghanem, B., and Vedaldi, A. Ges: Gen- Gojcic, Z. 3d gaussian ray tracing: Fast tracing of particle
eralized exponential splatting for efficient radiance field scenes. ACM Transactions on Graphics (TOG), 43(6):
rendering. In Proceedings of the IEEE/CVF Conference 1–19, 2024.
on Computer Vision and Pattern Recognition, pp. 19812–
19822, 2024. Müller, T., Evans, A., Schied, C., and Keller, A. Instant
neural graphics primitives with a multiresolution hash
Hearn, D. D., Baker, M. P., and Carithers, W. Computer encoding. ACM Transactions on Graphics (ToG), 41(4):
graphics with open GL. Prentice Hall Press, 2010. 1–15, 2022.

9
 RaySplats: Ray Tracing based Gaussian Splatting

Parker, S. G., Bigler, J., Dietrich, A., Friedrich, H., Hobe-

rock, J., Luebke, D., McAllister, D., McGuire, M., Mor-
ley, K., Robison, A., et al. Optix: a general purpose ray
tracing engine. Acm transactions on graphics (tog), 29
(4):1–13, 2010.
Waczyńska, J., Borycki, P., Kaleta, J., Tadeja, S., and
Spurek, P. D-miso: Editing dynamic 3d scenes using
multi-gaussians soup. arXiv preprint arXiv:2405.14276,
2024.
Wang, Z., Bovik, A. C., Sheikh, H. R., and Simoncelli, E. P.
Image quality assessment: from error visibility to struc-
tural similarity. IEEE transactions on image processing,
13(4):600–612, 2004.
Xie, T., Chen, X., Xu, Z., Xie, Y., Jin, Y., Shen, Y., Peng, S.,
Bao, H., and Zhou, X. Envgs: Modeling view-dependent
appearance with environment gaussian. arXiv preprint
arXiv:2412.15215, 2024.

Yang, Z., Yang, H., Pan, Z., and Zhang, L. Real-time photo-
realistic dynamic scene representation and rendering with
4d gaussian splatting. arXiv preprint arXiv:2310.10642,
2023.
Yifan, W., Serena, F., Wu, S., Öztireli, C., and Sorkine-
Hornung, O. Differentiable surface splatting for point-
based geometry processing. ACM Transactions on Graph-
ics (TOG), 38(6):1–14, 2019.
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., and Wang,
O. The unreasonable effectiveness of deep features as a
perceptual metric. In Proceedings of the IEEE conference
on computer vision and pattern recognition, pp. 586–595,
2018.

10
 RaySplats: Ray Tracing based Gaussian Splatting

A. Theoretical Analysis
A.1. Computing a Ray-Gaussian Intersection
As explained in Section 3, the ray-Gaussian intersection is the intersection between the ray and the Gaussian confidence
ellipsoid parameterized by the value of the configurable parameter Q, which is the desired quantile of order α of the χ2 (3)
distribution, where α is the Gaussian confidence level. Obviously, the points x ∈ R3 belonging to the above ellipsoid satisfy
the following equation:

T
(x − mi ) Σ−1
i (x − mi ) = Q

Substituting the ray parametric equation in the t variable into the above equation yields:

T T
(r(t) − mi ) Σ−1 −1
i (r(t) − mi ) = Q ⇐⇒ (r(t) − mi ) Σi (r(t) − mi ) − Q = 0

Σi = Ri Si SiT RiT = Ri Si Si RiT , therefore:

Σ−1
i = Ri Si−1 Si−1 RiT

and

T
(r(t) − mi ) Ri Si−1 Si−1 RiT (r(t) − mi ) − Q = 0 ⇐⇒

T −1 T
⇐⇒ Si−1 RiT (r(t) − mi )


Si Ri (r(t) − mi ) − Q = 0 ⇐⇒

T −1 T
⇐⇒ Si−1 RiT (o + td − mi )


Si Ri (o + td − mi ) − Q = 0 ⇐⇒
  T   
⇐⇒ Si−1 RiT (o − mi ) + td − ⃗0 Si−1 RiT (o − mi ) + td − ⃗0 − Q = 0

Let us define o′ := Si−1 RiT (o − mi ) and d′ := Si−1 RiT d. Then:

  T   
Si−1 RiT (o − mi ) + td − ⃗0 Si−1 RiT (o − mi ) + td − ⃗0 − Q = 0 ⇐⇒
 T  
⇐⇒ o′ + td′ − ⃗0 o′ + td′ − ⃗0 − Q = 0

So we transformed the problem of the ray-Gaussian intersection into the problem of the ray-sphere intersection for the
sphere with the center at the origin of the coordinate system. Now:

 T  
o′ + td′ − ⃗0 o′ + td′ − ⃗0 − Q = 0 ⇐⇒
T
⇐⇒ (o′ + td′ ) (o′ + td′ ) − Q = 0 ⇐⇒
⇐⇒ ⟨o′ , o′ ⟩ + 2 ⟨o′ , d′ ⟩ (t) + ⟨d′ , d′ ⟩ t2 − Q = 0 ⇐⇒

⇐⇒ ⟨d′ , d′ ⟩ t2 + 2 ⟨o′ , d′ ⟩ (t) + (⟨o′ , o′ ⟩ − Q) = 0

Let us define: a := ⟨d′ , d′ ⟩, b := 2 ⟨o′ , d′ ⟩ and c := ⟨o′ , o′ ⟩ − Q. Since the classical way of computing the ∆ to solve the
above quadratic equation suffers from numerical stability problems, we will show, according to [Hearn and Baker], how to

11
 RaySplats: Ray Tracing based Gaussian Splatting

compute it in a more numerically robust way.

∆ = b2 − 4ac =
 2 
b
= 4a −c =
4a
!
′ ′ 2
′
= 4 ⟨d , d ⟩ ′ A⟨o , d ⟩ − (⟨o′ , o′ ⟩ − Q) =
4
A4 ⟨d′ , d′ ⟩
!
2
⟨o′ , d′ ⟩
= 4 ⟨d′ , d′ ⟩ ′ ′
− (⟨o , o ⟩ − Q) =
⟨d′ , d′ ⟩
!!
2
⟨o′ , d′ ⟩
= 4 ⟨d′ , d′ ⟩ ′
Q − ⟨o , o ⟩ − ′
=
⟨d′ , d′ ⟩
!!
1 ′ 2
= 4 ⟨d′ , d′ ⟩ ′
Q − ⟨o , o ⟩ − ′ ′
2 ⟨o , d ⟩ =
∥d′ ∥
 2 !!
1
= 4 ⟨d′ , d′ ⟩ ′
Q − ⟨o , o ⟩ − ′ ′
⟨o , d ⟩′
=
∥d′ ∥
2 !!
d′

= 4 ⟨d′ , d′ ⟩ ′ ′
Q − ⟨o , o ⟩ − o , ′ ′
∥d ∥

Using the Pythagorean theorem and some vector arithmetic, we finally get the following:
 2!
d′ d′

′ ′ ′ ′
∆ = 4 ⟨d , d ⟩ Q − o − ′ o, ′
∥d ∥ ∥d ∥

Hence:
√
−b ∓ ∆
t12 = =
2a s  
D E 2
′ ′ d′ d′
−2 ⟨o , d ⟩ ∓ 4 ⟨d′ , d′ ⟩ Q− o′ − ∥d′ ∥ o′ , ∥d′∥

= =
2 ⟨d′ , d′ ⟩
s  
D E 2
′ ′ ′ , d′ ⟩ Q − o′ − d′ d′
−2
A ⟨o , d ⟩ ∓ 2
A ⟨d ∥d′ ∥ o′ , ∥d′∥

= =
A ⟨d′ , d′ ⟩
2
s  
D E 2
d′ d′
− ⟨o′ , d′ ⟩ ∓ ⟨d′ , d′ ⟩ Q − o′ − ∥d′ ∥ o′ , ∥d′∥

=
⟨d′ , d′ ⟩
Since naive roots can be numerically unstable when
v !
u 2
d′ d′
u  
′ ′ ′ ′ ′ ′
− ⟨o , d ⟩ ≈ ⟨d , d ⟩ Q − o − ′
t o, ′ ,
∥d ∥ ∥d ∥

we obtain the desired t parameter value for the nearest intersection point using the Viete formula, in case it would lead to
numerical instability:
s  D E 2
′ ′ ′ ′ d′ ′ , d′
− ⟨o , d ⟩ − sgn (⟨o , d ⟩) ⟨d′ , d′ ⟩ Q − o′ − ∥d ′∥ o ∥d′ ∥
t∗ =
⟨d′ , d′ ⟩

12
 RaySplats: Ray Tracing based Gaussian Splatting

, ⟨o′ , d′ ⟩ ≥ 0
(
t∗
t1 = ⟨o′ ,o′ ⟩−Q
⟨d′ ,d′ ⟩t∗ , ⟨o′ , d′ ⟩ < 0

A.2. Computing αi Based on the Learned Per-Gaussian Opacity α̂i

n 3p
o n 1 T −1
o
αi = α̂i max (2π) 2 |Σi |fN (mi ,Σi ) (r(t)) = α̂i max e− 2 (r(t)−mi ) Σi (r(t)−mi )
t≥0 t≥0

Since ex is the increasing function, we obtain:

n 1 T −1
o max{− 12 (r(t)−mi )T Σ−1
i (r(t)−mi )}
max e− 2 (r(t)−mi ) Σi (r(t)−mi ) = e t≥0 =
t≥0
− min{ 21 (r(t)−mi )T Σ−1
i (r(t)−mi )}
=e t≥0
=
− 12 min {(r(t)−mi )T Σ−1
i (r(t)−mi )}
=e t≥0

Σi = Ri Si SiT RiT = Ri Si Si RiT , therefore:

Σ−1
i = Ri Si−1 Si−1 RiT

and

n o n o
T T
min (r(t) − mi ) Σ−1
i (r(t) − m i ) = min (r(t) − m i ) R S
i i
−1 −1 T
Si R i (r(t) − m i ) =
t≥0 t≥0
n
T −1 T
o
= min Si−1 RiT (r(t) − mi ) Si Ri (r(t) − mi ) =
t≥0
n
T −1 T
o
= min Si−1 RiT (o + td − mi ) Si Ri (o + td − mi ) =
t≥0
n
T −1 T
o
= min Si−1 RiT (td + (o − mi )) Si Ri (td + (o − mi ))
t≥0

Let us define: o′ := Si−1 RiT (o − mi ) and d′ := Si−1 RiT d. Then:

n o n
T −1 T
o
T
min (r(t) − mi ) Σ−1
i (r(t) − mi ) = min Si−1 RiT (td + (o − mi )) Si Ri (td + (o − mi )) =
t≥0 t≥0
n
T −1 T
o
= min Si−1 RiT (td′ + o′ ) Si Ri (td′ + o′ ) =
t≥0

= min {∥td′ + o′ ∥} =
t≥0

= min {∥td′ − o′ ∥} ,
t≥0

where the last equality holds because the distance between the line and the point in R3 is exactly the same as the distance
between the line and the symmetry of that point about the origin of the coordinate system. So we conclude that t minimizes
the Euclidean distance between td′ and o′ , and the exact minimum is the Euclidean distance between the line and the point
o′ , given by the following formula:

d′ d′
 
min {∥td′ − o′ ∥} = o′ − , o′
t≥0 ∥d′ ∥ ∥d′ ∥

13
 RaySplats: Ray Tracing based Gaussian Splatting

Thus:
n 1 T −1
o
αi = α̂i max e− 2 (r(t)−mi ) Σi (r(t)−mi ) =
t≥0
1
2 min {(r(t)−mi )T Σ−1
i (r(t)−mi )}
= α̂i · e t≥0
=
′ ′
− 12 min {∥td −o ∥}
= α̂i · e t≥0
=
 
′ ′ d′
− 12 o − d′ ,o′
∥d ∥ ∥d′ ∥
= α̂i · e

dL
A.3. Computing d□ for Some Trainable Parameter □.
Recall that since we are using the RGB color space, the loss function L will be the average of the loss functions computed
separately for the R, G, and B components of the successive pixels of both the rendered and the reference image:
1
L= (LR + LG + LB )
3
Thus, the gradient is given by the following formula:
 
dL 1 dLR dLG dLB
= + +
d□ 3 d□ d□ d□

Considering that each of the loss function components is of the form:

L∗ = (1 − λ)L∗2 + λL∗D-SSIM ,
dL∗
the gradient d□ can be calculated as follows:

dL∗ dL∗2 dL∗

= (1 − λ) + λ D-SSIM =
d□ d□  d□ 
h,w
1 − L∗SSIM
2  
d  1 X ∗ ˆ∗ d
= (1 − λ) I − Ii,j  +λ =
d□ w · h i,j=1 i,j d□ 2
 
h,w 2 ∗
1 X d 
 − λ · dLSSIM =
= (1 − λ)  ∗
Ii,j − Iˆi,j
∗
w · h i,j=1 d□ 2 d□
   
h,w  dI ∗ h,w ∗ ∗
1 X 
i,j λ X dL dIi,j
= (1 − λ)  ∗
2 Ii,j − Iˆi,j
∗ −  SSIM
∗ · =
w · h i,j=1 d□ 2 i,j=1 dIi,j d□
   
h,w  dI ∗ h,w ∗ ∗
X 2 (1 − λ) 
i,j
X λ dL dIi,j
= ∗
Ii,j − Iˆi,j
∗  − · SSIM 
∗ =
i,j=1
w·h d□ i,j=1
2 dIi,j d□
h,w
!
 λ dL∗ ∗
X 2 (1 − λ)  ∗ dIi,j
= Ii,j − Iˆi,j − ·
∗ SSIM
∗ ,
i,j=1
w·h 2 dIi,j d□

where h and w are the image height and width, respectively (they have the same value for the rendered and the reference
∗
image), while Ii,j and Iˆi,j
∗
are the color intensities at the point (i, j) for the rendered and the reference image, respectively.
For brevity in the later derivation, we will deliberately omit the indices: i and j in the color intensity formula, as well as the
asterisk to indicate that the formula will be identical for all color components: R, G and B. As:
 
X i−1
Y
I= ci αi  (1 − αj ) ,
i∈N j=1

14
 RaySplats: Ray Tracing based Gaussian Splatting

where N is the set of indices (Kopanas et al., 2021; 2022) of Gaussians that intersect the ray after reordering the set of
Gaussians so that the Gaussians with smaller indices are those hit "earlier" by the ray and the Gaussians with indices greater
than N are those having empty intersection, we obviously have:
 !
 i−1
Q



 αi (1 − αj ) , (□ = c) ∧ (i ∈ N )
dI  j=1
= 0 , (□ = c) ∧ (i ∈/ N)
d□i dI
· dαi

, (□ ̸= c) ∧ (i ∈ N )


 dαi d□i


0 , (□ ̸= c) ∧ (i ∈
/ N)

dI dαi dLSSIM
Now it remains to show how to compute dαi , d□i and dIi,j .

dI
A.4. Computing dα1 Using I.
Since
 
N
X i−1
Y
I= ci α i  (1 − αj ) ,
i=1 j=1

we have:
 
N
dI X  i−1
Y 
= c1 − ci α i  (1 − αj )
dα1 i=2

j=1

j̸=1

Let us multiply both sides of the equation by 1 − α1 . Then:

 
N i−1
dI X Y
(1 − α1 ) = c1 (1 − α1 ) − ci αi  (1 − αj ) =
dα1 i=2 j=1
  
XN i−1
Y
= c1 (1 − α1 ) − (c1 α1 − c1 α1 ) + ci αi  (1 − αj ) =
i=2 j=1
 
N
X i−1
Y
= c1 ((1 − α1 ) + α1 ) − ci αi  (1 − αj ) =
i=1 j=1

= c1 − I

Hence:
dI c1 − I
= ,
dα1 1 − α1

as long as α1 ̸= 1. If α1 = 1 we use the “regular” formula.

dI dI
A.5. Computing dαi Using the Previous Value of dαi−1 for i > 1.
Since:
   
i−1 N j−1
dI Y X Y
= ci  (1 − αj ) − cj α j  (1 − αk )
 
dαi j=1 j=i+1 k=1
k̸=i

15
 RaySplats: Ray Tracing based Gaussian Splatting

after multiplying both sides of the equation by 1 − αi we obtain:

 
j−1
i−1 N
!
dI Y X Y
(1 − αi ) = ci  (1 − αj ) (1 − αi ) −
 cj αj (1 − αk ) =
dαi j=1 j=i+1 k=1
   
j−1
i−1 i−1 N
!
Y Y X Y
= ci  (1 − αj ) (1 − αi ) + (ci αi − ci αi )  (1 − αj ) − cj αj (1 − αk ) =
j=1 j=1 j=i+1 k=1
 
j−1
i−1 N
!
Y X Y
= ci  (1 − αj ) ((1 − αi ) + αi ) − cj αj (1 − αk ) =
j=1 j=i k=1
 
j−1
i−1 N
!
Y X Y
= ci  (1 − αj ) − cj αj (1 − αk ) =
j=1 j=i k=1
   
j−1
i−1 i−1 N
!
Y Y X Y
= ci  (1 − αj ) + (ci−1 − ci−1 )  (1 − αj ) − cj αj (1 − αk ) =
j=1 j=1 j=i k=1
   
j−1
i−1 i−1 N
!
Y Y X Y
= (ci − ci−1 )  (1 − αj ) + ci−1  (1 − αj ) − cj αj (1 − αk ) =
j=1 j=1 j=i k=1
      
i−1
Y i−2
Y N
X j−1
Y
= (ci − ci−1 )  (1 − αj ) + (1 − αi−1 ) ci−1  (1 − αj ) − cj αj  (1 − αk ) =
  
j=1 j=1 j=i k=1
k̸=i−1
 
i−1
Y dI
= (ci − ci−1 )  (1 − αj ) + (1 − αi−1 )
j=1
dαi−1

Hence:

!
i−1
dI
Q
dαi−1 (1 − αi−1 ) + (ci − ci−1 ) (1 − αj )
dI j=1
=
dαi 1 − αi

i−1
dI
Q
as long as αi ̸= 1. If αi = 1 we use the “regular” formula. The quantities dαi−1 (1 − αi−1 ) and (1 − αj ) can easily be
j=1
accumulated in two separate variables and updated each time the ray encounters the new Gaussian in the backward phase.
Note, however, that the computational complexity is still linear in the case where αi = 1, since the Gaussians with indices
greater than i are not rendered because their transmittance Ti is 0.

dαi
A.6. Computing d□i .

Recall that:

 
d′ d′
1 − 21 o′ − ,o′
∥d′ ∥ ∥d′ ∥
αi = · e
1 + e−α̂i

16
 RaySplats: Ray Tracing based Gaussian Splatting

Therefore, we can immediately compute:

  !
′ d′
dαi d 1 − 12 o′ − d′ ,o′
∥d ∥ ∥d′ ∥
= ·e =
dα̂i dα̂i 1 + e−α̂i
 
  ′ d′
d 1 − 12 o′ − d′ ,o′
∥d ∥ ∥d′ ∥
= ·e =
dα̂i 1 + e−α̂i
 
d −α̂i

′ d′
dα̂i 1 + e − 12 o′ − d′
∥d ∥ ∥d′ ∥
,o′
=− 2 · e =
(1 + e−α̂i )
 
d′ d′
−e−α̂i − 12 o′ −
∥d′ ∥ ∥d′ ∥
,o′
=− 2 ·e =
(1 + e−α̂i )
 
d′ d′
e−α̂i − 12 o′ −
∥d′ ∥ ∥d′ ∥
,o′
= 2 ·e =
(1 + e−α̂i )
 
1 + e−α̂i − 1 d′ d′


− 12 o′ − ,o′
∥d′ ∥ ∥d′ ∥
= 2 ·e =
(1 + e−α̂i )
!  
′ d′
1 1 − 21 o′ − d′ ′ ,o′
∥d ∥ ∥d ∥
= − 2 ·e =
1 + e−α̂i (1 + e−α̂i )
 
  ′ d′
1 1 − 21 o′ − d′ ,o′
∥d ∥ ∥d′ ∥
= 1− ·e
1 + e−α̂i 1 + e−α̂i

As for the gradient value for the other trainable parameters □ ̸= α̂, we have:

  !
d′ d′
dαi d 1 − 12 o′ − ,o′
∥d′ ∥ ∥d′ ∥
= · e =
d□i d□i 1 + e−α̂i
  !
d′ d′
1 d − 12 o′ − ,o′
∥d′ ∥ ∥d′ ∥
= − α̂
· e =
1+e i d□i
 
d′ d′
o′ − ,o′ d′ d′
   
1 1 − 12 d
∥d′ ∥ ∥d′ ∥ ′
=− · · e · o − ′ , o′
2 1 + e−α̂i d□i ∥d ∥ ∥d′ ∥

Since:

d′
 ′ 
′ d ′
o − ′ ,o =
∥d ∥ ∥d′ ∥
d′ d′ d′
    ′ 
′ ′ ′ d ′
= o − ′ ,o ,o − ′ ,o =
∥d ∥ ∥d′ ∥ ∥d ∥ ∥d′ ∥
d′ d′ d′ d′ d′
       ′ 
′ ′ ′ ′ ′ d ′
= ⟨o , o ⟩ − 2 o , ′ ,o + ,o , ′ ,o =
∥d ∥ ∥d′ ∥ ∥d′ ∥ ∥d′ ∥ ∥d ∥ ∥d′ ∥
* + * +
d′ d′ d′
= ⟨o′ , o′ ⟩ − 2 o′ , 2 ⟨d′ , o′ ⟩ + 2 ⟨d′ , o′ ⟩ , ′ ′
2 ⟨d , o ⟩ =
∥d′ ∥ ∥d′ ∥ ∥d′ ∥
2 2
⟨d′ , o′ ⟩ ⟨d′ , o′ ⟩
= ⟨o′ , o′ ⟩ − 2 + =
⟨d′ , d′ ⟩ ⟨d′ , d′ ⟩
2
⟨d′ , o′ ⟩
= ⟨o′ , o′ ⟩ − ,
⟨d′ , d′ ⟩

17
 RaySplats: Ray Tracing based Gaussian Splatting

we have:

d′ d′
   
d
o′ − , o′ =
d□i ∥d′ ∥ ∥d′ ∥
!
2
d ′ ⟨d′ , o′ ⟩
′
= ⟨o , o ⟩ − =
d□i ⟨d′ , d′ ⟩
!
2
d ′ ′ d ⟨d′ , o′ ⟩
= (⟨o , o ⟩) − =
d□i d□i ⟨d′ , d′ ⟩
 
′ ′ 2 2

do ′
 d
d□ i
⟨d , o ⟩ · ⟨d′ , d′ ⟩ − ⟨d′ , o′ ⟩ · d
d□i (⟨d′ , d′ ⟩)
′
=2 o, − 2 =
d□i ⟨d′ , d′ ⟩
D E
2 dd′

do ′
 2 · ⟨d′ , d′ ⟩ · ⟨d′ , o′ ⟩ · d
d□i (⟨d′ , o′ ⟩) − 2 · ⟨d′ , o′ ⟩ · d′ , d□i
= 2 o′ , − 2 =
d□i ⟨d′ , d′ ⟩
D E D E D E
dd′ ′ do′ ′ 2 ′ dd′

do′
 2 · ⟨d′ , d′ ⟩ · ⟨d′ , o′ ⟩ o′ , d□ i
+ d , d□i − 2 · ⟨d′
, o ⟩ · d , d□i
= 2 o′ , − 2 =
d□i ⟨d′ , d′ ⟩
′
⟨d′ , o′ ⟩ ′
⟨d′ , o′ ⟩ ′
⟨d′ , o′ ⟩ ⟨d′ , o′ ⟩ ′
       
′ do ′ do ′ dd ′ dd
=2 o, −2· ′ ′ · d, −2· ′ ′ · o, +2· ′ ′ · ′ ′ · d, =
d□i ⟨d , d ⟩ d□i ⟨d , d ⟩ d□i ⟨d , d ⟩ ⟨d , d ⟩ d□i
⟨d′ , o′ ⟩ do′ ⟨d′ , o′ ⟩ ⟨d′ , o′ ⟩ dd′
   
= 2 o′ − d′ ′ ′ , − 2 · ′ ′ · o′ − d′ ′ ′ , =
⟨d , d ⟩ d□i ⟨d , d ⟩ ⟨d , d ⟩ d□i
* + * +
′ d′ ′ ′ do′ ⟨d′ , o′ ⟩ ′ d′ ′ ′ dd′
=2 o − 2 ⟨d , o ⟩ , d□ −2· ′ ′ · o − 2 ⟨d , o ⟩ , d□ =
∥d′ ∥ i ⟨d , d ⟩ ∥d′ ∥ i

d′
 ′
do′ ⟨d′ , o′ ⟩ d′
 ′
dd′
     
d d
= 2 o′ − ′ , o ′
, − 2 · · o ′
− , o ′
, =
∥d ∥ ∥d′ ∥ d□i ⟨d′ , d′ ⟩ ∥d′ ∥ ∥d′ ∥ d□i
d′
 ′
do′ ⟨d′ , o′ ⟩ d′
 ′
dd′
     
d d
=2 o′ − ′ , o ′
, − · · o′
− , o ′
,
∥d ∥ ∥d′ ∥ d□i ⟨d′ , d′ ⟩ ∥d′ ∥ ∥d′ ∥ d□i

′ ′
Now it remains to show how to compute d□ do
i
dd
and d□ i
. By definition: o′ = Si−1 RiT (o − mi ) and d′ = Si−1 RiT d. As for
the gradient for the trainable parameter mi,x , we have:

do′ d d dmi
Si−1 RiT (o − mi ) = Si−1 RiT (o − mi ) = −Si−1 RiT = −Si−1 RiT εx


=
dmi,x dmi,x dmi,x dmi,x

dd′ d
Si−1 RiT d = ⃗0,


=
dmi,x dmi,x

18
 RaySplats: Ray Tracing based Gaussian Splatting

where εx is the vector of the canonical basis corresponding to the OX axis of the coordinate system. On the other hand for
si,x we obtain:

do′ d
Si−1 RiT (o − mi ) =


=
dsi,x dsi,x
dSi−1 T
= R (o − mi ) =
dsi,x i
 1
−1 
−si,x 0 0
d  1+e 1   T
= 0 1+e−si,y
0   Ri (o − mi ) =
dsi,x

1
0 0 1+e−si,z
−si,x
 
1+e 0 0
d 
= 0 1 + e−si,y 0  RiT (o − mi ) =
dsi,x −si,z
0 0 1+e
−si,x
 d 
dsi,x (1 + e ) 0 0
d −si,y  T
=
 0 dsi,x (1 + e ) 0  Ri (o − mi ) =
d −si,z
0 0 dsi,x (1 + e )
 −s 
−e i,x
0 0
= 0 0 0 RiT (o − mi ) =
0 0 0
= −e−si,x εy εTy RiT (o − mi ) .

Using exactly the same reasoning, we conclude that:

dd′
= −e−si,x εy εTy RiT d
dsi,x

The derivation for the y and z components of the above parameters is analogous. Finally, the parameter qi =
(qi,r , qi,i , qi,j , qi,k ) (note the notation conflict: we use i both as the index and as the subscript of the quaternion to in-
dicate that it is the first imaginary component of the quaternion):

do′ d dRiT
Si−1 RiT (o − mi ) = Si−1


= (o − mi )
dqi,□ dqi,□ dqi,□

dd′ d dRiT
Si−1 RiT d = Si−1


= d.
dqi,□ dqi,□ dqi,□
dRT
To complete our derivation, we need to provide the formula for dq i . Let’s recall that for the quaternion qi =
i,□
(qi,r , qi,i , qi,j , qi,k ) which parametrizes the rotation matrix Ri , the rotation matrix Ri itself can be obtained from qi
using the following formula:
 
1 − cc − dd bc − ad bd + ac
R =  bc + ad 1 − bb − dd cd − ab 
bd − ac cd + ab 1 − bb − cc
where:
2
s= 2 2 2 + q2
qi,r + qi,i + qi,j i,k

bs = qi,i · s cs = qi,j · s ds = qi,k · s

ab = qi,r · bs ac = qi,r · cs ad = qi,r · ds
bb = qi,i · bs bc = qi,i · cs bd = qi,i · ds
cc = qi,j · cs cd = qi,j · ds dd = qi,k · ds

19
 RaySplats: Ray Tracing based Gaussian Splatting

2
Let us define aa := qi,r s, a := qi,r , b := qi,i , c := qi,j , and d := qi,k . Then, after some computation, we finally obtain:
 
T sa ((1 − aa) + (1 − bb)) s (−d (1 − aa) − ab · c) s (c (1 − aa) − ab · d)
dRi
=  s (d (1 − aa) − ab · c) sa ((1 − aa) + (1 − cc)) s (−b (1 − aa) − a · cd) 
dqi,r
s (−c (1 − aa) − ab · d) s (b (1 − aa) − a · cd) sa ((1 − aa) + (1 − dd))
 
sb ((1 − bb) + (1 − aa)) s (c (1 − bb) − (−ab · d)) s (d (1 − bb) − ab · c)
dRiT
=  s (c (1 − bb) − ab · d) −sb ((1 − bb) + (1 − dd)) s (−a (1 − bb) − b · cd) 
dqi,i
s (d (1 − bb) − (−ab · c)) s (a (1 − bb) − b · cd) −sb ((1 − bb) + (1 − cc))
 
−sc ((1 − cc) + (1 − dd)) s (b (1 − cc) − (−a · cd)) s (a (1 − cc) − b · cd)
dRiT
=  s (b (1 − cc) − a · cd) sc ((1 − cc) + (1 − aa)) s (d (1 − cc) − (−ab · c)) 
dqi,j
s (−a (1 − cc) − b · cd) s (d (1 − cc) − ab · c) −sc ((1 − cc) + (1 − bb))
 
T −sd ((1 − dd) + (1 − cc)) s (−a (1 − dd) − b · cd) s (b (1 − dd) − a · cd)
dRi
=  s (a (1 − dd) − b · cd) −sd ((1 − dd) + (1 − bb)) s (c (1 − dd) − (−ab · d))
dqi,k
s (b (1 − dd) − (−a · cd)) s (c (1 − dd) − ab · d) sd ((1 − dd) + (1 − aa))

dLSSIM
A.7. Computing dIi,j .
Recall that:
h,w
1 X  
LSSIM = SSIM Ik,l , Iˆk,l =
h·w
k,l=1
  
1
h,w
X 2 · µI,k,l · µ ˆ
I,k,l + c1 2 · σ I,k,l, ˆ
I,k,l + c2
=   
h·w µ2 + µ2 + c σ2 + σ2 + c
k,l=1 I,k,l ˆ
I,k,l 1 I,k,l ˆ
I,k,l 2

2 2
where c1 = (k1 L) = k12 and c2 = (k2 L) = k22 for k1 = 0.01 and k2 = 0.03, because we assume that the maximum
allowed color intensity is 1. Let us define:

Ak,l := 2 · µI,k,l · µI,k,l

ˆ + c1

Bk,l := 2 · σI,k,l,I,k,l
ˆ + c2
Ck,l := µ2I,k,l + µ2I,k,l
ˆ + c1
2 2
Dk,l := σI,k,l + σI,k,l
ˆ + c2
Then:
   
dLSSIM d  1 2 ·h,w
Xµ I,k,l · µ ˆ
I,k,l + c1 2 · σ ˆ
I,k,l,I,k,l + c 2
=   =
dIi,j dIi,j h · w µ 2 + µ 2 + c σ 2 + σ 2 + c
k,l=1 I,k,l ˆ
I,k,l 1 I,k,l ˆ
I,k,l 2
 
h,w
d  1 X Ak,l · Bk,l 
= =
dIi,j h · w Ck,l · Dk,l
k,l=1
h,w  
1 X d Ak,l · Bk,l
= =
h·w dIi,j Ck,l · Dk,l
k,l=1
h,w d d
1 X dIi,j (Ak,l · Bk,l ) · Ck,l · Dk,l − Ak,l · Bk,l · dIi,j (Ck,l · Dk,l )
= 2 2 =
h·w Ck,l · Dk,l
k,l=1
   
dAk,l dBk,l dCk,l dDk,l
1
h,w
X dIi,j · Bk,l + Ak,l · dIi,j · Ck,l · Dk,l − Ak,l · Bk,l · dIi,j · Dk,l + Ck,l · dIi,j
= 2 · D2
h·w Ck,l k,l
k,l=1

20
 RaySplats: Ray Tracing based Gaussian Splatting

By definition:
r
X
µI,k,l = wm,n Ik+m,l+n
m,n=−r

r
X
2 2
σI,k,l = wm,n (Ik+m,l+n − µI,k,l ) =
m,n=−r
X r
2
− 2 · Ik+m,l+n · µI,k,l + µ2I,k,l =


= wm,n Ik+m,l+n
m,n=−r
r
! r
! r
!
X X X
2
= wm,n · Ik+m,l+n − 2 · µI,k,l wm,n · Ik+m,l+n + µ2I,k,l wm,n =
m,n=−r m,n=−r m,n=−r
r
!
X
2
= wm,n · Ik+m,l+n − 2 · µI,k,l · µI,k,l + µ2I,k,l · 1 =
m,n=−r
r
!
X
2
= wm,n · Ik+m,l+n − 2 · µ2I,k,l + µ2I,k,l =
m,n=−r
r
!
X
2
= wm,n · Ik+m,l+n − µ2I,k,l
m,n=−r

σI,k,l,I,k,l
ˆ =

r
X  
= wm,n (Ik+m,l+n − µI,k,l ) Iˆk+m,l+n − µI,k,l
ˆ =
m,n=−r
X r  
= wm,n Ik+m,l+n · Iˆk+m,l+n − µI,k,l
ˆ · Ik+m,l+n − µI,k,l · Iˆk+m,l+n + µI,k,l · µI,k,l
ˆ =
m,n=−r

r
!
X
= wm,n · Ik+m,l+n · Iˆk+m,l+n −
m,n=−r
r
! r
!
X X
− µI,k,l
ˆ wm,n · Ik+m,l+n − µI,k,l wm,n · Iˆk+m,l+n +
m,n=−r m,n=−r
r
!
X
+ µI,k,l · µI,k,l
ˆ wm,n =
m,n=−r

r
!
X
= wm,n · Ik+m,l+n · Iˆk+m,l+n − µI,k,l
ˆ · µI,k,l − µI,k,l · µI,k,l
ˆ + µI,k,l · µI,k,l
ˆ ·1=
m,n=−r
r
!
X
= wm,n · Ik+m,l+n · Iˆk+m,l+n − µI,k,l · µI,k,l
ˆ − µI,k,l · µI,k,l
ˆ + µI,k,l · µI,k,l
ˆ =
m,n=−r
r
!
X
= wm,n · Ik+m,l+n · Iˆk+m,l+n − µI,k,l · µI,k,l
ˆ
m,n=−r

21
 RaySplats: Ray Tracing based Gaussian Splatting

where wm,n is the Gaussian window with radius r summing up to 1 defined for indices: m, n ∈ {−r, . . . , r} × {−r, . . . , r}
and equal to 0 otherwise. Hence:
r
! r r
dµI,k,l d X X d X
= wm,n · Ik+m,l+n = (wm,n · Ik+m,l+n ) = wm,n · δi,k+m · δj,l+n = wi−k,j−l
dIi,j dIi,j m,n=−r m,n=−r
dIi,j m,n=−r

r
! !
2
dσI,k,l d X
2
= wm,n · Ik+m,l+n − µ2I,k,l =
dIi,j dIi,j m,n=−r
r
!
X d 2

dµ2I,k,l
= wm,n · Ik+m,l+n − =
m,n=−r
dIi,j dIi,j
r
!
X dµI,k,l
= 2 · wm,n · Ii,j · δi,k+m · δj,l+n − 2 · µI,k,l · =
m,n=−r
dIi,j
= 2 · wi−k,j−l · Ii,j − 2 · wi−k,j−l · µI,k,l

r
! !
dσI,k,l,I,k,l
ˆ d X
= wm,n · Ik+m,l+n · Iˆk+m,l+n − µI,k,l · µI,k,l
ˆ =
dIi,j dIi,j m,n=−r
r
!
X d  ˆ
 d  
= wm,n · Ik+m,l+n · Ik+m,l+n − µI,k,l · µI,k,l
ˆ =
m,n=−r
dIi,j dIi,j
r
!
X
ˆ dµI,k,l
= wm,n · Ik+m,l+n · δi,k+m · δj,l+n − µI,k,l
ˆ · =
m,n=−r
dIi,j

= wi−k,j−l · Iˆi,j − wi−k,j−l · µI,k,l

ˆ

dLSSIM
Now we can calculate each of the two summands that appear in the numerator of the fraction in the formula for dIi,j :
 
dAk,l dBk,l
· Bk,l + Ak,l · · Ck,l · Dk,l =
dIi,j dIi,j
 
d   d 
= 2 · µI,k,l · µI,k,l
ˆ + c1 · Bk,l + Ak,l · 2 · σI,k,l,I,k,l
ˆ + c2 · Ck,l · Dk,l =
dIi,j dIi,j
dσI,k,l,I,k,l
 
dµI,k,l ˆ
= 2 · µI,k,l
ˆ · Bk,l · + 2 · Ak,l · · Ck,l · Dk,l =
dIi,j dIi,j
  
= 2 · µI,k,l
ˆ · Bk,l · wi−k,j−l + 2 · Ak,l · wi−k,j−l · Iˆi,j − wi−k,j−l · µI,k,l ˆ · Ck,l · Dk,l =
 
= 2 · Ck,l · Dk,l · µI,k,l
ˆ (Bk,l − Ak,l ) wi−k,j−l + Iˆi,j ((2 · Ak,l · Ck,l · Dk,l ) wi−k,j−l )

and
 
dCk,l dDk,l
Ak,l · Bk,l · · Dk,l + Ck,l · =
dIi,j dIi,j
 
d  2  d  2
= Ak,l · Bk,l · µI,k,l + µ2I,k,l
ˆ + c1 · D k,l + Ck,l · σI,k,l + σ 2
ˆ
I,k,l
+ c2 =
dIi,j dIi,j
 
dµI,k,l
= Ak,l · Bk,l 2 · µI,k,l · Dk,l · + Ck,l (2 · wi−k,j−l · Ii,j − 2 · wi−k,j−l · µI,k,l ) =
dIi,j
= Ak,l · Bk,l (2 · µI,k,l · Dk,l · wi−k,j−l + Ck,l (2 · wi−k,j−l · Ii,j − 2 · wi−k,j−l · µI,k,l )) =
= (2 · Ak,l · Bk,l · µI,k,l (Dk,l − Ck,l )) wi−k,j−l + Ii,j ((2 · Ak,l · Bk,l · Ck,l ) wi−k,j−l )

22
 RaySplats: Ray Tracing based Gaussian Splatting

Let’s define:
2  
Ek,l := 2 · D2 Ck,l · Dk,l · µ ˆ
I,k,l (B k,l − A k,l ) − A k,l · B k,l · µI,k,l (Dk,l − C k,l )
Ck,l k,l

2
Fk,l := 2 · D 2 · Ak,l · Ck,l · Dk,l
Ck,l k,l

2
Gk,l := 2 · D 2 · Ak,l · Bk,l · Ck,l
Ck,l k,l
dLSSIM
Substituting the above-mentioned quantities into the formula for dIi,j yields:

dLSSIM
=
dIi,j
   
dAk,l dBk,l dCk,l dDk,l
1
h,w
X dIi,j · Bk,l + Ak,l · dIi,j · Ck,l · Dk,l − Ak,l · Bk,l · dIi,j · Dk,l + Ck,l · dIi,j
= 2 · D2 =
h·w Ck,l k,l
k,l=1
h,w
1 X  
= Ek,l · wi−k,j−l + Iˆi,j · Fk,l · wi−k,j−l − Ii,j · Gk,l · wi−k,j−l =
h·w
k,l=1
     
h,w h,w h,w
1  X X X
= Ek,l · wi−k,j−l  + Iˆi,j  Fk,l · wi−k,j−l  − Ii,j  Gk,l · wi−k,j−l  =
h·w
k,l=1 k,l=1 k,l=1
1  
= (E ∗ w)i,j + Iˆi,j (F ∗ w)i,j − Ii,j (G ∗ w)i,j
h·w
dLSSIM
In summary, we have expressed the formula for dIi,j as a weighted sum of three convolutions, which can be computed
efficiently using the Fast Fourier Transform.

23