1. Opacity $\alpha$

Role:

• Opacity controls the contribution of each point or splat to the final rendered image. In 3DGS, each point is treated as a Gaussian blob, and its opacity determines how “transparent” or “solid” that blob appears when composited onto the final image.

Mathematics:

• Typically, opacity is represented by ${\alpha }^{(i)}\in [0,1]$, where $i$ is the index of the point. A value of 1 means the point is fully opaque, while 0 means the point is completely transparent.
• Opacity is often used in alpha compositing when rendering the point cloud. For example, in front-to-back compositing, the contribution of each point to the final pixel color is modulated by its opacity:
  $$C_{\text{final}}=\sum _{i=1}^N{\alpha }^{(i)}{C}_i\prod _{j<i}(1-{\alpha }^{(j)})$$
  where $C_{\text{final}}$ is the final color, $C_i$ is the color of the $i$-th point, and ${\alpha }^{(i)}$ is the opacity of the (i)-th point.

Purpose in Rendering:

• Opacity helps to achieve smooth transitions between points and can prevent harsh, blocky artifacts when points overlap. It also allows for rendering semi-transparent surfaces, such as glass or translucent materials.

2. Means $\mu$

Role:

• The means are the 3D coordinates of the center of each Gaussian. They define where each point is positioned in the 3D space. This feature is fundamental for spatial organization and for correctly placing the point cloud in the scene.

Mathematics:

• Each point $i$ in the point cloud is associated with a 3D mean vector ${\mu }^{(i)}=(x_i,y_i,z_i)$. This represents the center of the Gaussian distribution in 3D space:
  $$f(\mathbf{x})=\exp \left(-\frac{1}{2}(\mathbf{x}-{\mu }^{(i)}{)}^{\top }{\Sigma }^{-1}(\mathbf{x}-{\mu }^{(i)})\right)$$
  where $\mathbf{x}=(x,y,z)$ is a point in space, and $\Sigma$ is the covariance matrix that defines the shape and orientation of the Gaussian.

Purpose in Rendering:

• The means define the spatial layout of the points. In 3D reconstruction, the correct placement of these means ensures that the shape of the object or scene is accurately represented.

3. Feature dc (Direct Component)

Role:

• The direct component (dc) represents the base color or albedo of a point. It captures the color information of the point under direct illumination, without accounting for complex effects like reflections or shadows.

Mathematics:

• The direct component can be seen as a vector ${\mathbf{c}}_i\in {\mathbb{R}}^3$ (for RGB color) or ${\mathbb{R}}^4$ (for RGBA color), where each component represents the red, green, and blue (and possibly alpha) channels of the point’s color.

  If combined with opacity, the final contribution of point $i$ to the rendered image could be:
  $$C_{\text{final}}={\alpha }^{(i)}{\mathbf{c}}_i$$
  where ${\mathbf{c}}_i$ is the direct color of the point.

Purpose in Rendering:

• The direct component provides the fundamental color information for the point. It is often the result of texture mapping or material properties and is essential for determining the final appearance of the point or splat.

4. Feature rest (Residual Component)

Role:

• The residual component (rest) captures additional lighting or shading information that is not part of the direct component. This can include effects like ambient occlusion, indirect lighting, or high-frequency details that improve the realism of the rendered scene.

Mathematics:

• Mathematically, this is an additional feature vector that adds more detail to the point’s color, beyond what is captured by the direct component. It could be represented as ${\mathbf{r}}_i\in {\mathbb{R}}^3$ or ${\mathbb{R}}^4$, similar to ${\mathbf{c}}_i$, and it would be combined with the direct component:
  $${\mathbf{c}}_{\text{final}}={\mathbf{c}}_i+{\mathbf{r}}_i$$
• The residual component could also represent non-color information, such as higher-order lighting effects, which are added on top of the direct lighting.

Purpose in Rendering:

• This feature is crucial for adding realism to the render. It accounts for effects like soft shadows, indirect lighting, and other complex lighting phenomena that are not captured by the direct color component alone.

5. Scales $\mathbf{s}$

Role:

• The scales control the size and spread of the Gaussian for each point. They define how large or small the Gaussian blob is and how much area it covers in 3D space.

Mathematics:

• The scale of a Gaussian distribution determines the spread of the point in space. In the isotropic case, each point has a scalar scale $s_i$, and the Gaussian distribution is spherical. In the anisotropic case, the scale is a vector ${\mathbf{s}}_i=(s_x,s_y,s_z)$, which represents different spreads in each dimension.

  The general form of the Gaussian function would be:

$$f(\mathbf{x})=\exp \left(-\frac{1}{2}(\mathbf{x}-{\mu }_i{)}^{\top }{\Sigma }_i^{-1}(\mathbf{x}-{\mu }_i)\right)$$

where ${\Sigma }_i$ is the covariance matrix, which can be defined in terms of the scales. For an anisotropic Gaussian:

$${\Sigma }_i=\text{diag}(s_x^2,s_y^2,s_z^2)$$

In isotropic cases, ${\Sigma }_i=s_i^{2}I$, where $I$ is the identity matrix.

Purpose in Rendering:

• The scales determine the smoothness or sharpness of the point cloud. Larger scales produce smoother transitions between points, while smaller scales produce sharper details. This affects how well the point cloud approximates the surface of the object being represented.

6. Quaternions (quats) $\mathbf{q}$

Role:

• Quaternions represent the orientation of each Gaussian in 3D space. When the Gaussian is anisotropic, its orientation must be specified to ensure it aligns with the surface or direction of the point cloud correctly.

Mathematics:

• A quaternion ${\mathbf{q}}_i=(w_i,x_i,y_i,z_i)$ is a four-dimensional vector used to represent rotation in 3D space. Quaternions avoid the gimbal lock problem that can occur with Euler angles and provide smooth interpolations for rotations (known as slerp or spherical linear interpolation).

  To rotate a point $\mathbf{v}=(x,y,z)$ by quaternion $\mathbf{q}$, the operation is:
  $${\mathbf{v}}_{\text{rotated}}=\mathbf{qv}{\mathbf{q}}^{-1}$$

  where ${\mathbf{q}}^{-1}$ is the inverse of the quaternion $\mathbf{q}$.

Purpose in Rendering:

• Quaternions are crucial when working with anisotropic Gaussians, where the Gaussian blobs must be rotated to align with the surface or the features of the object. Proper orientation ensures that the splats align with the surface normals or other directional properties of the point cloud.

Summary of Features with Mathematical Aspects: