【gaussian-splatting】用自己的数据复现高斯泼溅(二)

1.环境准备

1.1.下载diff-gaussian-rasterization

        这里本来没啥说的,直接从github上下载就行了,但是我踩坑了,下的版本不对,后续运行报错参数个数对不上,特在此给大家避坑,注意一定要下带3dgs版本的diff-gaussian-rasterization,否则后续会报错。

或者直接用我贴的下载地址:

https://github.com/graphdeco-inria/diff-gaussian-rasterization/tree/3dgs_accel

不要直接下载main版本的!不要直接下载main版本的!不要直接下载main版本的!

否则就会跟我一样报错:

Number of points at initialisation :  107 [06/08 17:53:37]
Training progress:   0%|                                                                                                                                                                | 0/30000 [00:00<?, ?it/s]Traceback (most recent call last):
  File "/mnt/geovis/zhouy/001_project/gaussian-splatting-main/train.py", line 282, in <module>
    training(lp.extract(args), op.extract(args), pp.extract(args), args.test_iterations, args.save_iterations, args.checkpoint_iterations, args.start_checkpoint, args.debug_from)
  File "/mnt/geovis/zhouy/001_project/gaussian-splatting-main/train.py", line 111, in training
    render_pkg = render(viewpoint_cam, gaussians, pipe, bg, use_trained_exp=dataset.train_test_exp, separate_sh=SPARSE_ADAM_AVAILABLE)
  File "/mnt/geovis/zhouy/001_project/gaussian-splatting-main/gaussian_renderer/__init__.py", line 36, in render
    raster_settings = GaussianRasterizationSettings(
TypeError: GaussianRasterizationSettings.__new__() got an unexpected keyword argument 'antialiasing'
Traceback (most recent call last):
  File "/mnt/geovis/zhouy/001_project/gaussian-splatting-main/train.py", line 282, in <module>
    training(lp.extract(args), op.extract(args), pp.extract(args), args.test_iterations, args.save_iterations, args.checkpoint_iterations, args.start_checkpoint, args.debug_from)
  File "/mnt/geovis/zhouy/001_project/gaussian-splatting-main/train.py", line 111, in training
    render_pkg = render(viewpoint_cam, gaussians, pipe, bg, use_trained_exp=dataset.train_test_exp, separate_sh=SPARSE_ADAM_AVAILABLE)
  File "/mnt/geovis/zhouy/001_project/gaussian-splatting-main/gaussian_renderer/__init__.py", line 102, in render
    rendered_image, radii, depth_image = rasterizer(
ValueError: not enough values to unpack (expected 3, got 2)
Training progress:   0%|

1.2.下载glm

这个没啥特殊注意,直接在github上下载就行

命令下载:

git clone https://github.com/icaven/glm.git

下载完成后将其放在./gaussian-splatting-main/submodules/diff-gaussian-rasterization/third_party/目录下,文件夹名称为glm,如下图

1.3.安装diff-gaussian-rasterization

cd 到diff-gaussian-rasterization目录下,运行:

python setup.py install

正常安装的话应该会如下图所示,有一些warning,不用管它。直至出现Installed /root/anaconda3/envs/gaussian/lib/python3.10/site-packages/diff_gaussian_rasterization-0.0.0-py3.10-linux-x86_64.egg
Processing dependencies for diff-gaussian-rasterization==0.0.0
Finished processing dependencies for diff-gaussian-rasterization==0.0.0

1.4.安装simple-knn

安装没啥注意的,建议都下载下来离线包,再上传上服务器解压安装。

下载地址:

https://github.com/camenduru/simple-knn

将下载的压缩包上传至./gaussian-splatting-main/submodules/下,解压,运行以下命令即可完成安装:

git clone https://github.com/camenduru/simple-knn && cd simple-knn
pip install .

1.5.安装fused-ssim

        同上,先下载离线文件,上传服务器,解压

下载地址:

https://github.com/rahul-goel/fused-ssim

cd到./gaussian-splatting-main/submodules/fused-ssim目录下,运行以下代码实现安装:

pip install . --no-build-isolation

2.训练过程

(gaussian) [root@t97 gaussian-splatting-main]# python train.py -s ./data -m ./output/mydata
Optimizing ./output/mydata
Output folder: ./output/mydata [06/08 18:21:32]
Tensorboard not available: not logging progress [06/08 18:21:32]
Reading camera 4/4 [06/08 18:21:32]
Loading Training Cameras [06/08 18:21:32]
Loading Test Cameras [06/08 18:21:32]
Number of points at initialisation :  107 [06/08 18:21:32]
Training progress:  23%|??????????????????????????                                                                                    | 6990/30000 [00:34<02:03, 186.02it/s, Loss=0.0057908, Depth Loss=0.0000000]
[ITER 7000] Evaluating train: L1 0.004467152804136276 PSNR 40.17698516845704 [06/08 18:22:07]

[ITER 7000] Saving Gaussians [06/08 18:22:07]
Training progress: 100%|?????????????????????????????????????????????????????????????????????????????????????????????????????????????| 30000/30000 [02:59<00:00, 167.57it/s, Loss=0.0038100, Depth Loss=0.0000000]

[ITER 30000] Evaluating train: L1 0.002960224240086973 PSNR 44.79262390136719 [06/08 18:24:31]

[ITER 30000] Saving Gaussians [06/08 18:24:31]

Training complete. [06/08 18:24:33]

训练完成会出现

Training complete. [xxxxxxx]

3.输出结果查看

输出的结果保存在./gaussian-splatting-main/output/mydata/point_cloud/iteration_30000/point_cloud.ply

将其下载到本地,用本地软件,如blender打开,或者在浏览器中,在线打开。

在线查看地址:

https://superspl.at/editor

### 2D Gaussian Splatting 实现详解 #### 背景介绍 2D Gaussian Splatting 是一种利用高斯分布来表示图像或场景的技术。它能够高效地处理大规模数据集并提供高质量的重建效果。这种方法的核心在于使用高斯面元(Gaussian surfels),它们可以存储位置、尺度、旋转、颜色、不透明度以及语义信息。 --- #### 数据准备与环境配置 在开始复现之前,需确保开发环境已正确设置。推荐使用的框架包括 PyTorch 或 TensorFlow,并安装必要的依赖库。以下是基本的 Python 环境配置命令: ```bash pip install torch torchvision numpy matplotlib opencv-python scikit-image ``` 如果需要 GPU 加速支持,则还需确认 CUDA 和 cuDNN 驱动版本兼容性[^4]。 --- #### 核心算法流程解析 ##### 1. 初始化阶段 对于每个高斯面元,其初始参数可以通过输入轨迹或其他先验条件设定。具体而言: - **位置**:通常由点云坐标决定; - **尺度**:定义为标准差 σ 的大小; - **旋转角度**:用于调整椭圆方向; - **颜色/不透明度**:从 RGB 值映射而来; 上述过程可通过如下伪代码描述: ```python import numpy as np def initialize_gaussians(trajectory_points, sigma=0.1): """ Initialize a set of 2D Gaussians from trajectory points. Args: trajectory_points (np.ndarray): Array of shape (N, 2), representing the xy coordinates. sigma (float): Standard deviation for all gaussians. Returns: dict: Dictionary containing initialized parameters. """ N = len(trajectory_points) params = { "position": trajectory_points, "scale": np.full((N,), sigma), "rotation": np.zeros(N), "color": np.random.rand(N, 3) # Random colors for visualization purposes } return params ``` ##### 2. 渲染模块设计 渲染目标是将所有高斯面元投影到屏幕空间上形成最终图像。此操作涉及积分计算以模拟光照效应。假设像素值 P(x,y) 受多个重叠的高斯影响,则有公式表达式[^3]: \[ P(x, y) = \sum_{i} w_i G(x - μ_x^{(i)}, y - μ_y^{(i)}; Σ^{(i)}) \] 其中 \(μ\) 表示均值向量而Σ代表协方差矩阵形式化写法。 实际编码时建议调用现有工具包简化复杂运算逻辑比如 `diff-gaussian-rasterization` 提供的功能接口[^1]: ```python from diff_gaussian_rasterization import rasterize_gaussians # Example usage with dummy data positions = np.array([[0., 0.], [1., 1.]]) scales = np.array([[[0.1, 0.], [0., 0.1]], [[0.1, 0.], [0., 0.1]]]) # Covariance matrices rotations = np.radians(np.array([0., 45.])) colors = np.ones((2, 3)) image_size = (256, 256) rendered_image = rasterize_gaussians( positions.astype('f'), scales.astype('f'), rotations.astype('f'), colors.astype('f'), image_size=image_size ).cpu().numpy() ``` ##### 3. 参数优化策略 采用梯度下降方法更新模型权重直至收敛为止。损失函数 L 定义取决于应用场景可能包含但不限于以下几种成分: - 图像相似性误差 MSE(L2 norm); - 边缘锐利程度约束 TV(total variation regularization); - 如果存在额外传感器读数则加入相应项如 LiDAR height supervision; 完整训练脚本片段展示如下所示: ```python import torch.optim as optim class LossFunction(torch.nn.Module): def __init__(self): super().__init__() def forward(self, rendered_img, target_img): l_mse = ((rendered_img - target_img)**2).mean() # Mean Squared Error grad_x = rendered_img[:, :, :-1] - rendered_img[:, :, 1:] grad_y = rendered_img[:, :-1, :] - rendered_img[:, 1:, :] tv_loss = torch.abs(grad_x).mean() + torch.abs(grad_y).mean() total_loss = l_mse + 0.01 * tv_loss return total_loss optimizer = optim.Adam(gaussian_params.values(), lr=0.01) loss_fn = LossFunction() for epoch in range(num_epochs): optimizer.zero_grad() output = render_function(**gaussian_params) loss_value = loss_fn(output, ground_truth) loss_value.backward() optimizer.step() ``` --- #### 注意事项 - 显存管理至关重要尤其是在大批量实例情况下考虑分批加载减少内存占用情况发生; - 初始猜测质量直接影响后续迭代速度因此尽可能依据领域知识选取合理默认值; - 不同硬件平台间性能差异较大测试期间注意监控资源利用率防止过载崩溃现象产生. ---
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

zy_destiny

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值