PointCNN xconv变换矩阵部分代码的解析

最新推荐文章于 2023-09-25 01:00:11 发布

L793404933

最新推荐文章于 2023-09-25 01:00:11 发布

阅读量789

点赞数 2

CC 4.0 BY-SA版权

分类专栏： 3D图像文章标签： PointCNN Point Cloud

本文链接：https://blog.csdn.net/L793404933/article/details/103442799

3D图像专栏收录该内容

1 篇文章

订阅专栏

本文深入探讨了PointCNN的工作原理及其实现细节，包括如何通过KNN获取邻居点，使用密集层提升点云特征，以及如何应用X变换和深度可分离卷积来处理3D点云数据。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

参考：https://github.com/yangyanli/PointCNN/blob/master/pointcnn.py：原始代码
https://cloud.tencent.com/developer/news/222905：PointCNN原理+代码讲解
https://www.jianshu.com/p/5a1cde05e19c：PointCNN代码阅读笔记
https://blog.csdn.net/mao_xiao_feng/article/details/78003476：实现depthwise_conv2d
https://blog.csdn.net/mao_xiao_feng/article/details/78002811:实现separable_conv2d

def xconv(pts, fts, qrs, tag, N, K, D, P, C, C_pts_fts, is_training, with_X_transformation, depth_multiplier,sorting_method=None, with_global=False)

pts：points 矩阵 [N,M,3] N是每个批次传入的点云数量,M是点的个数
fts：features 矩阵 [N,M,F]
qrs：queries
K：近邻数
D：膨胀率 P：点的个数
C：通道数（深度）
with_X_transformation：是否需要X变换

_, indices_dilated = pf.knn_indices_general(qrs, pts, K * D, True)
indices = indices_dilated[:, :, ::D, :]

得到KNN的索引

nn_pts = tf.gather_nd(pts, indices, name=tag + 'nn_pts') # (N, P, K, 3)
nn_pts_center = tf.expand_dims(qrs, axis=2, name=tag + 'nn_pts_center') # (N, P, 1, 3)
nn_pts_local = tf.subtract(nn_pts, nn_pts_center, name=tag + 'nn_pts_local') # (N, P, K, 3)

1.利用原始点 [N,M,3]+索引，使用gather方法将其合并得到[N,P,K,3]
2.取样中心点：[N,P,1,3]
3.将邻居采样点坐标-中心点坐标（广播） = [邻居点-中心点]的差异性。 [N,P,K,3]

# Prepare features to be transformed

nn_fts_from_pts_0 = pf.dense(nn_pts_local, C_pts_fts, tag + 'nn_fts_from_pts_0', is_training)
nn_fts_from_pts = pf.dense(nn_fts_from_pts_0, C_pts_fts, tag + 'nn_fts_from_pts', is_training)
if fts is None:
    nn_fts_input = nn_fts_from_pts
else:
    nn_fts_from_prev = tf.gather_nd(fts, indices, name=tag + 'nn_fts_from_prev')
    nn_fts_input = tf.concat([nn_fts_from_pts, nn_fts_from_prev], axis=-1, name=tag + 'nn_fts_input')

1.dense层（fc层/MLP）提升维度[N,P,K,3]->[N,P,K,64]类似于1维卷积操作。
2.判断除了坐标之外，是否有额外特征
3.利用原始的特征矩阵[N,M,F]，[N,P,K] ->[N,P,K,F]
4.concat: [N,P,K,3]||[N,P,K,F] = [N,P,K,3+F] :最终K近邻的特征矩阵

if with_X_transformation:

######################## X-transformation #########################
    X_0 = pf.conv2d(nn_pts_local, K * K, tag + 'X_0', is_training, (1, K))
    X_0_KK = tf.reshape(X_0, (N, P, K, K), name=tag + 'X_0_KK')
    X_1 = pf.depthwise_conv2d(X_0_KK, K, tag + 'X_1', is_training, (1, K))
    X_1_KK = tf.reshape(X_1, (N, P, K, K), name=tag + 'X_1_KK')
    X_2 = pf.depthwise_conv2d(X_1_KK, K, tag + 'X_2', is_training, (1, K), activation=None)
    X_2_KK = tf.reshape(X_2, (N, P, K, K), name=tag + 'X_2_KK')
    fts_X = tf.matmul(X_2_KK, nn_fts_input, name=tag + 'fts_X')
###################################################################
else:
    fts_X = nn_fts_input

注意：X变换矩阵的获得只使用坐标信息。【nn_pts_local】
1.卷积[N,P,K,3]->[N,P,1,KK]
2.reshape: [N,P,1,KK]->[N,P,K,K]
3.depthwise_conv2d：深度可分卷积[就是通道之间得到的卷机值独立，不会和普通卷积一样相加]：input:X_0[N,P,K,K][1,1,1,K]=[N,P,K,K]
4.fts_X: 得到的变换矩阵X:[N,P,K,K][N,P,K,3+F] ->[N,P,K,3+F]

fts_conv = pf.separable_conv2d(fts_X, C, tag + 'fts_conv', is_training, (1, K), depth_multiplier=depth_multiplier)

fts_conv_3d = tf.squeeze(fts_conv, axis=2, name=tag + 'fts_conv_3d')

if with_global:
fts_global_0 = pf.dense(qrs, C // 4, tag + 'fts_global_0', is_training)
fts_global = pf.dense(fts_global_0, C // 4, tag + 'fts_global', is_training)
return tf.concat([fts_global, fts_conv_3d], axis=-1, name=tag + 'fts_conv_3d_with_global')
else:
return fts_conv_3d

1.separable_conv2d(和上述的类似的卷积):[N,P,K,C]
2.squeeze axis=2:压榨一个维度[N,P,K*C]
----------------------------------- with_global=true
3.做分类：利用qrs，中心点的[N,P,3] ->dense->[N,P,C//4]
4.[N,P,C//4]||[N,P,KC] = [N,P,C//4+KC] 再一次融合坐标信息
------------------------------------ with_global=false
3.返回[N,P,K*C] 不融合坐标信息。