[point cloud processing paper crazy reading cutting-edge version 12] - adaptive graph revolution for point cloud analysis

Abstract

• problem ： The standard convolution operation cannot be performed in 3D The feature correspondence is distinguished between points
• Method ： In this paper, Adaptive Graph Convolution(AdaptConv), according to 3D Point to the characteristics of dynamic learning Generate adaptive kernel
  1. Use and fixing / Isotropic kernel comparison ,AdaptConv Improved point cloud Flexibility of convolution , Effectively and accurately get a variety of relationships between different semantic parts
  2. Different from the method of using attention weight ,AdaptConv Make convolution operation more adaptive , Not simply for neighboring points Assign different weights
• Code ：PyTorch edition

introduction

Graph CNNs According to the space between points / Feature similarity will point cloud Expressed as graph data , And will images Upper 2D Convolution is extended to 3D light .

The standard Graph CNNs Usually, the shared weight function is used to extract the corresponding edge features of each pair of points , This will lead to a fixed / Convolution in the same direction kernel, When applied to all point pairs , Will ignore the corresponding relationship between different characteristics .

The key contribution of this work is AdaptConv In the graph Use in convolution , Instead of a weight function based on the characteristics of the results .

Besides , Some feature convolution designs have also been developed , It can carry out adaptive convolution more flexibly .

Method

Adaptive graph convolution

remember $\mathcal{X}=\{x_i\mid i=$$1,2,\dots ,N\}\in {\mathbb{R}}^{N\times 3}$ Enter a point cloud for ,$\mathcal{F}=\left\{f_i\mid i=1,2,\dots ,N\right\}\in {\mathbb{R}}^{N\times D}$ Is the corresponding feature , among $x_i$ Means the... Th $i$ Point $(\mathbf{x},\mathbf{y},\mathbf{z})$ coordinate , In other cases , It can also be combined with other features .

Then calculate the directed graph according to the given point cloud $\mathcal{G}(\mathcal{V},\mathcal{E})$, among $\mathcal{V}=\{1,\dots ,N\}$ and $\mathcal{E}\subseteq \mathcal{V}\times \mathcal{V}$ Represents a collection of vertices and edges . By including self-loop Of $k$-nearest neighbors (KNN) structure graph.

In the given input $D$ After Witt's sign ,AdaptConv layer A new set of $M$ Whitman's sign , The number of points is the same as the input . And previous graph convolution Layer by layer , It can more accurately reflect the local structural characteristics .

remember $x_i$ yes graph convolution The center of ,$\mathcal{N}(i)=\{j:(i,j)\in \mathcal{E}\}$ Is the index of adjacent points . Due to the irregularity of the point cloud , The previous method is usually in $x_i$ All of the neighbored points Apply fixed kernel function , Used to capture patch The geometric information of . however , Different neighbored points May get corresponding $x_i$ Different characteristics , Especially when $x_i$ Located in a prominent area , Such as corners or edges . under these circumstances , fixed kernel It may not be possible to graph convolution Get the geometric representation information for classification or segmentation .

In the method of this paper , Designed an adaptive kernel, Used to calculate the significant relationship between each pair of points . about $M$ Every channel of dimensional output features ,AdaptConv Will dynamically generate a kernel, It is applied in points features $\left(f_i,f_j\right)$ The function on ：
$${\hat{e}}_{ijm}=g_m\left(\Delta f_{ij}\right),j\in \mathcal{N}(i).$$
among $m=1,2,\dots ,M$ Express $M$ One of the output dimensions , Corresponds to a separate filter.$\Delta f_{ij}=\left[f_i,f_j-f_i\right]$ Used to capture global structure and local domain features ,$[\cdot ,\cdot ]$ The splicing operation is ,$g(\cdot )$ Is the feature mapping function , namely $MLP$.

And 2D The calculation in convolution is the same , take $D$ Dimension input and corresponding filter The weight is convoluted to get $M$ One dimension in dimensional output , This article will adaptive kernel And the corresponding point $\left(x_i,x_j\right)$ Convolution ：
$$h_{ijm}=\sigma *{\hat{e}}_{ijm},\Delta x_{ij}*,$$
among $\Delta x_{ij}$ Is defined as $\left[x_i,x_j-x_i\right]$ similarity ,$*\cdot ,\cdot *$ Represents the inner product of two vectors , Output is $h_{ijm}\in \mathbb{R}$,$\sigma$ It's a nonlinear activation function .

Pictured 2 Shown , The first $m$ individual adaptive kernel${\hat{e}}_{ijm}$ And corresponding points $x_j\in {\mathbb{R}}^3$ Of spatial relations $\Delta x_{ij}$ combination , Express kernel The size of should match the inner product , Feature mapping $g_m:{\mathbb{R}}^{2D}\to {\mathbb{R}}^6$. Store for each channel $h_{ijm}$, Get the connection point $\left(x_i,x_j\right)$ Edge features between $h_{ij}=$$\left[h_{ij1},h_{ij2},\dots ,h_{ijM}\right]\in {\mathbb{R}}^M$.

Last , By using the aggregation function of all edge features in the neighborhood central point $x_i$ The output characteristics of ：
$$f_i^{\prime }=\underset{j\in \mathcal{N}(i)}{\max }{h}_{ij},$$
among max It is in the unit of channel max-pooling function . All in all ,AdaptConv Of convolution weights Is defined as defined as $\Theta =\left(g_1,g_2,\dots ,g_M\right)$.

Feature decisions

Is to use features to find spatial relationships .

If input $x_i\in {\mathbb{R}}^E$ Contains more information , That's another option , In the experiment, we will consider .

Space information $\Delta x_{ij}$ Replace with characteristic information $\Delta f_{ij}$, You'll get different ${\hat{e}}_{ijm}$, This is also an option to consider .

This article chooses to use $\Delta x_{ij}$ As a transform domain, we have the following considerations ：

1. Point features have been used to generate adaptive kernel 了 , Convolution using features will lead to redundancy of feature information
2. The dimension of features is high ,MLP It's difficult to learn in high-dimensional space
3. Large memory consumption , High computational complexity

Network architecture

graph The structure of is dynamically updated at every level

experiment

Classifcation

Part segmentation

Indoor scene segmentation

Ablation studies

Adaptive convolution vs Fixed kernels & Feature decisions

Robustness test

Efficiency

Visualization and learned features