[point cloud processing paper crazy reading frontier edition 13] - gapnet: graph attention based point neural network for exploring local feature

Abstract

• Method ： This paper presents a new method for point cloud neural network GAPNet, By way of graph attention mechanism Embedded in stacked Multi-Layer-Perceptron (MLP) layers Middle school learning point cloud Local geometric representation of
  1. introduce GAPLayer, Learn the weight of each point by emphasizing the different weights of the neighborhood attention features
  2. utilize multi-head mechanism, To be able to make GAPLayer From separate head Aggregate different features
  3. Propose in the neighborhood attention pooling layer obtain local signature, It is used to improve the robustness of the network
• Code ：TenserFlow edition

Method

remember $X=\left\{x_i\in {\mathbb{R}}^F,i=1,2,\dots ,N\right\}$ For input point cloud set, In this paper ,$F=3$, Representation coordinates $(x,y,z)$.

GAPLayer

Local structure representation

Considering the real application point cloud The number is huge , So the use of $k$-nearest neighbor Tectonic orientation graph $G=(V,E)$, among $V=\{1,2,\dots ,N\}$ Representation node ,$E\subseteq V\times N_i$ edge ,$N_i$ Indication point $x_i$ Set of neighborhoods . Define the edge feature as $y_{ij}=\left(x_i-x_{ij}\right)$, among $i\in V,j\in N_i$,$x_{ij}$ Express $x_i$ Of neighboring point $x_j$.

Single-head GAPLayer

Single-head GAPLayer The structure of the is shown in the figure below 2(b).

To give everyone neighbors Allocate attention , They put forward respectively self-attention mechanism and neighboring-attention mechanism To get each point to its neighbors The attention coefficient of , Pictured 1 Shown . To be specific ,self-attention mechanism By considering the self-geometric information Study self-coefficients;neighboring-attention mechanism By considering neighborhood Focus on local-coefficients.

As an initialization step , Yes point cloud The vertices and edges of , Features mapped to higher dimensions , The dimension of the output is $F$:
$$\begin{array}{rl}{x}_i^{\prime }& =h\left(x_i,\theta \right)\\ y_{ij}^{\prime }& =h\left(y_{ij},\theta \right)\end{array}$$
among $h()$ Is a parameterized nonlinear function , Selected in the experiment as single-layer neural network ,$\theta$ yes filter Set of learnable parameters .

Through fusion self-coefficients $h\left(x_i^{\prime },\theta \right)$ and local-coefficients $h\left(y_{ij}^{\prime },\theta \right)$ To get the final attention coefficients, among $h\left(x_i^{\prime },\theta \right)$ and $h\left(y_{ij}^{\prime },\theta \right)$ Yes output as 1 Dimensional single-layer neural network , LeakyReLU() Is the activation function ：
$$c_{ij}=\mathrm{LeakyRe}LU\left(h\left(x_i^{\prime },\theta \right)+h\left(y_{ij}^{\prime },\theta \right)\right)$$

Use softmax Normalize these coefficients ：
$${\alpha }_{ij}=\frac{\exp \left(c_{ij}\right)}{{\sum }_{k\in N_i}\exp \left(c_{ik}\right)}$$
Single-head GAPLayer The goal of is to calculate the value of each point ontextual attention feature. So , Use the calculated normalization coefficient to update the feature of the vertex ${\hat{x}}_i\in {\mathbb{R}}^{F^{\prime }}$ ：
$${\hat{x}}_i=f\left(\sum _{j\in N_i}{\alpha }_{ij}{y}_{ij}^{\prime }\right)$$
among $f()$ It's a nonlinear activation function , Used in experiments RELU function .

Multi-head mechanism

In order to obtain sufficient structural information and stable network , We will $M$ Independent single-head GAPLayers Splicing , The number of generated channels is $M\times F^{\prime }$ Of multi-attention features：
$${\hat{x}}_i^{\prime }={\Vert }_m^M{\hat{x}}_i^{(m)}$$
Pictured 2 Shown ,multi-head GAPLayer The output of is multi-attention features and multi-graph features.${\hat{x}}_i^{(m)}$ It's No $m$ individual head Of attention feature, $M$ yes heads The number of ,$\Vert$ Indicates the splicing operation between feature channels .

Attention pooling layer

In order to improve the stability and performance of the network , stay multi-graph features Defined on adjacent channels of attention pooling layer：
$$Y_i={\Vert }_m^M\underset{j\in N_i}{\max }{y}_{ij}^{\prime (m)}$$

GAPNet architecture

This structure is related to PointNet Yes 3 It's different ：

1. Use attention-aware spatial transform network bring Point cloud It has some transformation invariance
2. Do not process single points , Instead, it extracts local features
3. Use attention pooling layer obtain local signature, Connect with the middle layer , Used to obtain global descriptor

experiment

Classification

Ablation study

Semantic part segmentation