3D restoration paper: shape painting using 3D generative advantageous networks and recurrent revolutionary networks

2019/09/16

Address of thesis ：https://arxiv.org/abs/1711.06375

  Abstract

Repair the three-dimensional model for semantic rationality and context details , This paper introduces the concept of 3D Codec generates countermeasure network (3D-ED-GAN) Convolution with long-term cycle (LRCN) A hybrid architecture .

among ：

3D-ED-GAN Used to fill in the missing at low resolution 3D data ;

LRCN It uses a circular network to minimize GPU Use of video memory , And integrate the codec into LSTM In the network . hold 3D Model as 2D Slice sequence ,LRCN You can put the rough 3D The shape becomes a more complete and high-resolution block , Because I caught 3D The context of the shape ,LRCN Have fine-grained details locally .

One 、 brief introduction

 3D sensor （ Such as LiDAR、Kinect） A lot of data obtained , Blocked 、 The influence of sensor noise and illumination , The 3D model is incomplete and noisy , For example, scanning buildings are blocked by trees , The obtained model is incomplete .

Usually , Handle 3D Voxel model , Use 3D Convolution . The current problems ,3D Convolution computation is too large , It takes up too much video memory , This limits 3D Convolution resolution .

Methods of this paper , There are no such restrictions , So it can be done ：
（1） Repair missing or damaged parts , Refactor a completed 3D structure
（2） Predict a high resolution with fine-grained details 3D shape

Two 、 Network structure

  The overall network structure is shown in the figure 1 Shown ：

chart 1 The paper corresponds to the network structure

 Slice On the left is 3D-ED-GAN, The function is to repair the low resolution 3D Model ; On the right is LRCN part , Use RNN Avoid video memory restrictions , To achieve high-resolution prediction .

1、3D-ED-GAN The Internet

chart 2 3D-ED-GAN

  The network structure is shown in the figure 2, Codec generates network .

There are two in this part loss, One is between the patched result and the complete result loss：

Between the patched model and the complete model loss

One is the discriminator loss：

Judging device loss

 、 Express x、 No i The individual element , The meanings of other symbols are shown in the figure 2.

 2、LRCN The Internet

chart 3 LRCN The Internet

  Structure is shown in figure 3 Shown , Steps are as follows ：

• PCA In order to get the most context information , Make sure there are as many non blank slices as possible , Used PCA To align 3D Model .
• Put the slices into 3D CNN Code in
• The coding result is shown as LSTM, Form context
• meanwhile LSTM The output of is put into 2D Full-CNN decode , Upsampling into high-resolution results .

  There is only one network loss, The author tried L1 and L2, Find out L1 better .

3、 ... and 、 experimental result

  Experiment on the scanning results of real objects , The result is shown in Fig. 4

chart 4 Experiment on scanning results of real objects

There are also experimental results of introducing a small amount of noise 、 The result of the classification , No post here .