Sparse Tensor-based Point Cloud Attribute Compression

This paper uses sparse convolution , Stack multiple sparse convolutions into VAE Basic framework ; Use this basic framework to do learning-based Methods . It is worth mentioning that , This article is used in attribute coding , and learning-based Rarely used in attribute coding .

Ideas

Background

Related work

End-to-End (E2E) Learning

To a E2E The architecture of , It is generally divided into two encoders ： One is the main encoder （Main codec） And a super prior encoder （Hyper codec）.

In a super prior encoder , It will be Hyper Encoder Extract prior information , After use Hyper Decoder To decode main codec Mean and variance of $({\mu }_i,{\sigma }_i)$.

Sparse convolution

The problem to be solved is to extract more effective features from sparse data . We know , Point clouds are sparse , If we use the traditional convolution calculation , The cost is bound to be great . therefore , People use sparse convolution to compute sparse data effectively , Not for picture pixels （2D） And the voxel points of space are scanned one by one .[1]

Quantization and Rate-Distortion

The roughness of quantification determines the performance of the final model （ Namely distortion $D$ The amount of ）. And the bit rate $R$ It directly determines the roughness of quantification .

Model

Definition

First you need to define a Sparse Tensor：$\{\overrightarrow{\mathbf{C}},\overrightarrow{\mathbf{F}}\}$. among ,$\overrightarrow{\mathbf{C}}=\left\{\left(x_i,y_i,z_i\right)\mid i\in [0,N-1]\right\}$、$\overrightarrow{\mathbf{F}}=\left\{\left(R_i,G_i,B_i\right)\mid i\in [0,N-1]\right\}$

In the framework of this model , The entropy coding model will be used . For entropy coding model , The author uses [2] Prior knowledge in , To improve the performance of entropy coding model .

Reference

[1] https://towardsdatascience.com/how-does-sparse-convolution-work-3257a0a8fd1

[2] VARIATIONAL IMAGE COMPRESSION WITH A SCALE HYPERPRIOR