Structured light 3D reconstruction (1) Striped structured light 3D reconstruction

前言

In the field of close range 3 d reconstruction,Structured light technology is the most widely used,尤其在工业领域.The technology currently has the advantages of high precision and ignore the weak texture,But the complex outdoor environment will cause a certain degree of interference with the technology.Now with more structured light respectively two kinds：Linear structured light and structured light stripe.Then I'll make a brief introduction of structured light stripe 3 d reconstruction,事实上,The principle of structured light stripe and binocular stereo matching is a very similar.

---

1 Monocular scale uncertainty

Below is a typical binocular structured light stripe equipment,Is composed of two cameras and projectors,By structured light projector projection,Camera to capture the image.

In fact, if less projector,This equipment can be regarded as a binocular stereo matching camera.我们知道,Working principle of binocular stereo matching is through the camera around the parallax images to reconstruct 3 d scene,The principle of which is similar in the eyes of man.Which one of the most important work,计算视差,Is about to look for the same point in the camera image,效果如下所示：

The computer to find points with the same way,Is not so smart as the human eye,We by describing the son(sift,surf,orb等)Find more significant feature points in the picture,And then to match them(匹配,In fact because of polar constraint relations,This match is notn²那么复杂),最终的效果如下所示：

With the same name point,We can according to the parallax formula(Triangle calculation relationship)For 3 d reconstruction.但是这个方法,存在一个致命的问题,That is if the object's surface without abundant texture information.这种情况,For most of the newly processed workpiece is a very normal,比如下图：

那如何解决呢？We can measure projecting structured light to stay,Active for surfaces to create“纹理”,那么问题就迎刃而解了.但事实上,This way is still under the influence of surface material,For example some places appear high reflective phenomenon, etc,这里先略过.

2 相移法 + 多频外差法

In the structured light technology,We often projectionNStep phase-shift stripe to active tag area,Projection of the stripe as shown below：

2.1 Stripe generated
先来看下“Computer generated ideal,For projector compute projective stripes”与”After the actual deformation,Camera stripes“的特点.特点1：Single direction code：Polar constraints decrease search;特点2：Gray level change again and again：A field with multiple cycles of stripes.

Assumes that the phase shift step forN,Generate a stripe phase shift formula：

A：背景光强
B：调制强度
$\varphi$：相位值
$\nabla \varphi$：相移值
f = 1 / T ：Phase shift frequency,其中TThe number of pixels for a single cycle,Also known as the stripe cycle,Some also become pitch
x：On the direction of coding pixel coordinates
（注意：Usually only one direction at a phase shift method code,如果要对y方向,Also need to regenerate,In fact use polar constraints,Structured light coding algorithms of single direction can finish rebuilding.）

四步相移：

2.2 包裹相位
Solution phase formula(The analytical solution of the above equations of multiple)：

（注意：This formula is established for a single pixel,Because he is a heterodyne method(Similar to the GPS,The difference in value between multiple signal),So there are two：）
1.In view of the projected onto the object deformation in the aftermath of the stripe is still valid：Also is we can only mark point,In the absence of error,The left camera solve phase$\varphi$_lAnd the right camera solve phase$\varphi$_r是相同的,Using binocular distance measurement model,We can calculate the point3D坐标;
2.Structure light algorithm is easy to deploy inFPGA或者GPU端：To achieve fast reconstruction speed.
（注2：Solve the phase divided by2Πf,You can get it herex,但是由于某些原因(多频外差),When calculating absolute phase needs to be normalized,Because we don't use this way to calculate the abscissa of it.）
调制度B(Used to filter out noise area)And calculation of the error term is as follows：

$\sigma$_n：Projection of the brightness of the standard deviation,Is determined by the projector technology
N：Phase set number;
$\alpha$：Camera corresponding coefficient;
t：曝光时间;
r：Surface emissivity;
B：调制光强.

Two conclusions can be by the type：1.步长N越长,The smaller the phase error,事实上,增大步长NBy many benefits,Such as increasing the performance of measuring high reflective、Reduce the effects of projector nonlinear,But the longer the step length means increasing measuring time,For some need dynamic measurement of the scene is difficult to accept！2.Increase the light intensity modulation,可以减小误差,But the projector nonlinear area,调制度BCan't increase wireless.

Wrapped phase calculation results are as follows：

2.3 多频外差法：绝对相位
Assume that the resolution of the projector as1280 x 720,编码x方向,Even if we choose the biggest stripe cycle：T=255,That is, a gray coding a stripes,A single cycle of all stripes are still not enough coding field,Also is unable to eliminate the phase ambiguity：
在这种情况下,Suppose we know decoding of stripe belong to k 个周期,Also is the stripe order,So its actual phase values,Is the absolute phase is the：
To understand how to calculate the absolute phase,Here we use the multifrequency heterodyne method,The principle of which is through a number of different cycle sine grating phase to do bad,将小周期的相位主值转化为大周期的相位差,从而使得相位差信号覆盖整个视场,And then according to the absolute phase distribution of phase difference to get whole image.
Here in double-frequency heterodyne, for example,原理如下图所示,其中T₁、T₂Cycle for stripes,$\varphi$₁、$\varphi$₂For the wrapped phase values,For the phase difference of the two$\varphi$₁₂：

And about the cycle,假设$\varphi$₁ < $\varphi$₂,显然有：

See from this cycle：With a short cycle phase shift stripe superposition,Thus and get cycle longer phase shift stripes,In order to be able to achieve full unambiguous phase unwrapping,我们必须选择合适的T₁、T₂,使得T₁₂Over the entire field of view,In order to gain the only phase.
（注意：Dual-frequency extrapolation or not enough,We need to introduce frequencyT₃,To get more than field width of synthetic cycle）.

3. 相移法+格雷码法

3.1 Binary code and gray code
In the simplest three binary code, for example,Bright area：编码为1,Dark area：编码为0、Suppose we now to order the projection object to be tested the three binary encoding of the pattern,如下所示:(注,Because of the polar constraint,We only need to code a direction,即可完成3D重建)

Consider a single pixel of encoding in the case of clear photos,This kind of coding way, of course, there will not be any problem.问题就出在：Camera to the boundary point of black and white alternate with of transition is often a gray,It is easy to cause the decoding error.In the last image coding,Stripes are very thin,以上面3A code, for example,如果是

格雷码特点

3.2 格雷码+Phase shift method required image sample

Move is photogenic stripes(N = 3), 取n = 4,Besides the traditional phase shift+Black and white gray code technology need projection image,For binary,生成的图像如下所示：

Actual pictures(N = 4):

3.3 Gray code binarization
For the gray code decoding,主要的困难在于：Gray code binarization,That is how to measure a point is bright stripe or dark stripes？Due to the ambient light、And the causes of surface material,A pair of gray levels of pixels in the image is usually uneven,But this article conclusion is certain：Binarization principle：对于同一位置,It is the brightness of the light stripe to is always higher than its being the brightness of the dark stripe light！So for a pixel in an image binarization of,We can combine images to judge,具体的做法是：

但是,This kind of method for encoding for all0、或者全为1The point is invalid！The point is,Such as the following picture made up
码图,区域 0 全暗,区域10全亮：

Because of these points will not experience light and shade change,So bad judgment is bright stripe or dark stripes.We have a lot of ways to avoid the phenomenon of,比如说：To avoid the code;Comparing with other pixel brightness.其中,There is a better robustness solution is,Additional let all coding coding location can experience the whole0或者全1的过程,This is the traditional gray code with phase-shift technology needs additional projection two all black and white pattern.

3.4 Gray code decoding
Once you determine the binarization strategy,We can to each pixel to solve its gray code value,And map it to the natural sequence of,According to the following formula：

解码代码如下：

3.4 相移法+Gray code specific principle

问题：

最终结果：

总结

Finally from the efficiency remarkably improved algorithm has a lot of、The algorithm of accuracy higher than on the,But need to read the latest paper,Here is the light of the structure under a simple introduce,Theory is simple,这里就不放源码了.