Three dimensional reconstruction of deep learning

be based on MVS Foundation of 3D reconstruction

Three dimensional information representation

It is generally divided into depth map / Parallax map 、 Point cloud 、 grid . They are all expressions 3D A way of information , We will choose different ways to express according to different actual application scenarios . For example, do some background sequencing 、 Face effects can only use depth maps ; And if we want to rebuild a large scene , Such as museums , It needs to be displayed for everyone to browse , You can use grids to represent ; When positioning , We just need to use some clouds . But if we want to make point clouds or grids , Must use depth map , This step must be experienced . Only with depth map can we get point cloud or three-dimensional grid .

• Depth map / Parallax map

1. Depth map ： The distance from each point in the scene to the camera ;
2. Parallax map ： The position deviation of pixels imaged under two cameras in the same scene dis
3. Relationship between them ：depth=bf/dis
4. It is a common representation of three-dimensional information

In the diagram above ,Ol and Or It's two cameras , We generally call it binocular camera , The distance between them is called the baseline (Baseline). A point in space P, The distance from it to the baseline Z It's called depth . The two red lines in the above figure are different imaging of two cameras .p Point and p' The difference is P stay Ol and Or Points in camera imaging . parallax d Equal to the column coordinates of the same point pair in the left view minus the column coordinates in the right view , yes Pixel unit

The above picture is taken by binocular camera , The parallax of the electric rearview mirror is 80-35=45.

In stereovision , The parallax concept is used in baseline corrected image pairs . That is to say, the two cameras are right , parallel , They all shoot objects forward , Only then can the parallax map be used . Generally speaking, we use depth maps , Parallax map is easier to shoot , Then convert it into a depth map . Parallax is measured in pixels , The unit of depth is mm (mm), Change the formula to depth=bf/dis, here b Is the baseline distance of the binocular camera , This is known ,f Represents the normalized focal length , That is, in the internal parameter fx, This is also known ,dis Is the parallax value .

• Three dimensional point cloud

1. A 3D point cloud is a data set of points in a coordinate system
2. Contains a wealth of information , Including three-dimensional coordinates XYZ, Color RGB Etc

3D point cloud is actually data , It can be viewed intuitively without showing to human beings .

• Three dimensional grid

1. It is composed of polygons composed of adjacent point clouds of objects .
2. Usually consists of triangles 、 Quadrilateral or other simple convex polygons .

As can be seen from the above figure , 3D mesh is a form of point cloud , It is generally without color information .

• Texture map model

1. 3D mesh model with color information
2. All color information is stored on a texture map , When displaying, the high-resolution color model is rendered according to the texture coordinates of each mesh and the corresponding texture map .

Image coordinate system 、 Camera coordinate system 、 World coordinate system