CV learning notes - camera model (Euclidean transformation and affine transformation)

Camera model

1. Camera model principle

1. Camera imaging

Camera imaging is a small hole imaging model , Map the three-dimensional space object in real life to the two-dimensional imaging plane , And then generate a two-dimensional image .

Coordinate system in camera model ：

• World coordinate system Pw(Xw, Yw, Zw)： Coordinate system corresponding to the real and objective three-dimensional world , Also known as the objective coordinate system , Represent the position coordinates of the object in the real world . The world coordinate system changes with the size and position of the object , Units are generally units of length .
• Camera coordinate system PO(x, y, z)： Corresponding to the coordinate system with the optical center of the camera as the origin , It is parallel to the image x and y Direction is x Axis y Axis ,z The axis is parallel to the optical axis , The unit is also the unit of length .
• Image physical coordinate system P’(x’, y’)： Corresponding to the coordinate system with the intersection of the main optical axis and the image plane as the coordinate origin , The unit is still the unit of length .
• Image pixel coordinate system P(u, v)： Corresponding to the coordinate system with the vertex of the image as the coordinate origin ,u and v The direction of is parallel to x’ and y’ Direction , Unit is pixel

Camera imaging process ：

2. Coordinate transformation

European transformation ： Objects in any three-dimensional space , Can be expressed in three-dimensional coordinate system , That is, the point of an object in real space , The coordinates are different in different coordinate systems , Because the base is different , So a fixed point , Coordinates in a coordinate system , If you convert to another coordinate system , So that another coordinate system can determine this position , You need to use Euclidean transformation . The transformation from any coordinate system to another coordinate system can be completed by rotation and Translation .

Translation transformation & Rotation transformation & Homogeneous coordinates ：

Translation transformation It refers to the transformation of the coordinate systems of both sides for a finite number of times translation Sure coincidence , For example, the high hat is directly above the face , In this way, the central coordinate system of the face is converted to the central coordinate system of the high hat , Just translate and transform the coordinate origin upward , There is no rotation offset of the coordinate axis , The translation transformation in linear algebra is the transformation of the position in the spatial coordinate system , It is customary to use O Vector representation of , By adding constants to vectors , Multiple translation transformations can be realized by adding vectors directly . Translation transformation is relatively simple and easy to understand , The rotation transformation is more complex , It can be extended to three-dimensional Cartesian coordinate system through two-dimensional coordinate rotation transformation , For rigid bodies in space , There are six degrees of freedom , Around the xyz There are three degrees of freedom of rotation , And in the two-dimensional plane , Only rotate about the axis of the vertical plane , That is, there can only be one degree of freedom , Then for the three-dimensional coordinate system, you can write 3×3 Of R matrix , Write it as a rotation matrix

The above formula means from B Coordinate system conversion to A Each column of sub vectors in the rotation matrix of the coordinate system , The equivalent is B The coordinate axis unit vector of the coordinate system is A The representation in the coordinate system . It is not difficult to find that this rotation matrix is an orthogonal unit matrix , Therefore, the following inference ：

The above formula explains , Every row of the matrix is A The coordinate axis of the coordinate system is B The representation in the coordinate system . The formula of rotation transformation is as follows ：

Continuous rotation transformation can only be achieved by matrix multiplication . When the rotation translation transformation occurs at the same time , Introduce the concept of homogeneous coordinate transformation ;

From the above conclusion, it is not difficult to deduce this transformation process , If this process is represented by a matrix , It's a lot simpler , Homogeneous coordinate transformation matrix also came into being , The transformation process of the above figure is equivalent to the following expression ：

T Matrix is the homogeneous coordinate transformation matrix in this transformation , Homogeneous coordinate transformation matrix T It's a fourth-order square matrix , It can represent rotation transformation and translation transformation at the same time , The formula of homogeneous coordinate transformation matrix is expressed as follows ：

Simplify the angle sign of the above derivation process , for instance ：

If vector a Perform two Euclidean transformations , The rotation and translation are R1,t1 and R2,t2, Get... Separately ：

b = R1*a + t1; c = R2*b + t2; c = R2*(R1*a + t1) + t2

World coordinate system to camera coordinate system ：

The rotation matrix is no exception in the transformation from the world coordinate system to the camera coordinate system R Peaceshift matrix t, The relationship is expressed as ：

Camera coordinate system to image physical coordinate system ：

• principle ： Similar triangles

• The matrix is expressed as ：f For focus

Image physical coordinate system to image pixel coordinate system ：

• principle ： Coordinate offset (dx、dy Express x and y How many length units does a pixel in the direction occupy ,dx、dy It could be a decimal )

• Homogeneous coordinates represent ：

Continuous Euclidean transformation from camera coordinate system to image pixel coordinate system ：

World coordinate system to image pixel coordinate system ：

• Derivation process ：

• Conclusion ：

2. Lens distortion

1. Lens distortion

Due to the deviation of manufacturing accuracy and assembly process , The finished lens often produces distortion , Cause distortion of the original image . Lens distortion is divided into radial distortion and tangential distortion .

2. Radial distortion

• reason ： The distortion caused by the shape of the lens is called radial distortion , Schematic diagram of point displacement after radial distortion of lens ：

Occipital distortion

Barrel distortion

3. Tangential distortion

• reason ： Tangential distortion is due to the lens itself and the camera sensor plane （ Imaging plane ） Or the image plane is not parallel .（ Mostly due to the installation deviation of the lens pasted on the lens module ）

4. Distortion correction

Method ：

• Radial distortion and tangential distortion models have 5 Distortion parameters , stay Opencv They are arranged in a 5*1 Matrix , Include in turn k1、k2、p1、p2、k3（ this 5 The first parameter is the camera parameter that needs to be determined in camera calibration 5 Distortion coefficients ）, Often defined as Mat Matrix form

  Mat distCoeffs = Mat(1.5, CV_32FC1, Scalar::all(0));
  

• Get this 5 After a parameter , The distortion of the image caused by lens distortion can be corrected .

3. Perspective transformation

1. Definition

Perspective transformation is also called projection mapping (Projective Mapping), It is to project a picture from the original plane to a new viewing plane (Viewing Plane) The process of .

2. Purpose

Real objects are straight edges, which may be oblique lines in the picture , This oblique line can be transformed into a straight line through perspective transformation .

3. And affine transformation

Affine transformation is a special case of perspective transformation , Affine transformation （Affine Transformation or Affine Map） It refers to a linear transformation and a translation of an image from a vector space in geometry , The process of transforming to another vector space .

4. The transformation process

• Change the formula ：

​ In the following formula X,Y Is the coordinates of the original picture （ Upper type x,y）, Corresponding to the transformed picture coordinates （X’;Y’;Z’） among Z’=1：

In a general way , We make a33=1, Expand the above formula , Get a point ：

---

Personal study notes , Just learn to communicate , Reprint please indicate the source ！