当前位置：网站首页>Overview of camera image quality

Overview of camera image quality

2022-06-12 06:07:00 【A maverick zhu】

Preface ：

For many newcomers camera tuning My little friend said , Maybe I don't know much about image quality , Even when I first came into this field, it was the same , Unclear ISP What is it? , Why should I adjust it , What are the factors that affect the image quality , Which are ISP Adjustable , What can't be adjusted , So this article mainly introduces the concept of image quality , From the image quality introduction 、 Introduction to imaging system 、 Examples of typical factors affecting image quality , Three pieces of content , I hope it will be helpful to my new friends , Can deepen the understanding of image quality ;

Catalog

1、 Introduction to image quality

1-1、 Classification of imaging products

1-2、 Image quality classification

1-3、 Image quality requirements for imaging products

2、 Introduction to imaging system

2-1、 Composition of imaging system

2-2、 Two main lines of imaging system

2-3、 Real time control of imaging system

3、 Examples of typical factors affecting image quality

3-1、 Optics and lenses 、IR Related image quality problems

3-2、CMOS Sensor Related image quality problems

3-3、ISP Related image quality problems

1、 Introduction to image quality

1-1、 Classification of imaging products

1-2、 Image quality classification

Aesthetic （ The human eye judges ）

no artefacts ! color,noise,detail,focus speed、etc

Informative （ Machine identification 、 testing ）

fast,no information lost,visibility,certain level tolerace to artefacts、etc

Classification of imaging products according to image quality requirements

1-3、 Image quality requirements for imaging products

Image quality requirements for aesthetic imaging products

SLR camera -- The real world , Certain objective imaging standards should be followed CIPA,ISO, If there is a need for artistic creation, you can use PS And so on ;
Mobile camera -- Favors human vision , in personam / There are certain requirements for face shooting ;
Unmanned aerial vehicle (uav) / Motion camera -- Video stability （ Shake proof 、 It is not allowed to change the picture ）, in personam / There are certain requirements for face shooting ;
Surveillance cameras -- Low illumination noise （ Time domain , airspace ）、HDR、 High sharpness 、 in personam / There are certain requirements for face shooting ;

Image quality requirements for information imaging products

Machine vision camera -- people / Face 、 High sensitivity ;
Home Camera -- Home camera , Similar to a surveillance camera , There are certain requirements for face recognition ;
Car camera -- High dynamic range 、 High sensitivity 、 hfr 、motion artefacts、stitching artifacts、 Meet certain human vision （ Forward looking / The rear view camera is seen by the human eye ）;

2、 Introduction to imaging system

2-1、 Composition of imaging system

To understand image quality , It is necessary to understand the composition of the whole imaging system , Find out which modules in the whole imaging system will affect the image quality , The imaging process is described above , The light source （ Contains artificial light 、 Natural light 、 The flash ） Shine on an object , Light is reflected into the imaging system , Pass the camera （ For the zoom lens, there is a driving motor to adjust the lens position , Zoom ）、 aperture 、 Filter 、 Shutter reaches Sensor Imaging on , Then the digital signal is sent to ISP Image processing ,ISP Output YUV Formatted data , Used for displaying or storing after coding , Or for machine vision ; So there are many factors that affect image quality , Including light source 、 The lens 、 Filter 、Sensor、ISP、 The display on the monitor 、 code ........
Understand that it is the light source 、 The lens 、 aperture 、 Filter 、 Shutter 、Sensor It constitutes a complete optical imaging system , However, the defects and limitations of Optics and semiconductors will lead to a series of imaging problems , You need to go through ISP, Using digital signal processing method to overcome optical , Imaging problems caused by defects in semiconductors , After processing and getting a better image , It is used to display images 、 Storage 、 Or the detection and recognition of machine vision ;
summary ： The whole imaging system is a combination of hardware and software 、 The complex system of optical electromechanical combination , Do some system architecture and hardware architecture , It is necessary to understand the whole system , To design the right Architecture , Including image quality, it also deals with problems from a system perspective , Also understand the whole system , In order to better complete the work ;

2-2、 Two main lines of imaging system

Image reconstruction --- Draw the right scene clearly （ The process of image reconstruction , Is to use digital signal processing methods to overcome optical 、 Defects in semiconductors , Draw the right scene clearly ）

Optical defects and limitations ： MTF、 Aberrations 、 A song 、vignetting（ Halo , Dark angle ）/luma shading（ Brightness shadow ）、glare（ glare ）、Geometry Distortion（ Geometric distortion ）
Semiconductor defects and limitations ：defect pixel、sensitivity、dynamic range、noise：temperal noise、fix pattern noise、color noise、spatial noise

Color reconstruction --- Restore accurate and rich colors （ Sometimes it must conform to people's subjective tendency ）

Optical defects and limitations ：Color shading、chromatic aberration、purple fringe
Semiconductor defects and limitations ：Cross talk、balck level

2-3、 Real time control of imaging system

Generally, the same products are obtained from hardware manufacturers , The imaging effect is almost the same , Therefore, to have a better user experience, we must add our own optimization to the software , Such as 3A Algorithm 、 Post processing algorithm ;

Static parameters of the system ：black level、shading、noise profile、white balance、chromatic aberration correction
AE\AWB\AF（ Auto exposure 、 Automatic white balance 、 Autofocus ）： Reach exposure / white balance / Focused “ fast ”、“ accurate ”、“ steady ”
Dynamic parameters of the system ：shading,CCM, black level,noise reduction,sharpening

3、 Examples of typical factors affecting image quality

3-1、 Optics and lenses 、IR Related image quality problems

distortion

expand ： https://app.yinxiang.com/fx/9e7f52b0-0fd6-4ab4-96fd-7f150ce8eebd

Reasons for appearance ：

The root cause of distortion is that the magnification of the central area of the lens image field is inconsistent with that of the edge area ,（ Why not ？） Barrel distortion occurs when the central magnification is greater than the edge magnification , Common short focus lens , On the contrary, pillow distortion occurs , Common in telephoto lenses ;

measures ： There is a corresponding distortion correction module to correct the distortion , But the angle of view will be lost , Therefore, it is necessary to weigh a certain range ;

luma shading

Reasons for appearance ：

1、 The lens target is smaller than sensor Target surface ;

2、 Lens CRA Less than Sensor CRA, The edge of the chip is not receiving enough light ;

notes ①：Lens CRA： From the lens to the sensor side , The maximum angle of light that can be focused on a pixel .

notes ②：Sensor CRA： Micro Lens In ensuring the pixel photosensitive efficiency as the center 80% Under the premise of , The maximum optical path angle that can be corrected （ It's just like that Micro Lens And Photodiode l Location exists A horizontal error , To match Lens）.

measures ：

1、Sensor and Lens When selecting the model , The target surface is matched correctly 、CRA The angle is as close as possible to , Ask for in 3 Within degrees ;

2、ISP There is a corresponding lens shadow correction module , But it can't be too bright around , Otherwise, it will bring noise ;

color shading

See ： What is CRA_ Hardware Engineer refining road blog -CSDN Blog

Reasons for appearance ：

LensCRA Greater than SensorCRA, The light refracts to the nearby pixel, Lead to pixel Crosstalk between ;

notes ①：Lens CRA： From the lens to the sensor side , The maximum angle of light that can be focused on a pixel .

notes ③： Pictured above 1,sensor On micro lens Its function is to focus light , Introduce the incident light to the correct pixel , When the incident light CRA The angle is more than sensor Of CRA when , It will lead to passing through R-filter In the light , I got it G On pixels , Cause crosstalk before pixels , appear color shading;

notes ④： Pictured above 2,Lens Of CRA It's hard to be 0, A lens with a larger field of view , Its CRA The more it's worth （ That is, the incident light is sensor The greater the angle between the upper and the optical axis ）,sensor The micro lens on the should match the lens , You need to move the micro lens to guide the large angle incident light to the correct pixel , To ensure that the image is normal , avoid color shading appear ; Pictured 2 if Lens CRA Greater than Sensor CRA, be sensor Can't move through the micro lens , Correct all large angle incident light to the correct pixel , Will lead to color shading, if Lens CRA Less than Sensor CRA, be sensor The edge pixels will not receive enough light , Lead to a luma shading .

measures ：

1、sensor and lens When selecting the model , First of all, make sure CRA The angle should be as close as possible , Ask for in 3 Within degrees ;

2、ISP There will also be corresponding Lens shading calibration Module to do further processing , Usually luma shading and color shading Put them together to do ,luma shading Part of , It mainly focuses on the uniformity compensation around and in the center ;color shading Partly based on R、G、B 3 Two different channels , do RG,BG The compensation , Ensure that the center and surrounding areas RG、BG Is an equal constant ;

glare 、 Ghost

See ： https://blog.csdn.net/qq_35247586/article/details/114155559

Reasons for appearance ： Strong light directly enters the imaging system , Or incident light on the lens 、IR Reflection between 、 scattering , This leads to similar halos and abnormal imaging phenomena , The phenomenon similar to atomization around the light source is called glare , The abnormal images in the picture are called ghosts ;

measures ： Theoretically, it cannot be eliminated , Only by coating 、 perhaps IR To weaken ;

Purple edge

Reasons for appearance ：

The lens does not focus light of different wavelengths on the same focal plane （ The focal length of light at different wavelengths is different ）;

The imaging system usually focuses the green channel accurately , However, due to the color difference of the lens , The blue and red channels cannot focus completely accurately , So that the edge of the object appears purplish red ;

notes ：1>. In fact, it is the color difference that leads to R≠G≠B, That's why there's color ; Purple is R、B The component of the channel > G The component of the channel ;

measures ： Correct the color difference of the lens 、ISP There is a corresponding purple edge removal module in the ;

3-2、CMOS Sensor Related image quality problems

Bad point

Reasons for appearance ：

1、 In terms of Technology ： stay sensor There is dust in the process of making ; The life of electronic products is limited , This will increase the service time and cause bad points ;

2、noise: sensor gain increase , Higher temperature, etc ;

measures ：sensor and ISP There are relevant modules to remove bad points ,ISP If the intensity of removing bad points is too high , Loss of clarity , Need to weigh

noise （Fix pattern noise 、 Thermal noise ）

Reasons for appearance ：

1、FPN and sensor relevant , The immovable vertical stripes like the one above are similar to sensor relevant , Horizontal stripes are hardware related ;

2、 Thermal noise , Namely camera The internal temperature of the product is too high , As a result of sensor Noise appears , It may be related to the high power consumption of the chip 、 Small shell space ;

measures ：

1、bringup sensor when , Increase gain , Verify if there is FPN, If you have any, you need to find sensor Factory modification sensor Corresponding register ;

2、 Master control frequency reduction , Reduce power consumption 、 Change organization （ in 4 Laminate 、 Main control chip and sensor Separate 、 Replace the shell and large heat sink with larger space ）、sensor There is a similar modification of high temperature parameters ;

sensor filp/mirror

sensor mirror/filp The function is hardware design , To ensure that the imaging direction is consistent with the design , Cannot be used as a function ;

reason ：

1、 after mirror/filp after sensor The reading position has changed ,bayer The format has also changed , For some to rely on calibration Modules of data , The data must be different , such as shading;

2、 after mirror after , It shows colorcheck Of raw The graph is also mirrored , There's no way to ccm、blc Correction of ( ISP The debugging process of , Generally, it is based on the pictures taken first raw chart , To correct ）;

measures ：ISP To do filp/mirror The function of 、 The back-end application layer does ;

3-3、ISP Related image quality problems

Sports drag

1、 Drag caused by excessive noise reduction intensity in time domain

measures ： Reduce the intensity of time domain noise reduction ( When debugging, you can pass bypass The corresponding module is used to verify whether it is caused ) , But it may cause obvious dynamic noise , Need to weigh ;

2、 Smear caused by long exposure time

Reasons for appearance ： The whole moving process of a moving object is cmos Imaging on , So the photos taken have drag shadows , Only if the exposure time is fast enough , An object is relatively stationary relative to the camera , There will be no drag shadow ;

measures ： Because of inadequate exposure , So the exposure time is long , It can be changed to high sensitivity sensor, In this way, the exposure time can be reduced ;

3、HDR In the algorithm, , Drag caused by multi frame fusion

4、 The jelly effect

Reasons for appearance ： Use line by line exposure sensor Caused by , because sensor It is exposed line by line , There is a time difference between lines , When shooting fast moving objects , It will cause the imaging object to be distorted , Elongated , Pay attention to distinguish it from the smear caused by exposure time , If the exposure time is too long, there will be a smear , But it will not cause the imaged object to deform ;

measures ：

1. If you use a roller shutter （Rolling Shutter） The video camera , Try to increase the frame rate of the camera , In fact, it reduces the scanning time used to form a frame , In this way, the movement of the object is not obvious in a short time , It can alleviate the jelly effect .

2. Switch to the global shutter （Global Shutter） The camera , use CCD The photosensitive element completely solves the jelly effect . Global shutter （Global shutter）： All pixels on the sensor collect light at the same time , Exposure at the same time . For capturing objects moving at high speed CCD Advantages and disadvantages ： Due to technical limitations , It is difficult to handle huge amounts of data at the same time .

3. Set the speed of line by line reading , Set it faster and higher , This also avoids the jelly effect , Minimize the vibration of the environment . Increase shutter speed , Take the high-speed shutter directly .

3-4、Encoder Related image quality problems

Encoder The aim is to transform the original ISP The output image is compressed , Make the transmission 、 Stored size smaller , Due to the compression process , Algorithm or insufficient resources , Resulting in loss of image quality , There are many forms , As follows ;

mosaic

ringing （ Jingling ）

There are many edges on the edges of words and images , It is common to take text images ;

contouring

Similar contour phenomenon ;

blurry（ Fuzzy ）

mosquito （ Mosquito noise ）

3-5、 Display related image quality problems

The TV PQ（ Picture-Quality） It can lead to Camera The display effect is inconsistent （ brightness 、 color 、 Noise, etc ）;
Display color gamut 、 Differences in color levels can lead to Camera The display effect is inconsistent ;

notes ：【 Hardware Science 】 What is the color gamut and color accuracy of the display ？ What experiences will he influence _ Bili, Bili _bilibili

As shown in the figure below , The monitor has different specifications , Including color gamut 、 Color space 、 Color accuracy , All of these will affect camera Output effect , Pay attention to these during the effect test ;