[Beijiao] image processing: basic concepts, image enhancement, morphological processing, image segmentation

1. Basic concepts

1.1 Human visual characteristics

• Polysemy （ The same picture will have different meanings because of the exchange of foreground and background ）
• illusion
• Mach belt ： Near the part where the brightness changes , Darker dark areas , The bright area is brighter

1.2 Machine vision

• Handle 、 understand 、 Perceive digital images

1.3 Image sampling and quantization

• Spatial resolution
• Brightness resolution

1.4 Histogram

• Gray histogram ： have i The number of gray level pixels （ area ）
• An image corresponds to a unique histogram , One histogram corresponds to multiple images （ The histogram does not store the position information of the image , Only the pixel components are recorded ）
• application ：
  • Histogram equalization ： Image enhancement
  • Segmentation threshold ： Image segmentation
  • Histogram comparison ： Image classification

2. Image enhancement

2.1 Image enhancement reasons

• The visual effect is not good ： Gray scale transformation to human eye sensitive area
• Noise pollution ： wave filtering （ Space domain 、 frequency domain ）
• Difficult to analyze and understand

2.2 Purpose

• Highlight or remove certain information
• Did not increase the amount of information , It may also lose the amount of information
• There is no unified and objective evaluation standard , Specific use specific method

2.3 Method

2.3.1 Spatial domain enhancement

Directly operate the gray level of image pixels

• Gray scale transformation

  • linear transformation , Piecewise linear transformation ： Grayscale stretch （ prominent ） And compression （ Inhibition ）
  • Logarithmic transformation ： Gray stretch when gray level is low , High gray levels compress , Suitable for highlighting images with low gray levels
  • Power transformation ： Power index <1, Suitable for processing low gray level ,>1 Suitable for high gray level
  • Histogram change ： Use the constant number of pixels before and after the change as a bridge , The relationship between the histogram before and after the change is derived as shown in the figure below ：

The relationship between the histogram before and after the change

• Algebraic operations
  • Add operation ： Remove the superimposed random noise （ The mean value of noise is 0, Find the average of Duofu image , Mostly used for video processing ）
  • Subtraction operation ： Split specific areas （ Subtract the background area ）, Detect scene changes （ Subtract the template image ）
  • Multiplication operation ： Get the specified area （ Multiply with mask ）
• Spatial domain filtering
  Lowpass wave filtering ： Integral operations , Denoise , Image smoothing
  • mean value wave filtering ： Low pass filtering
  • gaussian wave filtering ： Low pass filtering
    The above two will cause Edge blur （ The original clean pixel mutation edge is turned into a slope of pixel changes ）
  • The median wave filtering ： Can still get Clear borders
  • qualcomm wave filtering ： Differential operation , Edge sharpening （ The original image is superimposed with the gradient ）
    • Robert operator ： Cross difference algorithm $\left|\begin{array}{cc}-1& 0\\ 0& 1\end{array}\right|$$\left|\begin{array}{cc}0& -1\\ 1& 0\end{array}\right|$
    • Prewitt operator ：$\left|\begin{array}{ccc}-1& 0& 1\\ -1& 0& 1\\ -1& 0& 1\end{array}\right|$$\left|\begin{array}{ccc}-1& -1& -1\\ 0& 0& 0\\ 1& 1& 1\end{array}\right|$
    • Sobel operator $\left|\begin{array}{ccc}-1& 0& 1\\ -2& 0& 2\\ -1& 0& 1\end{array}\right|$$\left|\begin{array}{ccc}-1& -2& -1\\ 0& 0& 0\\ 1& 2& 1\end{array}\right|$
    • Laplace operator ： Second order difference , Sensitive to noise , Cannot detect edge direction , Edge positioning is possible $\left|\begin{array}{ccc}0& -1& 0\\ -1& 4& -1\\ 0& -1& 0\end{array}\right|$$\left|\begin{array}{ccc}-1& -1& -1\\ -1& 8& -1\\ -1& -1& -1\end{array}\right|$
      The first derivative can detect whether the pixel is on the edge , The second derivative can detect whether the edge point is on the bright side or the dark side

2.3.2 Frequency domain enhancement （ spectrum shaping ）

• Fourier series ： Periodic signals can be expressed as the superposition of countless sine waves
• DFT： One dimensional discrete signal Fourier transform
• 2-D-DFT： Two dimensional discrete signal Fourier transform
• Digital images are all real functions , Fourier transform is symmetric , Spectrum amplitude spectrum symmetry
• The average value of the gray level of the image corresponding to the origin of the frequency domain
• Original image -FFT- spectrum shaping -IFFT- Enhanced image
• Only additive noise can be reduced , Multiplicity cannot be reduced 、 Convolution noise
• The main properties ：
  • Translation properties ： Phase shift corresponds to frequency domain translation
  • Rotation characteristics ： Space domain rotation , Fourier transform rotates the same angle
  • Scale scaling ： Amplitude spectrum reverse amplification and contraction
  • Convolution property ： Spatial convolution corresponds to frequency domain product , Space product corresponds to frequency domain convolution
  • Related features ： Spatial correlation corresponds to frequency domain multiplication （ Will speed up the calculation ）
  • Separation properties ： Two dimensions are decomposed into two one dimensions , One dimension can be used FFT Speed up
• Low pass filtering
  • Ideal low pass filter ： Ringing phenomenon （ Bright and dark circles ）
  • Butterworth low pass filter ： Order n The bigger it is , The better the performance , The closer to the ideal performance （ Passband does not decay , The stopband attenuation is 0,n<=2 The ringing phenomenon can be ignored ）
  • Gaussian low pass filter ： As the cut-off frequency increases , Performance improvement , The smoothing effect is not as good as the former , But there is no ringing , Most widely used
• High pass filtering
  • Ideal high pass filter ： Ringing phenomenon （ Bright and dark circles ）
  • Butterworth High pass filter ： Order n The bigger it is , The better the performance , The closer to the ideal performance （ Passband does not decay , The stopband attenuation is 0,n<=2 The ringing phenomenon can be ignored ）
  • Gaussian high pass filter ： As the cut-off frequency increases , Performance improvement , The smoothing effect is not as good as the former , But there is no ringing , Most widely used

2.3.3 Homomorphic filtering

Subtractive multiplicity 、 Convolution noise （ Uneven illumination ）

• Illumination component ： Light variation , The whole space changes slowly （ Low frequency ）
• Reflection component ： The junction between objects changes sharply （ high frequency ）
• Homomorphic filtering ： Compress low frequency , Enhance high frequency , Stretch other parts to increase contrast
• be based on Retinex wave filtering ： Based on the principle of retina and cerebral cortex , People will first look for standard white light , Get the object information by calculating the color difference
  • Incident light ： Dynamic range of image gray level
  • Reflected light ： The inner essence of image （ To be asked ）

3. Morphological processing

• morphology ： A branch of biology that studies the structure of animals and plants
• Digital morphology ： Set theory method describes geometric structure
• Image mathematical morphology processing ： Analyze the image based on morphology
• Extract the boundary ： The original image is different from the image after morphological processing , You can get the boundary
• Hit miss change ： Identify shape , Use the structure to detect the inside and outside of the target shape

3.1 inflation

• A By B inflation ： Yes B Reflection of （ About the origin symmetry ） Translate , To that of the A The intersection of is not a set of empty points
• Fill the void
• Do with the operation

3.2 corrosion

• A By B corrosion ： take B translation z after , Be included in A Set of points for
• It can be used for matching （ Corrode into a dot ）
• Remove burrs
• Do or operation

3.3 Open operation

• Corrosion before expansion
• Eliminate small objects 、 Separate objects in slender
• Smooth the boundary of large objects , Do not significantly change the area

3.4 Closed operation

• Expand before corrode
• Fill the tiny hole in the object
• Connecting objects
• Smooth boundary , Do not significantly change the area

4. Image segmentation

• Region of interest segmentation - Region of interest recognition
• Area ： Have common attributes （ Grayscale 、 Color 、 texture 、 Pattern ） Connected set of pixels
• Image segmentation ： Divide the image into several disjoint small areas

4.1 Segmentation based on threshold

• Global threshold 、 Local thresholds 、 Dynamic threshold
• Threshold selection ：
  • Histogram technique ： High contrast between target and background , Set the threshold according to the histogram distribution to select the target area
  • Minimum error technique ： Minimize the probability that the target is misclassified into the background and the probability that the background is misclassified into the target （ You need to know the probability distribution of the target and background , The applicability is not strong ）
  • Maximum variance technique ： Maximize the variance between different segmentation regions （ High operability , Widely applied ）

4.2 Edge based segmentation

• Spot check
  • Templates ：$\left|\begin{array}{ccc}-1& -1& -1\\ -1& 8& -1\\ -1& -1& -1\end{array}\right|$
• Line detection
  • Templates ： Template for first-order derivation
• Image edge ： A collection of connected pixels , These pixels are located at the boundary of the two regions
• Edge extraction ： First or second derivative 、 High pass filtering
• Edge extraction ： check the accuracy 、 Positioning accuracy 、 Unilateral response （ Where one side exists, there is no multilateral ）
• Canny operator ：
  • Noise reduction ： Gaussian low pass filter
  • gradient ：4 Detect gradient in two directions
  • Non maximum suppression ： Find the local maximum along the gradient
  • Edge tracking ： Double threshold detection 、 Track edges
• Hough transform
  • Edge discontinuity （ Effects of lighting or noise ）
  • Find the most likely straight line / Curve equation
  • Convert to parameter plane , The point with the most intersecting lines is the parameter of the line
  • Express a straight line in the form of polar coordinates （ Because the slope is infinite when it is vertical ）
  • Use intersection accumulators or histograms
  • It can be used to correct bills
  • advantage ： Anti noise 、 The signal-to-noise ratio is required to be low 、 Detect straight lines or analytical curves
  • shortcoming ： Binarization and edge detection are needed first , A lot of information will be lost

4.3 Region based segmentation

• Regional growth method , In small areas （ Seed pixels ） Expand outward , Segment similar areas
• Determine seed pixels ：
  • human-computer interaction （ Medical images are widely used ）
• Judge the similarity ：
  • Based on gray difference
  • Based on the regional gray distribution characteristics ： Compare cumulative gray histogram （Kolmogorov-Smirnov testing ,Smoothed-Difference testing ）
• Regional division and consolidation ： Split until the variance is 0, Consistent variance merging

4.4 Segmentation based on learning