Introduction to color coding format

Simple classification of color coding formats

• YUV
  • YUV444
  • YUV 4:2:2
  • YUV 4:2:0
    • YUV420sp
      • NV12
      • NV21
    • YUV420p
      • YU12
      • YV12
• RGB

Grading introduction

YUV, It's divided into three parts ,“Y” Indicates brightness （Luminance or Luma）, That is, gray value ; and “U” and “V” It's color （Chrominance or Chroma）, It is used to describe image color and saturation , Used to specify the color of pixels . Mainly used in TV system and analog video field , It will send the brightness information （Y） And color information （UV） Separate , No, UV Information can also display complete images , It's just black and white , This design solves the problem of compatibility between color TV and black and white TV . also ,YUV Unlike RGB That requires three independent video signals to be transmitted at the same time , So use YUV Mode transfer takes up very little bandwidth .

YUV Two major categories of formats

• planar : First, store all pixels in a row Y, And then it stores all the pixels U, And then all the pixels V.
• packed : Every pixel point Y,U,V It's continuous * Stored .

frequently-used YUV Format :

• YUV444: every last Y Corresponding to a group of UV component , 1 Pixel rgb The memory occupied by data is 8 + 8 + 8 = 24bit = 3byte;
• YUV422: Every two Y Share a group UV component , 1 Pixel rgb The memory occupied by data is 8 + 4 + 4 = 16bit = 2byte;
• YUV420: Every four Y Share a group UV component , 1 Pixel rgb The memory occupied by data is 8 + 2 + 2 = 12bit = 1.5byte;

YUV420p

YUV420p： Also called planer Flat pattern ,Y ,U,V And then different planes , That is, there are three planes .

• YU12 (I420, Android mode ) : The storage order is save first Y, Save again U, Last deposit V. Such as :YYYYYYYYUUVV;
• YV12: The storage order is save first Y, Save again U, Last deposit V. Such as : YYYYYYYYVVUU;

YUV420sp

YUV420sp： Also called bi-planer or two-planer Biplane ,Y A plane ,UV Cross store on the same plane .

• NV12 (iOS There is only this mode in ) : The storage order is save first Y, Again UV Alternate storage . Such as :YYYYYYYYUVUV;
• NV21( Android mode ): The storage order is save first Y, Again VU Alternate storage . Such as : YYYYYYYYVUVU;

Calculate the size

Y component width * height = Y( The sum of the )
U component Y/4 = U; V component = Y/ 4;
therefore YUV420 The length of the data in memory is width * height * 3 / 2 ( One YUV Occupy 1.5 Bytes ); So the size of a data collection is width * height * 1.5 * Frame rate * time;
Generally speaking , The video data collected directly is RGB24 The format of ,RGB24 The size of a frame size＝width×heigth×3 byte ;
So it is usually in the collection of RGB24 after , You need to compress the amount data in this format , Change the color space of the image from RGB To YUV; This will not only reduce the amount of data , and X264 When coding , Need standard YUV420 Format ; then adopt Some coding algorithms , The data will be greatly reduced , then Package the encoded data , Conduct RTP Real time transmission , At the receiving end , After receiving the data, decode it and pass OpenGL Render to screen , Or will YUV420 To RGB24 Post render to screen ;

stay iOS in

stay iOS in Use The camera perhaps Create decoder , Or create a render buffer ; All need to specify PixelFormatType; however iOS By default, the medium and hard disk decoding only supports NV12 The format of , stay PixelFormatType In the enumeration of , There are four

• kCVPixelFormatType_420YpCbCr8Planar
• kCVPixelFormatType_420YpCbCr8PlanarFullRange
• kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange
• kCVPixelFormatType_420YpCbCr8BiPlanarFullRange

Literally , The first one is YUV420p The format of , It's three planes The last two are two planes , Or is it call CVPixelBufferGetPlaneCount（pixel） Function to get the number of planes , The same conclusion will be reached ; But the scope is narrowed down to the last two ;

If the video The resolution is 480*640 ( The difference is The calculation steps are different ) Finally decide to use kCVPixelFormatType_420YpCbCr8BiPlanarFullRange;

• kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange : Y and UV In steps of 480（ Actual length , Not completed ） Y The row width of is 640,UV Row width is 320
• kCVPixelFormatType_420YpCbCr8BiPlanarFullRange : Y and UV In steps of 512（ Adopted 64 Byte alignment Misaligned complement 0） Y The row width of is 640,UV Row width is 320