ffmpeg pixel format basics

ffmpeg Pixel format basics

----------------------------------------
甲. YUV Color space naming meaning
----------------------------------------
YComponents are not compressed,对UVThe chroma components are compressed to some extent.
YUV4:4:4
YUV4:4:0
YUV4:2:2
YUV4:2:0
YUV4:1:1
YUV4:1:0
具体的含义是什么？

考虑一个4*2的色块,共8个像素,2行（奇数行,偶数行）
The first value is always called4,Indicates that the luminance is sampled in both odd and even rows4次,is always sampled.
The second value is called4,2或1, The first row representing the chrominance data is sampled4次,2次或1次,The chromaticity here stands forU或V
The third value is called4,2或1,0, The second row representing the chrominance data is sampled4次,2次或1次,或0次.

the same sampling method,There are three ways to store data in memory：
1.packed: 按YUYVYUYV...intertwined for storage
2.planar: 按YYYYY...,UUUUU...,VVVV...分开存放
3.semi_planar: 按YYYY.... UVUVUV....分开存放

定义1：AVPixelFormat:         Defines the pixel format ID

enum AVPixelFormat {
    AV_PIX_FMT_NONE = -1,
    AV_PIX_FMT_YUV420P,   ///< planar YUV 4:2:0, 12bpp, (1 Cr & Cb sample per 2x2 Y samples)
    AV_PIX_FMT_YUYV422,   ///< packed YUV 4:2:2, 16bpp, Y0 Cb Y1 Cr
    AV_PIX_FMT_RGB24,     ///< packed RGB 8:8:8, 24bpp, RGBRGB...
    AV_PIX_FMT_BGR24,     ///< packed RGB 8:8:8, 24bpp, BGRBGR...
    AV_PIX_FMT_YUV422P,   ///< planar YUV 4:2:2, 16bpp, (1 Cr & Cb sample per 2x1 Y samples)
    ...                //忽略
    };

定义2：AVPixFmtDescriptor: This data structure defines how image data is organized in memory

(gdb) ptype av_pix_fmt_descriptors
  type = const struct AVPixFmtDescriptor {
      const char *name;
      uint8_t nb_components;
      uint8_t log2_chroma_w;
      uint8_t log2_chroma_h;
      uint64_t flags;
      AVComponentDescriptor comp[4];
      const char *alias;
  } [198]
  这198This kind of pixel definition I do notcopy了.See below“Part B description"

Below is a description of its members：
const char *name : Pixel format name
uint8_t nb_components: The number of pixel components, 取值范围 1 - 4.

例如 AV_PIX_FMT_GRAY8 只有 Y an amount,
AV_PIX_FMT_YUV420P 有 Y、U、V 三个分量,
AV_PIX_FMT_NV12 也有 Y、U、V 三个分量,
AV_PIX_FMT_ARGB 有 A、R、G、B 四个分量.

uint8_t log2_chroma_w:Horizontal chroma subsampling factor.

右移的位数.  Indicates how many bits to shift the width of the luma sample to the right to obtain the width of the chroma sample.
例如对于 yuv420p 格式,If the image resolution is 1280 x 720,
is the luminance sample width(The number of luminance samples in the horizontal direction)为 1280,
Chroma sample width(The number of chroma samples in the horizontal direction)为 1280/2 = 640,
则 log2_chroma_w 值为 1(右移 1 位).

uint8_t log2_chroma_h: Vertical chroma subsampling factor.

uint64_t flags: Pixel format flag bit combination,
例如如 AV_PIX_FMT_FLAG_BE ｜ AV_PIX_FMT_FLAG_HWACCEL .
标志 AV_PIX_FMT_FLAG_BE Represents a big endian format.
AV_PIX_FMT_FLAG_HWACCEL Indicates that this pixel format is used for hardware acceleration such as hard decoding or hard coding.

AVComponentDescriptor comp[4]
这个成员非常重要.Each element of the array represents a component,Note that it is one component and not one plane,一个 plane May contain multiple components

定义3：AVComponentDescriptor, AVComponent descriptor

(gdb) ptype av_pix_fmt_descriptors[0].comp
  type = struct AVComponentDescriptor {
      int plane;
      int step;
      int offset;
      int shift;
      int depth;
      int step_minus1;        // deprecated
      int depth_minus1;        // deprecated
      int offset_plus1;        // deprecated
  } [4]
AVComponentDescriptor Defines the actual organization of each component in memory,All details are included.

Included members are as follows,
int plane : Where the current component is located plane 中.

例如 p010 The format has three components：Y、U、V,两个 plane：Y、UV.
Y plane 的形式为YYYY...,UV plane 的形式为UVUVUV....
Y 分量的 plane 值是 0, U 分量和 V 分量的 plane 值是 1,U 样本和 V Sample interleaving is stored in plane 1中.

int step : 步长,Indicates the number of bytes between two consecutive samples in the horizontal direction(或比特),
If the pixel format is a bitstream format(标志 AV_PIX_FMT_FLAG_BITSTREAM 有效),This value represents the number of bits,Otherwise this value represents the number of bytes.

例如: p010 格式,Y plane 的形式为YYYY...,UV plane 的形式为UVUVUV...,位深是 10,
After considering the alignment,每一个 Y、每一个 U、每一个 V 都占 2 个字节,
因此 Y 分量的 step 是 2(两个 Y two bytes apart),
U 分量的 step 是 4(两个 U 相距 4 字节),
V 分量的 step 也是 4(两个 V 相距 4 字节).

int offset : 偏移,表示在当前 plane 中,How many bytes of data there are before the first sample of the current component,
If the pixel format is a bitstream format(标志 AV_PIX_FMT_FLAG_BITSTREAM 有效),This value represents the number of bits,Otherwise this value represents the number of bytes.

例如 p010 格式,每一个 U 或 V 都占 2 个字节,第一个 V sample before 2 个字节被 U sample accounted for,
所以 U 分量的 offset 值是 0,V 分量的 offset 值是 2.

int shift : 右移位数,Indicates how many bits to right-shift the value of the corresponding memory cell to get the actual value.

例如 p010 格式,位深是 10,And the memory is aligned after each one Y、U、V Sample accounts 16 bit,
那么 10 bits of data are placed 16 位的内存单元中, is occupying high 10 The bit is still occupied low 10 位,即是由 shift 值决定的.
p010 格式中,各分量的 shift 值都是 6 ,Indicates that the data is placed high 10 位.

从 Y plane 中获取第一个 Y 样本的值,示意代码如下：
uint8_t y_plane[1280*2];
uint16_t *p_y0 = (uint16_t *)y_plane;
uint16_t y0 = (*p_y0) >> 6;

int depth: The width in bits per sample of the current component,即位深.
上述参数中,
plane Indicates where the component is plane 的序号,
offset Indicates that multiple components are interleaved and stored in the same one plane The order of the middle time(如 p010 格式的 UV plane 中 U 在前 V 在后),
step、shift 和 depth It is related to memory alignment

后面的3member has been deprecated,从略.

----------------------------------------
乙： ffmpeg image used inpix_fmts
----------------------------------------

$ffmpeg -pix_fmts , 可以查看所有的pix_fmts,
The key is to define the following table

static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = {
    [AV_PIX_FMT_YUV420P] = {
        .name = "yuv420p",
        .nb_components = 3,        //3个平面YUV
        .log2_chroma_w = 1,        //Color width is small1倍 (>>1)
        .log2_chroma_h = 1,        //small color1倍 (>>1)
        .comp = {
            { 0, 1, 0, 0, 8, 0, 7, 1 },        /* Y */  Bright and deep is8bit, 后3Item is no longer needed,前4项plane,step,offset,shift
            { 1, 1, 0, 0, 8, 0, 7, 1 },        /* U */  色深1是8bit
            { 2, 1, 0, 0, 8, 0, 7, 1 },        /* V */  色深2是8bit
        },
        .flags = AV_PIX_FMT_FLAG_PLANAR,
    },
    [AV_PIX_FMT_YUYV422] = {
        .name = "yuyv422",
        .nb_components = 3,     //3个平面YUV
        .log2_chroma_w = 1,     //Color width is small1倍 (>>1)
        .log2_chroma_h = 0,     //Color height is normal (>>0)
        .comp = {
            { 0, 2, 0, 0, 8, 1, 7, 1 },        /* Y */
            { 0, 4, 1, 0, 8, 3, 7, 2 },        /* U */
            { 0, 4, 3, 0, 8, 3, 7, 4 },        /* V */
        },
    },
    .....
}

----------------------------------------
丙： ffmpeg audio used in sample_fmt
----------------------------------------
$ffmpeg -sample_fmts  All audio sample formats defined can be viewed.

关键是在samplefmt.c中, The following table is defined
static const SampleFmtInfo sample_fmt_info[AV_SAMPLE_FMT_NB] = {
    [AV_SAMPLE_FMT_U8]   = { .name =   "u8", .bits =  8, .planar = 0, .altform = AV_SAMPLE_FMT_U8P  },
    [AV_SAMPLE_FMT_S16]  = { .name =  "s16", .bits = 16, .planar = 0, .altform = AV_SAMPLE_FMT_S16P },
    [AV_SAMPLE_FMT_S32]  = { .name =  "s32", .bits = 32, .planar = 0, .altform = AV_SAMPLE_FMT_S32P },
    [AV_SAMPLE_FMT_S64]  = { .name =  "s64", .bits = 64, .planar = 0, .altform = AV_SAMPLE_FMT_S64P },
    [AV_SAMPLE_FMT_FLT]  = { .name =  "flt", .bits = 32, .planar = 0, .altform = AV_SAMPLE_FMT_FLTP },
    [AV_SAMPLE_FMT_DBL]  = { .name =  "dbl", .bits = 64, .planar = 0, .altform = AV_SAMPLE_FMT_DBLP },
    [AV_SAMPLE_FMT_U8P]  = { .name =  "u8p", .bits =  8, .planar = 1, .altform = AV_SAMPLE_FMT_U8   },
    [AV_SAMPLE_FMT_S16P] = { .name = "s16p", .bits = 16, .planar = 1, .altform = AV_SAMPLE_FMT_S16  },
    [AV_SAMPLE_FMT_S32P] = { .name = "s32p", .bits = 32, .planar = 1, .altform = AV_SAMPLE_FMT_S32  },
    [AV_SAMPLE_FMT_S64P] = { .name = "s64p", .bits = 64, .planar = 1, .altform = AV_SAMPLE_FMT_S64  },
    [AV_SAMPLE_FMT_FLTP] = { .name = "fltp", .bits = 32, .planar = 1, .altform = AV_SAMPLE_FMT_FLT  },
    [AV_SAMPLE_FMT_DBLP] = { .name = "dblp", .bits = 64, .planar = 1, .altform = AV_SAMPLE_FMT_DBL  },
};

(gdb) ptype sample_fmt_info
  type = const struct SampleFmtInfo {
      char name[8];
      int bits;
      int planar;
      enum AVSampleFormat altform;
  } [12]