Cann media data processing V2, jpegd interface introduction

This is the second blog about media data processing , About JPEG decode

Portal first article

---

1. flow chart
   

Note that this picture has the universe , What we have been exposed to before is media data processing V1 still v2 Other functional interfaces of , You can choose between single thread and multi thread , And look at this picture carefully , Indicates that the user needs to restart a new thread , To receive the decoded image . If only in one thread , The final result is invalid data .

2. API Introduce

• Receive code stream , Stop receiving code stream interface

//  Prototype hi_s32 hi_mpi_vdec_start_recv_stream(hi_vdec_chn chn)hi_s32 hi_mpi_vdec_stop_recv_stream(hi_vdec_chn chn) Parameters hi_vdec_chn chn： It's rising 710 AI On the processor , The value range of this parameter ：[0, 256), The maximum number of channels is 256.

• Send decoded data interface

//  Prototype hi_s32 hi_mpi_vdec_send_stream(hi_vdec_chn chn, const hi_vdec_stream *stream, hi_vdec_pic_info *vdec_pic_info, hi_s32 milli_sec)//  Parameters hi_vdec_stream *stream ： Pointer to input bitstream information , In the structure addr The address of parameter configuration is Device Memory address on .typedef struct {    hi_bool end_of_frame; Whether the current frame ends （ This value is currently invalid ）    hi_bool end_of_stream; Whether all code streams have been sent （ Pay attention when sending valid frames and end frames ）    hi_bool need_display; Whether the current frame output is displayed （ Whether to save to output memory ）0 Do not save ,1 preservation     hi_u64 pts; Timestamp of the bitstream packet , With us In units of . Valid only when sent by frame equals to the timestamp in the code stream packet , When sending by stream , The timestamp of the decoded image is equal to 0    hi_u64 private_data; Private data . Reserved parameters     hi_u32 len; Length of bitstream packet , With Byte In units of     hi_u8 *ATTRIBUTE addr; The address of the stream packet } hi_vdec_stream;hi_vdec_pic_info *vdec_pic_info： Pointer to output picture information typedef struct {    hi_u32 width; //  The original width of the image     hi_u32 height; //  The original height of the image     hi_u32 width_stride; //  Image alignment width     hi_u32 height_stride; //  Image alignment height     hi_pixel_format pixel_format; //  Pixel image format     hi_u64 vir_addr; //  Decode the starting address of output data in memory     hi_u32 buffer_size; //  The size of data in memory after width height alignment     hi_s16 offset_top; //  Image top clipping width .（ be used for JPEG Area decoding , Currently does not support ）    hi_s16 offset_bottom; //  Image bottom clipping width .（ be used for JPEG Area decoding , Currently does not support ）    hi_s16 offset_left; //  Left clipping width of image .（ be used for JPEG Area decoding , Currently does not support ）    hi_s16 offset_right; //  Clipping width on the right side of the image .（ be used for JPEG Area decoding , Currently does not support ）}hi_vdec_pic_info;

• Receive decoded data interface (JPEGD The image decoding rank only needs to pay attention to these annotated parameters. Other parameters may not be applicable to JPEG Decoding or currently reserved parameters , No internal processing )

//  Prototype hi_s32 hi_mpi_vdec_get_frame(hi_vdec_chn chn, hi_video_frame_info *frame_info, hi_vdec_supplement_info *supplement, hi_vdec_stream *stream, hi_s32 milli_sec)//  Parameters hi_video_frame_info *frame_info： Decoded image frame information structure .typedef struct {    hi_video_frame v_frame;  The structure for storing decoded image frames .    hi_u32        pool_id;    hi_mod_id      mod_id;} hi_video_frame_info;typedef struct {    hi_u32              width; //  The original width of the image     hi_u32              height; //  The original height of the image     hi_video_field      field;    hi_pixel_format     pixel_format; //  Video image pixel format     hi_video_format     video_format;    hi_compress_mode    compress_mode;    hi_dynamic_range    dynamic_range;    hi_color_gamut      color_gamut;    hi_u32              header_stride[HI_MAX_COLOR_COMPONENT];    hi_u32              width_stride[HI_MAX_COLOR_COMPONENT]; //  Image alignment width     hi_u32              height_stride[HI_MAX_COLOR_COMPONENT]; //  Image alignment height     hi_u64              header_phys_addr[HI_MAX_COLOR_COMPONENT];    hi_u64              phys_addr[HI_MAX_COLOR_COMPONENT];    hi_void* ATTRIBUTE  header_virt_addr[HI_MAX_COLOR_COMPONENT];    hi_void* ATTRIBUTE  virt_addr[HI_MAX_COLOR_COMPONENT];  The image is in Device Starting virtual address in memory .    hi_u32              time_ref;    hi_u64              pts; //  Image timestamp .    hi_u64              user_data[HI_MAX_USER_DATA_NUM];    hi_u32              frame_flag;    hi_video_supplement supplement;} hi_video_frame;

---

Some understanding of concatenation between media data processing components

In real programming scenarios , It is unlikely that only one component , It is possible to face the situation that the components are connected in series . for example
JPEG Image decoding ==》 The zoom 、 tailoring 、 Cutout ==》 Into the model ==》 Encoding preservation
Video decoding ==》 The zoom 、 tailoring 、 Cutout ==》 Into the model ==》 Encoding preservation
So memory reuse is particularly important in this process , Although the contents can be saved to the hard disk first and then read again , But it's obviously inefficient .
So there are a few things you need to know about memory reuse ：

• Image data we are waiting to process , Is it exactly the same as the data we originally stored on the hard disk ？
  The answer is not always ！！
  
  The above picture is enough to explain why not , Because the width and height of some images just meet the requirements of alignment , Therefore, it is consistent with the source data , In case of non-compliance, invalid green edges will be generated and filled with data that meets the requirements , This is why sometimes we violently save the data in the memory, and there is a green edge on the lower right side .

• The input and output width and height alignment requirements of different components are different , Incompatibilities are inevitable , Then how do I fill in the width and height of it ？
  The answer is the alignment requirements with the previous component
  Mr. Mao's analysis of memory reuse in media data processing