Analysis of openh264 decoded data flow

openh264dec in finish_frame Logic and avviddec It's simpler , stay handle_frame in ,buffer First send it to the decoder for decoding （DecodeFrameNoDelay）, And then call gst_video_decoder_finish_frame Will decode the buffer Send to downstream.

DecodeFrameNoDelay decode , return yuvdata, After completion unmap input_buffer：

openh264dec->decoder->DecodeFrameNoDelay (map_info.data, map_info.size,
                                          yuvdata, &dst_buf_info);
gst_buffer_unmap (frame->input_buffer, &map_info);

gst_video_frame_map The function of

gst_video_frame_map() Will be in GstVideoFrame Fill in the structure buffer Pixels and all kinds of video information required .

gboolean
gst_video_frame_map (GstVideoFrame * frame, GstVideoInfo * info, GstBuffer * buffer, GstMapFlags flags)

Use @info and @buffer To fill in @frame, The address will be passed to the #GstVideoFrame structure .

You can use the accessor macro to access , Such as ：

 GST_VIDEO_FRAME_COMP_DATA()
 GST_VIDEO_FRAME_PLANE_DATA()
 GST_VIDEO_FRAME_COMP_STRIDE()
 GST_VIDEO_FRAME_PLANE_STRIDE()

The purpose of this function is to make it easy for you to get video pixels , Don't worry about too many details .

For example, is video data allocated in a continuous memory block or multiple memory blocks ( for example , For each of these plane There is a memory block ), Or whether to use customization strides And customization plane The offset （ By GstVideoMeta Record ）.

This function just fills in with the correct value #GstVideoFrame structure , Use accessor macros , Data can be easily accessed . It also maps the underlying memory blocks for you .

gst_video_frame_map Use of from ffmpeg avviddec and openh264dec You can also see the implementation of , It is convenient to put the data decoded by the decoder into plane In the memory block .

GstVideoFrame The definition of

/** * GstVideoFrame: * @info: the #GstVideoInfo * @flags: #GstVideoFrameFlags for the frame * @buffer: the mapped GstBuffer's buffer * @meta: pointer to metadata if any * @id: id of the mapped frame. the id can for example be used to * identify the frame in case of multiview video. * @data: pointers to the plane data * @map: mappings of the planes * * A video frame obtained from gst_video_frame_map() */
struct _GstVideoFrame {
    
  GstVideoInfo info;
  GstVideoFrameFlags flags;

  GstBuffer *buffer;
  gpointer   meta;
  gint       id;

  gpointer   data[GST_VIDEO_MAX_PLANES];
  GstMapInfo map[GST_VIDEO_MAX_PLANES];

  /*< private >*/
  gpointer _gst_reserved[GST_PADDING];
};

gst_video_decoder_allocate_output_frame

Auxiliary function , by @decoder Current #GstVideoCodecState Allocate a buffer to hold a video frame . The subclass should have configured the video state and set src pad caps. from openH264dec and avdec_h264 You can see , Finally, the decoded data must first allocate_output_frame, then copy Data into this ,avviddec The processing inside needs to see get_output_buffer Implementation of function .

gst_video_decoder_finish_frame

stay openh264dec in , adopt gst_video_decoder_finish_frame Decoded data push To downstream, If no output data is provided ,frame Be regarded as skipped. In any case ,frame Are considered finished and released.

gst_openh264dec_handle_frame

• received frame Then hand it over to openh264dec decode ->DecodeFrameNoDelay
• gst_video_frame_map： mapping frame->output_buffer, What will be decoded yuvdata Fill in .
• gst_video_decoder_set_output_state： stay srcpad Create a new codec state As the output state of the decoder
• gst_video_decoder_finish_frame Send to downstream

because output state Part of the configuration is I420, So we are doing data copy When , yes yuv successively copy：

// i = 0; Y plane
// i = 1; U plane
// i = 2; V plane
for (i = 0; i < 3; i++) {
    
    p = GST_VIDEO_FRAME_COMP_DATA (&video_frame, i);
    row_stride = GST_VIDEO_FRAME_COMP_STRIDE (&video_frame, i);
    component_width = GST_VIDEO_FRAME_COMP_WIDTH (&video_frame, i);
    component_height = GST_VIDEO_FRAME_COMP_HEIGHT (&video_frame, i);
    src_width =
        i <
        1 ? dst_buf_info.UsrData.sSystemBuffer.
        iStride[0] : dst_buf_info.UsrData.sSystemBuffer.iStride[1];
    for (row = 0; row < component_height; row++) {
    
        memcpy (p, yuvdata[i], component_width);
        p += row_stride;
        yuvdata[i] += src_width;
    }
}