Ffmpeg (I) AV_ register_ all()

av_register_all

FFmpeg in av_register_all() The function is used to register all muxers、demuxers And protocols.FFmpeg4.0 Used to store with a linked list muxer/demuxer,FFmpeg4.0 Later, it will be stored in array , also av_register_all Method has been marked obsolete ,av_register_input_format and av_register_output_format Also marked obsolete .

av_register_all() The statement for is located at libavformat/avformat.h Header file , In the new version 4.0 in the future , You don't need to call this method , All modules can be used directly .

The document mentions avfilter modular ：avfilter_register()、avfilter_register_all() and avfilter_next() Deprecated , add to av_filter_iterate() To iterate through .avformat modular ：av_register_input_format()、av_register_output_format()、av_register_all() Etc. are out of date , add to av_demuxer_iterate() and av_muxer_iterate() To iterate through .avcodec modular ：avcodec_register()、avcodec_register_all()、av_register_codec_parser() Etc. are out of date , add to av_codec_iterae() and av_parser_iterate() To iterate through .

avformat Module registration function changes , Change linked list storage to array storage , Add... At run time muxer And demuxer To the linked list , It is automatically generated at compile time and saved in the array .

Generally we use FFMpeg When doing encoding and decoding, you will call av_register_all() This function starts with , Complete the basic initialization work . As for what it initializes , Let's look directly at the code .

    void av_register_all(void)
    {
        static int initialized; // Flag bit indicates whether it has been initialized 

        if (initialized)
            return;
        initialized = 1;

        avcodec_register_all(); //  Register codecs 

        /* (de)muxers */  // Register multiplexer and demultiplexer 
        REGISTER_MUXER   (A64,              a64);
        REGISTER_DEMUXER (AA,               aa);
        REGISTER_DEMUXER (AAC,              aac);
        REGISTER_MUXDEMUX(AC3,              ac3);
        REGISTER_DEMUXER (ACM,              acm);
        REGISTER_DEMUXER (ACT,              act);
        ..........
    }

Speaking of this , It should be explained first FFMpeg The design idea inside . Register these codecs and with us here （ Explain ） Multiplexer at FFMpeg What is the concept in the source code of .
stay FFMepg in , Like codec , They are all maintained by a global linked list . Registration is actually adding nodes to these global linked lists , Each node maintains the relevant information of an encoder or decoder and the interoperable function pointers .
Understand the concept of such a registration . So we can take a look at the code that follows . First look at avcodec_register_all() This function .

void avcodec_register_all(void)
{
    static int initialized;
    if (initialized)
        return;
    initialized = 1;

    /* hardware accelerators */
    REGISTER_HWACCEL(H263_VAAPI,        h263_vaapi);
    REGISTER_HWACCEL(H263_VIDEOTOOLBOX, h263_videotoolbox);
    REGISTER_HWACCEL(H264_D3D11VA,      h264_d3d11va);
    REGISTER_HWACCEL(H264_DXVA2,        h264_dxva2);
    REGISTER_HWACCEL(H264_MMAL,         h264_mmal);
    REGISTER_HWACCEL(H264_QSV,          h264_qsv);
    REGISTER_HWACCEL(H264_VAAPI,        h264_vaapi);
    REGISTER_HWACCEL(H264_VDA,          h264_vda);
    REGISTER_HWACCEL(H264_VDA_OLD,      h264_vda_old);
    ............

    /* video codecs */
    REGISTER_ENCODER(A64MULTI,          a64multi);
    REGISTER_ENCODER(A64MULTI5,         a64multi5);
    REGISTER_DECODER(AASC,              aasc);
    REGISTER_DECODER(AIC,               aic);
    REGISTER_ENCDEC (ALIAS_PIX,         alias_pix);
    REGISTER_ENCDEC (AMV,               amv);
    REGISTER_DECODER(ANM,               anm);
    REGISTER_DECODER(ANSI,              ansi);
    REGISTER_ENCDEC (APNG,              apng);
    REGISTER_ENCDEC (ASV1,              asv1);
    .............

}

Then let's take a look at REGISTER_HWACCEL(H263_VIDEOTOOLBOX, h263_videotoolbox); What has this code done .

#define REGISTER_HWACCEL(X, x)                                          \
{                                                                   \
    extern AVHWAccel ff_##x##_hwaccel;                              \
    if (CONFIG_##X##_HWACCEL)                                       \
        av_register_hwaccel(&ff_##x##_hwaccel);                     \
}

Obviously this is a macro . If it is expanded and restored according to the above sentence, it should be ：

extern AVHWAccel ff_h263_videotoolbox_hwaccel; 
if (CONFIG_H263_VIDEOTOOLBOX_HWACCEL)
    av_register_hwaccel(&ff_videotoolbox_hwaccel);

The first sentence indicates that such a variable should have been defined elsewhere ff_h263_videotoolbox_hwaccel Then we search this variable in the source code and find , Don't define it in libavcodec/videotoolbox.c Inside .

AVHWAccel ff_h263_videotoolbox_hwaccel = {
    .name           = "h263_videotoolbox",
    .type           = AVMEDIA_TYPE_VIDEO,
    .id             = AV_CODEC_ID_H263,
    .pix_fmt        = AV_PIX_FMT_VIDEOTOOLBOX,
    .alloc_frame    = ff_videotoolbox_alloc_frame,
    .start_frame    = videotoolbox_mpeg_start_frame,
    .decode_slice   = videotoolbox_mpeg_decode_slice,
    .end_frame      = videotoolbox_mpeg_end_frame,
    .uninit         = ff_videotoolbox_uninit,
    .priv_data_size = sizeof(VTContext),
};

Complete code as above , So let's see AVHWAccel The definition of this type .

/**
 * @defgroup lavc_hwaccel AVHWAccel
 * @{
 */
typedef struct AVHWAccel {
    /**
     * Name of the hardware accelerated codec.
     * The name is globally unique among encoders and among decoders (but an
     * encoder and a decoder can share the same name).
     */
    const char *name;

    /**
     * Type of codec implemented by the hardware accelerator.
     *
     * See AVMEDIA_TYPE_xxx
     */
    enum AVMediaType type;

    /**
     * Codec implemented by the hardware accelerator.
     *
     * See AV_CODEC_ID_xxx
     */
    enum AVCodecID id;

    /**
     * Supported pixel format.
     *
     * Only hardware accelerated formats are supported here.
     */
    enum AVPixelFormat pix_fmt;

    /**
     * Hardware accelerated codec capabilities.
     * see HWACCEL_CODEC_CAP_*
     */
    int capabilities;

    /*****************************************************************
     * No fields below this line are part of the public API. They
     * may not be used outside of libavcodec and can be changed and
     * removed at will.
     * New public fields should be added right above.
     *****************************************************************
     */
    struct AVHWAccel *next;

    /**
     * Allocate a custom buffer
     */
    int (*alloc_frame)(AVCodecContext *avctx, AVFrame *frame);

    /**
     * Called at the beginning of each frame or field picture.
     *
     * Meaningful frame information (codec specific) is guaranteed to
     * be parsed at this point. This function is mandatory.
     *
     * Note that buf can be NULL along with buf_size set to 0.
     * Otherwise, this means the whole frame is available at this point.
     *
     * @param avctx the codec context
     * @param buf the frame data buffer base
     * @param buf_size the size of the frame in bytes
     * @return zero if successful, a negative value otherwise
     */
    int (*start_frame)(AVCodecContext *avctx, const uint8_t *buf, uint32_t buf_size);

    /**
     * Callback for each slice.
     *
     * Meaningful slice information (codec specific) is guaranteed to
     * be parsed at this point. This function is mandatory.
     * The only exception is XvMC, that works on MB level.
     *
     * @param avctx the codec context
     * @param buf the slice data buffer base
     * @param buf_size the size of the slice in bytes
     * @return zero if successful, a negative value otherwise
     */
    int (*decode_slice)(AVCodecContext *avctx, const uint8_t *buf, uint32_t buf_size);

    /**
     * Called at the end of each frame or field picture.
     *
     * The whole picture is parsed at this point and can now be sent
     * to the hardware accelerator. This function is mandatory.
     *
     * @param avctx the codec context
     * @return zero if successful, a negative value otherwise
     */
    int (*end_frame)(AVCodecContext *avctx);

    /**
     * Size of per-frame hardware accelerator private data.
     *
     * Private data is allocated with av_mallocz() before
     * AVCodecContext.get_buffer() and deallocated after
     * AVCodecContext.release_buffer().
     */
    int frame_priv_data_size;

    /**
     * Called for every Macroblock in a slice.
     *
     * XvMC uses it to replace the ff_mpv_decode_mb().
     * Instead of decoding to raw picture, MB parameters are
     * stored in an array provided by the video driver.
     *
     * @param s the mpeg context
     */
    void (*decode_mb)(struct MpegEncContext *s);

    /**
     * Initialize the hwaccel private data.
     *
     * This will be called from ff_get_format(), after hwaccel and
     * hwaccel_context are set and the hwaccel private data in AVCodecInternal
     * is allocated.
     */
    int (*init)(AVCodecContext *avctx);

    /**
     * Uninitialize the hwaccel private data.
     *
     * This will be called from get_format() or avcodec_close(), after hwaccel
     * and hwaccel_context are already uninitialized.
     */
    int (*uninit)(AVCodecContext *avctx);

    /**
     * Size of the private data to allocate in
     * AVCodecInternal.hwaccel_priv_data.
     */
    int priv_data_size;
} AVHWAccel;

You can find The protected operation functions are ：alloc_frame , start_frame, decode_slice, end_frame, decode_mb, init, uninit . from ff_h263_videotoolbox_hwaccel We can find in the definition of , It's worth it alloc_frame, start_frame, decode_slice, end_frame, uninit Here are a few ways .

Let's look at it in detail av_register_hwaccel(&ff_videotoolbox_hwaccel) What does this function do .

void av_register_hwaccel(AVHWAccel *hwaccel)
{
    AVHWAccel **p = last_hwaccel;
    hwaccel->next = NULL;
    while(*p || avpriv_atomic_ptr_cas((void * volatile *)p, NULL, hwaccel))
        p = &(*p)->next;
    last_hwaccel = &hwaccel->next;
}

This is very straightforward , Is doing a linked list addition operation . And what is the definition of its linked list ？ as follows ：

static AVHWAccel *first_hwaccel = NULL;
static AVHWAccel **last_hwaccel = &first_hwaccel;

such , The entire hardware accelerator registration process is completed . With first_hwaccel Is the head node .

alike , We can find out , Codec registration is also the same routine , Only the codec uses AVCodec This structure represents . alike , Define a global ff_x_encoder perhaps ff_x_decoder, Then in their respective implementation files , Realization AVCodec The method defined in . The function pointer is assigned to ff_x_encode perhaps ff_x_decoder. Corresponding Parser This is how you register .
So down , All the registration work is finished ,avcodec_register_all Then you can retire with success . The result is several initialized global linked lists .

notes ： If we want to go to FFMpeg Add a codec inside . Then we just need to implement the corresponding methods , And then in avcodec_register_all Complete the registration operation , In this way, the codec added by itself can be FFMpeg Used .

Okay , The analysis forgot avcodec_register_all() This code , Then the next step is to register some multiplexers and demultiplexers . This one mainly looks at three macros ：

#define REGISTER_MUXER(X, x)                                            \
{                                                                   \
    extern AVOutputFormat ff_##x##_muxer;                           \
    if (CONFIG_##X##_MUXER)                                         \
        av_register_output_format(&ff_##x##_muxer);                 \
}

#define REGISTER_DEMUXER(X, x)                                          \
{                                                                   \
    extern AVInputFormat ff_##x##_demuxer;                          \
    if (CONFIG_##X##_DEMUXER)                                       \
        av_register_input_format(&ff_##x##_demuxer);                \
}

#define REGISTER_MUXDEMUX(X, x) REGISTER_MUXER(X, x); REGISTER_DEMUXER(X, x)

among AVOutputFormat The structure represents the recombiner .AVInputFormat Represents a decomposing device . After the registration is completed, two will be generated, respectively first_iformat and first_oformat It is the global linked list of the head node .

thus , Whole av_register_all The work of the function is done .

av_register_all() The function is in libavformat/allformates.c In the definition of , For initialization libavformat And register all the wrappers , Unpacker and protocol . In the old version （3.x In the early ） Of ffmpeg This function is the beginning of almost all applications . Use version 4.x Of FFMPEG In many cases, this function is not directly called to register the encapsulation and de encapsulation , This function is also declared as deprecated 了 . But this does not prevent preschoolers from reading and learning this function . Because there are many places worth studying , There are many others ffmpeg Functions will be used in the interface and ideas .
Say more , What is unpacking ：
If you want to convert common video source data to pictures that can be seen by the naked eye on the TV screen or computer 、 Video frame , The following steps are required
Video source （RTSP,RTMP,HTTP, Ordinary documents, etc ） -> [ After the settlement of the agreement ] -> Video file or video stream （FLV,MKV,AVI etc. ）-> [ After unpacking ] -> Video data （H264、HEVC Video data ） And audio data （MP3,WMA etc. ）-> [ Video data decoded ] ->YUV Video frame , Last YUV Video frames can be displayed and played and captured by human eyes .

go back to av_register_all(), The code content of the function is very simple , Only the following lines

void av_register_all(void) {
    ff_thread_once(&av_format_next_init, av_format_init_next);
}

A very short function contains three elements ,ff_thread_once（ A function ）,av_format_next_init（ A macro defined variable ） as well as av_format_init_next（ A function ）.
The actual meaning of these three lines of code is to use the function ff_thread_once Function to execute the function av_format_init_next() function , And the first parameter defines the variable for the macro av_format_next_init.
It is worth noting that , Below this function are two functions ：

void av_register_input_format(AVInputFormat *format)
{
    ff_thread_once(&av_format_next_init, av_format_init_next);
}

void av_register_output_format(AVOutputFormat *format)
{
    ff_thread_once(&av_format_next_init, av_format_init_next);
}

Function content , That is, what they do is exactly the same , Just different names , It's registration inputformat and outputformat Function follows register_all What is done is exactly the same .

ff_thread_once()
ff_thread_once() Is defined in libavutil/thread.h For thread safety . stay thread.h In the document , In a os2threads.h library , Will pass the following define Point to pthread_once function ：

#define ff_thread_once(control, routine) pthread_once(control, routine)

In all cases , It creates its own function ：

static inline int ff_thread_once(char *control, void (*routine)(void)) {
    if (!*control) {
        routine();
        *control = 1;
    }
    return 0;
} // This function is also very easy to see through a control To guarantee routine Function will be executed only once .

So actually pthread_once()：thread.h Different thread libraries are introduced according to different platforms , Cut out unnecessary code , The order of import and stock in is shown in the following code ： Find priority to use pthread.h library -> secondly os2threads.h library -> Again w32pthreads.h.pthread_once Mapping to different thread libraries on different platforms pthread_once function .

So in general , The main function of this function is to ensure that the incoming function will only be executed once in a multi-threaded environment . Different process functions are used according to different platforms .

av_format_next_init
This variable is defined in allformat.c Inside the document ：

static AVOnce av_format_next_init = AV_ONCE_INIT;

AVOnce It's also defined in thread.h Internal

#define AVOnce pthread_once_t

stay thread.h Inside , In different circumstances ,AVOnce It also has different contents

#define AVOnce pthread_once_t
#define AVOnce char

This variable is also used to ensure that the incoming function will be executed only once in the case of multithreading , For different environments ,AVOnce Different variables are used to implement this function .

av_format_init_next()
Yes ff_thread_once() The function passed in is av_formate_init_next(), The function is as follows :

static void av_format_init_next(void)
{
    AVOutputFormat *prevout = NULL, *out;
    AVInputFormat *previn = NULL, *in;

    ff_mutex_lock(&avpriv_register_devices_mutex);


    for (int i = 0; (out = (AVOutputFormat*)muxer_list[i]); i++) {
        if (prevout)
            prevout->next = out;
        prevout = out;
    }

    if (outdev_list) {
        for (int i = 0; (out = (AVOutputFormat*)outdev_list[i]); i++) {
            if (prevout)
                prevout->next = out;
            prevout = out;
        }
    }

    for (int i = 0; (in = (AVInputFormat*)demuxer_list[i]); i++) {
        if (previn)
            previn->next = in;
        previn = in;
    }

    if (indev_list) {
        for (int i = 0; (in = (AVInputFormat*)indev_list[i]); i++) {
            if (previn)
                previn->next = in;
            previn = in;
        }
    }

    ff_mutex_unlock(&avpriv_register_devices_mutex);
}

The purpose of this function is to muxer_list Output packaging structure in AVOutputFormat String together to form a one-way linked list , When outdev_list When it's not empty , take outdev_list Medium AVOutputFormat They are also strung in the back . In the same way demuxer_list The input encapsulation structure in AVInputFormat Wear them one by one to form a one-way linked list , At the same time indev_list When it's not empty , take indev_list Medium AVInputFormat They are also strung together and hung at the back .

Used before code ff_mutex_lock() function , To ensure the safety of function concurrency , Concurrent lock on . Pass in the parameter avpriv_register_devices_mutex Is defined as

static AVMutex avpriv_register_devices_mutex = AV_MUTEX_INITIALIZER;

It can be traced back to its definition as #define AVMutex pthread_mutex_t, It's defined in libavutil/thread.h in , And ptread_once The function is the same , Use different thread libraries for different platforms , Map to the corresponding mutex variable .

About muxer_list.c and demuxer_list.c
Function list, namely mutex_list and demutex_list Two static arrays are defined in libavformat/muxer_list.c Source files and libavformat/demuxer_list.c In the source file . The details are as follows ：

static const AVOutputFormat *muxer_list[] = {
&ff_a64_muxer,
......
&ff_yuv4mpegpipe_muxer,
NULL };

static const AVInputFormat *demuxer_list[] = {
&ff_aa_demuxer,
......
&ff_libmodplug_demuxer,
NULL };

It is worth noting that , The two here list Source file ,libavformat/muxer_list.c and libavformat/demuxer_list.c You can't see it in the downloaded source code , Only execution FFMPEG Root directory configure File can generate these two list file . The purpose of doing this is to make sure that the user ffmpeg When you add encapsulation and de encapsulation in , This allows the user to encapsulate / The unpacker is written to list in

outdev_list and indev_list And with avdevice_register_all() The relationship between
outdev_list && indev_list The definition of is also in allformat.c The file of , As shown below ：

static const AVInputFormat * const *indev_list = NULL;
static const AVOutputFormat * const *outdev_list = NULL;

Both of these pointers start with NULL, How can I mount other list On the top ？
The answer is allformat.c In file . There is a function in this code avpriv_register_devices(), Is for ffmpeg Other registered functions of . Find the header file for its declaration , by libavformat/internal.h

void avpriv_register_devices(const AVOutputFormat * const o[], const AVInputFormat * const i[]) {
    ff_mutex_lock(&avpriv_register_devices_mutex);
    outdev_list = o;
    indev_list = i;
    ff_mutex_unlock(&avpriv_register_devices_mutex);
#if FF_API_NEXT
    av_format_init_next();
#endif
}

So these two tables are actually initialized by other function calls , The function that calls this method is libavdevice/alldevice.c Of avdevice_register_all() function , It is also an initial registration function that is often used .

void avdevice_register_all(void)
{
    avpriv_register_devices(outdev_list, indev_list);
}

Incoming two list Parameters ,outdev_list It's defined in libavdevice/outdev_list.c,indev_list It's defined in libavdevice/indev_list.c in ,list All of them are AVInputFormat and AVOutputFormat object

static const AVInputFormat * const indev_list[] = {
&ff_alsa_demuxer,
&ff_fbdev_demuxer,
&ff_lavfi_demuxer,
&ff_oss_demuxer,
&ff_sndio_demuxer,
&ff_v4l2_demuxer,
&ff_xcbgrab_demuxer,
NULL };

static const AVOutputFormat * const outdev_list[] = {
&ff_alsa_muxer,
&ff_fbdev_muxer,
&ff_oss_muxer,
&ff_sdl2_muxer,
&ff_sndio_muxer,
&ff_v4l2_muxer,
&ff_xv_muxer,
NULL };

So far, actually av_register_all() We have also learned about the preliminary entry of the function , It's amazing , In the process of reading the source code and tracing the source , even avdevice_register_all() Some of the things we do are also absorbed by preschool students .

av_register_all

First, check whether it has been called , If called , Then go straight back . Initialize all encapsulation and unpacker ( Also called multiplexer and demultiplexer ), such as flv mp4 mp3
mov, But when the source code of this function starts, it will call avcodec_register_all(…) To register all codecs and decoders . So what is registration , Is to put av_register_all What is mentioned in the code is ffmpeg All supported encapsulation formats are added to the linked list , When you need it later, you can find it in the linked list .

avcodec_register_all

First, check whether it has been registered , If you register , Then return to .
Initialize all codecs , Such as h264,h265 etc. , The decoder and encoder also maintain a linked list to store all supported codecs .

avformat_network_init

Initialize all network libraries , such as rtmp,rtsp etc. .