Ffmpeg first learning (only for coding)

Learning resources

Official documents : Documentation

Thor blog
It can be moved [ summary ]FFMPEG Video and audio codec zero basic learning method

Structure learning :FFMPEG Structural analysis

Basic knowledge of

Video coding

The main function of video coding is to encode video pixel data （RGB,YUV etc. ） Compressed into a video stream , So as to reduce the amount of video data . The video is too big without compression .

Main video coding

name	Launch mechanism
HEVC(H.265)	MPEG/ITU-T
H.264	MPEG/ITU-T
MPEG4	MPEG
MPEG2	MPEG
VP9	Google
VP8	Google
VC-1	Microsoft Inc.

Pros and cons

HEVC(H.265) > VP9 > H.264 > VP8 > MPEG4 > H.263 > MPEG2

Structure learning

AVFrame Structures are generally used to store raw data （ That is, uncompressed data , For example, for video YUV,RGB, For audio, it's PCM）, In addition, it contains some related information

AVCodecContext yes FFmpeg The structure of the codec context , and AVCodec Is the structure of encoding and decoding parameters

AVPacket It's a structure that stores information about compressed encoded data

AVCodec Is a structure that stores codec information

Structure relation
Every AVStream Store a video / Relevant data of audio stream ; Every AVStream Corresponding to one AVCodecContext, Store the video / The audio stream uses decoded data ;
Every AVCodecContext Corresponding to one in AVCodec, Including the video / The decoder for the audio . Each decoder corresponds to one AVCodec structure .

official demo And encode.c Learn to understand

Learn official demo:encode_video.c

I'm personally responsible for the official demo Modified and annotated , The modified file can be run directly , Has been uploaded to the Resource download address

Process analysis :

1. The file path of the given output after the program runs
2. call avcodec_find_encoder Find the encoder AVCodec
3. call avcodec_alloc_context3 Assign encoder context AVCodecContext
4. call av_packet_alloc Distribute AVPacket, The encoder will put each AVFrame Encoded as AVPacket
5. to AVCodecContext Configuration parameters , Such as bit rate 、 Video width and height 、 Frame format and so on
6. call avcodec_open2 Turn on the encoder
7. call av_frame_alloc Distribute AVFrame Memory and set the format and width of the frame
8. call av_frame_get_buffer Allocate space for frame data
9. Start for loop , First call av_frame_make_writable Ensure frame data AVFrame Can write , Then prepare a frame of virtual image data to write frame->data , Finally, for each frame pts assignment , Tell the player when to display this frame data
10. Start coding , call avcodec_send_frame Put frame data AVFrame Send to encoder
11. call avcodec_receive_packet Put the encoded frame AVPacket Take it out of the encoder , Then write each encoded frame into the output file File , Call after writing av_packet_unref Release right AVPacket References to
12. If the encoder ID yes MPEG, You need to output files to File Add a fixed sequence at the end of to get a real MPEG file
13. Finally close the file File, And release AVCodecContext、AVFrame、AVPacket Memory footprint , call avcodec_free_context、av_frame_free、av_packet_free

API Study

avcodec_find_encoder

According to a AVCodecID Find a registered encoder

const AVCodec *avcodec_find_encoder(enum AVCodecID id);

common AVCodecID Casually list some

enum AVCodecID {
    
    AV_CODEC_ID_NONE,

    /* video codecs */
	AV_CODEC_ID_H263,
	AV_CODEC_ID_MPEG4,
	AV_CODEC_ID_H264,   
	AV_CODEC_ID_HEVC,
}

example ：

const AVCodec *codec;
enum AVCodecID codec_id;

codec_id = AV_CODEC_ID_HEVC;

codec = avcodec_find_encoder(codec_id);
if (!codec) {
    
    printf("Codec '%d' not found\n", codec_id);
    exit(1);
}

avcodec_alloc_context3

by AVCodecContext Structure allocates memory , And set each field as the default value , The structure returned by the result needs to use avcodec_free_context() Release memory

AVCodecContext *avcodec_alloc_context3(const AVCodec *codec);

example ：

    AVCodecContext *c= NULL;
    const AVCodec *codec;
    
    c = avcodec_alloc_context3(codec);
    if (!c) {
    
        fprintf(stderr, "Could not allocate video codec context\n");
        exit(1);
    }
    ...
    ...
    ...
    avcodec_free_context(&c);

av_packet_alloc

by AVPacket Structure allocates memory , And set each field as the default value , The structure returned by the result needs to use av_packet_free() Release memory

AVPacket *av_packet_alloc(void);

example ：

    AVPacket *pkt;
    pkt = av_packet_alloc();
    if (!pkt)
        exit(1);
    ...
    ...
    ... 
    av_packet_free(&pkt);   

avcodec_open2

initialization AVCodecContext To use given AVCodec , Before using this function, you need to use avcodec_alloc_context3() Allocate memory

int avcodec_open2(AVCodecContext *avctx, const AVCodec *codec, AVDictionary **options);

av_frame_get_buffer

Assign a new buffers Give video data
Before calling this function, you need to set frame Some parameters of format、width、height

int av_frame_get_buffer(AVFrame *frame, int align);

Parameters align Setting to 0, The system will be based on the current CPU Automatic setting alignment

example ：

    AVFrame *frame;
    
    frame = av_frame_alloc();
    frame->format = c->pix_fmt;
    frame->width  = c->width;
    frame->height = c->height;
    
	av_frame_get_buffer(frame, 32);

avcodec_send_frame

Send data frame to encoder . Use avcodec_receive_packet() Function to retrieve buffered output packets

int avcodec_send_frame(AVCodecContext *avctx, const AVFrame *frame);

avcodec_receive_packet

Read the encoded data from the encoder

int avcodec_receive_packet(AVCodecContext *avctx, AVPacket *avpkt);

Return type ：

0		 It means success 
AVERROR(EAGAIN)		 Output is currently unavailable , The user must give input 
AVERROR_EOF			 The encoder is completely flushed , There will be no output  packets
AVERROR(EINVAL)		codec It's not turned on , Or it is a decoder rather than an encoder 

example ：

        ret = avcodec_receive_packet(enc_ctx, pkt);
        if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF)
            return;
        else if (ret < 0) {
    
            fprintf(stderr, "Error during encoding\n");
            exit(1);
        }

Conclusion

Just started learning ffmpeg, Feel the stone to cross the river , Little by little demo And the source code is quite interesting

English reading needs improving , Otherwise, you won't see the comments given by the developer