Audio and video learning route and learning materials recommendation

Preface

The previous article learned something about C/C++ and NDK Related knowledge , Now let's simply get started with some related concepts about audio and video , Ten thousand storey buildings rise from the ground , Or start from the most basic .

Text

Check the packaging format of audio and video files

Here we need to clarify a concept , It's the encapsulation format , Like something we're familiar with mp4、avi etc. , These are encapsulation formats , Not any audio or video coding format , See the figure below for details. ：

 

 

Use MediaInfo Tools

Previously, we used the player to analyze the content of audio and video files by viewing properties , You can use a software called MediaInfo To do the same thing , This can be faster 、 View file information more clearly , Look at the picture below ：

 

 

adopt MediaInfo Check the audio and video information as follows ：

 

Video player principle

For the principle of playing player , In fact, it corresponds to the generation process of audio and video , Mainly including packaging technology 、 Video compression coding technology 、 Audio compression coding technology , If network transmission is considered , It also includes streaming media protocol technology .

 

Look at the picture above , We can see the general principle of the player .

 

Here's a summary , The main thing is to understand the basic steps , And common formats .

Streaming protocol

Since the figure above shows us some basic steps and protocols , Then know something about , Let's first look at some of the major streaming protocols .

 

It's mainly here 2 A direction , among RTSP+RTP Often used for IPTV field , Because it uses UDP Transmit audio and video , Support multicast , Efficient , The disadvantage is that packets will be lost when the network is bad . So the instability of the Internet Environment , Therefore, Internet video services are often used TCP As its streaming media transport layer protocol , So it's like RTMP、HTTP Such protocols are widely used in Internet audio and video services .

Package format

The encapsulation format is also mentioned above , Is to store the video code stream and audio code stream in a file according to a certain format .

name	Launch mechanism	Whether streaming media is supported	Video coding supported	Supported audio coding	The current field of use
AVI	Microsoft Inc	I won't support it	Almost all formats	Almost all formats	BT Download movies
MP4	MPEG	Support	MPEG-2, MPEG-4, H.264, H.263 etc.	AAC, MPEG-1 Layers I, II, III, AC-3 etc.	Internet video sites
TS	MPEG	Support	MPEG-1, MPEG-2, MPEG-4, H.264	MPEG-1 Layers I, II, III, AAC,	IPTV, digital television
FLV	Adobe	Support	Sorenson, VP6, H.264	MP3, ADPCM, Linear PCM, AAC etc.	Internet video sites
MKV	CoreCodec	Support	Almost all formats	Almost all formats	Internet video sites
RMVB	Real Networks Inc.	Support	RealVideo 8, 9, 10	AAC, Cook Codec, RealAudio Lossless	BT Download movies

Here are some commonly used , There is only AVI Streaming media is not supported , That is, you can't broadcast it down , Some encapsulation formats support more audio and video , Some support fewer formats , I won't go into details here , I'll talk about it later .

Video coding

The function of video coding is to convert pixel data (RGB、YUV etc. ) Compress into video stream , So as to reduce the amount of video data .

If the video is not compressed , It's very big , There are hundreds of movies G Size , So video coding is a very important technology .

Here are some of the most commonly used video coding ：

name	Launch mechanism	Launch time	Field of use
HEVC(H.265)	MPEG/ITU-T	2013	H.265 The purpose is to transmit higher quality network video with limited bandwidth , It only needs half of the original bandwidth to play video of the same quality , yes 4K、8k And other areas of UHD
H.264	MPEG/ITU-T	2003	H.264 It is the most widely used video codec protocol at present
VP9	Google	2013	VP9 At present, it is unstable , Used in substitution VP8
VP8	Google	2008	VP8 yes WebRTC Default video codec

Audio encoding

The main function of audio coding is to sample audio data, such as PCM Compressed into audio code stream , So as to reduce the amount of audio data . Audio coding is also a very important technology , But in general, the data of audio is much smaller than that of video , So even if you use a slightly backward audio coding standard , Cause the amount of audio data to increase a little , It will not have a great impact on the total amount of audio and video data .

The following is also a list of some commonly used audio coding ：

name	Launch mechanism	Launch time	Field of use
AAC	MPEG	1997	AAC Currently used in various fields
AC-3	Dolby Inc.	1992	The movie
MP3	MPEG	1993	All fields ( used ), Mostly by AAC replace

Comparison of network video service platforms

At present, network audio and video services mainly include 2 Kind of , One is the live broadcast platform , For example, network TV station 、 All kinds of live APP etc. , There is also an on-demand platform , The corresponding video programs can be played according to the needs of users , For example, video websites .

The main architecture of the live broadcast platform is RTMP+FLV+H.264+AAC This technical solution , On demand platforms are generally HTTP+FLV+H.264+AAC This architecture , These knowledge points will be further studied later, and we will further understand .

summary

This article mainly introduces some basic and common names of audio and video , Just have an impression , There will be more analysis later .

Finally, I recommend some learning materials for you :

Very comprehensive video learning materials : Click here