Connection and difference of network streaming media protocol (RTP RTCP RTMP HLS)

RTP RTCP RTSP RTMP HLS Differences and connections

RTP Streaming data 、RTCP Yes RTP Control , Sync 、RTSP launch / Stop streaming media

RTP and RTCP Sisterhood ,RTSP have access to RTP To transmit data , But it can be used with theout binding TCP/UDP

RTSP、RTMP、HLS Can do live and on demand , They are three different application layer protocols

Hierarchy diagram of streaming media protocols

About RTP(RTCP) Understanding of the location of the agreement ：

RTP It is actually between the application layer and the transport layer . At the same time, it has various characteristics of application layer and transport layer . This feature needs to be carefully identified .

From the perspective of application developers ,RTP It should be part of the application layer . At the sending end of the application , The developer must write a RTP Encapsulate groups （ Data packets ） The program code of , Then put the RTP Give it to UDP Socket interface . At the receiving end of the device ,RTP Group through UDP After the socket interface enters the application layer program , Will use the program code written by the developer to RTP The grouping is extracted from the application data block .RTP In fact, it provides a development platform for developers of video programs .

RTP It can also be understood as transport layer agreement , actually RTP The protocol focuses on the transport layer , yes UDP agreement （ User datagram ） The upper layer agreement . because RTP Encapsulates multimedia applications （ Including video streaming 、 Audio stream ） A block of data , The key is to provide transportation services （ Time stamp , Serial number 、 Synchronization source identifier, etc ）, So we can RTP As a layer UDP Transport layer sublayer protocol on , Special attention should be paid to the sublayer protocol .

Taken together ,RTP The protocol is actually a protocol between the transport layer and the application layer , Is used in UDP Complete the encapsulation of media stream on the protocol .

be based on RTP Streaming media for

Streaming transmission is the key technology to realize streaming media . Using streaming transmission, you can watch streaming media programs while downloading . because Internet It is based on packet transmission , Therefore, the data packets received by the receiving end often have delay and out of order ( Streaming is built on UDP On )

To achieve streaming , It is necessary to reduce the delay and recover the packet timing .

「 At the sending end , To reduce latency , The transmitted data is often preprocessed ( Reduced quality and efficient compression ( Common compression formats are aac/h264...))」

「 At the receiving end, in order to recover the timing , Receive buffer is often used , In order to realize the smooth play of media , Generally, play buffer is used 」

RTP

RTP(Real-time Transport Protocol) The agreement was created on UDP Always cooperate in the agreement RTSP and RTCP Use it together . It is used for real-time transmission of multimedia data in the network , Provide end-to-end real-time transmission services , But it does not guarantee the quality of service , The quality of service is determined by RTCP To provide .

RTP It is the most basic protocol in the streaming media protocol family , It is IETF One of the criteria proposed , Corresponding RFC file RFC3550,RFC3550 Not only does it define RTP Supporting related agreements are also defined RTCP.

RTP The main responsibility of the protocol is to be responsible for the real-time transmission of streaming media packets and media streams , Every RTP The data message consists of a header (header) And loading (Payload) Two parts , The front of the middle head 12 The meaning of a byte is fixed , The loaded data can be audio or video data .RTP The header format of the datagram is as follows :

「 Can be RTP Compared to a truck , You can load anything in the truck , But the volume is limited , If it's too big, you need to subcontract , One by one, it was loaded and sent , In addition, it may or may not reach the opposite end in sequence , Maybe even missing 」

RTP Load type

Because some types were born late , There is no specific PT value , Only dynamic （dynamic）PT value , namely 96-127, That's why it's so common to say H264 Of PT The value is 96.

rtp_payload_types[] = {
 {0, "PCMU", AVMEDIA_TYPE_AUDIO, AV_CODEC_ID_PCM_MULAW, 8000, 1},
 {3, "GSM", AVMEDIA_TYPE_AUDIO, AV_CODEC_ID_NONE, 8000, 1},
 {4, "G723", AVMEDIA_TYPE_AUDIO, AV_CODEC_ID_G723_1, 8000, 1},
 {5, "DVI4", AVMEDIA_TYPE_AUDIO, AV_CODEC_ID_NONE, 8000, 1},
 {6, "DVI4", AVMEDIA_TYPE_AUDIO, AV_CODEC_ID_NONE, 16000, 1},
 {7, "LPC", AVMEDIA_TYPE_AUDIO, AV_CODEC_ID_NONE, 8000, 1},
 {8, "PCMA", AVMEDIA_TYPE_AUDIO, AV_CODEC_ID_PCM_ALAW, 8000, 1},
 {9, "G722", AVMEDIA_TYPE_AUDIO, AV_CODEC_ID_ADPCM_G722, 8000, 1},
 {10, "L16", AVMEDIA_TYPE_AUDIO, AV_CODEC_ID_PCM_S16BE, 44100, 2},
 {11, "L16", AVMEDIA_TYPE_AUDIO, AV_CODEC_ID_PCM_S16BE, 44100, 1},
 {12, "QCELP", AVMEDIA_TYPE_AUDIO, AV_CODEC_ID_QCELP, 8000, 1},
 {13, "CN", AVMEDIA_TYPE_AUDIO, AV_CODEC_ID_NONE, 8000, 1},
 {14, "MPA", AVMEDIA_TYPE_AUDIO, AV_CODEC_ID_MP2, -1, -1},
 {14, "MPA", AVMEDIA_TYPE_AUDIO, AV_CODEC_ID_MP3, -1, -1},
 {15, "G728", AVMEDIA_TYPE_AUDIO, AV_CODEC_ID_NONE, 8000, 1},
 {16, "DVI4", AVMEDIA_TYPE_AUDIO, AV_CODEC_ID_NONE, 11025, 1},
 {17, "DVI4", AVMEDIA_TYPE_AUDIO, AV_CODEC_ID_NONE, 22050, 1},
 {18, "G729", AVMEDIA_TYPE_AUDIO, AV_CODEC_ID_NONE, 8000, 1},
 {25, "CelB", AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_NONE, 90000, -1},
 {26, "JPEG", AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_MJPEG, 90000, -1},
 {28, "nv", AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_NONE, 90000, -1},
 {31, "H261", AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_H261, 90000, -1},
 {32, "MPV", AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_MPEG1VIDEO, 90000, -1},
 {32, "MPV", AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_MPEG2VIDEO, 90000, -1},
 {33, "MP2T", AVMEDIA_TYPE_DATA, AV_CODEC_ID_MPEG2TS, 90000, -1},
 {34, "H263", AVMEDIA_TYPE_VIDEO, AV_CODEC_ID_H263, 90000, -1},
 {-1, "", AVMEDIA_TYPE_UNKNOWN, AV_CODEC_ID_NONE, -1, -1}
};


RTCP

RTCP(Real-time Transport Control Protocol or RTP Control Protocol） It's real time transport protocol （RTP） A sister agreement for .

RTCP Often with RTP Joint work ,RTP Implement the transmission of actual data ,RTCP Is responsible for sending the control package to everyone on the phone .

Its main function is to RTP Provide feedback on the quality of service provided , Simultaneously synchronize audio and video data , It doesn't pass any data by itself

. They can optimize the transmission efficiency with effective feedback and minimum overhead , Therefore, it is especially suitable for transmitting real-time data on the Internet .

stay RTP During the conversation , Participants periodically transmit RTCP package .RTCP The packet contains the number of packets that have been sent 、 Statistics such as the number of lost packets , therefore , The server can use this information to dynamically change the transmission rate , Even changing the payload type .

RTCP According to the control information carried RTCP Information packages can be divided into

RR

 ( Recipient report package )、

SR

 ( Source report package )、

SE

DS

 ( Source description package )、

BYE

 ( Leave statement ) and

APP

 ( Special packages )5 class

RTSP

Real time streaming protocol （RTSP,Real-time Streaming Protocol） It is a kind of streaming media used to control sound or image , And it allows you to control multiple streams , but RTSP The connection is not bound to a transport layer connection （ Such as RTP）, The server can even choose

TCP

perhaps

UDP

To transport streaming content .

Its grammar is similar to HTTP 1.1, But the synchronization of time is not emphasized , Therefore, the network delay can be tolerated , Is a text-based multimedia playback control protocol .RTSP Define the stream format , Stream data can be accessed via RTP transmission ; therefore RTSP The real-time effect of is very good , Suitable for video chat , Video surveillance, etc .

RTSP Your requests mainly include , Such as

「 describe (describe), Set up (setup), Play (play), Pause (pause), The playback (teardown), Options (options)」

etc. , stay RTSP During the conversation ,

Setup

You can specify RTP/RTCP Port used ,「

play

,

pause

,

teardown

」 Can start and pause RTP Sending of .

RTSP Request example

SETUP  request

SETUP The request specifies how to transfer a single media stream . This must be done before sending PLAY Complete before request . Request contains media stream URL And transport specifiers . The specifier usually includes a for receiving RTP data （ Audio or video ） The local port of , Another one for RTCP data （ Meta information ））. The server reply usually confirms the selected parameters , And fill in the missing part , For example, the port selected by the server . You must send an aggregate playback request before , Use SETUP Configure each media stream .

C->S: SETUP rtsp://example.com/media.mp4/streamid=0 RTSP/1.0CSeq: 3Transport: RTP/AVP;unicast;client_port=8000-8001

S->C: RTSP/1.0 200 OKCSeq: 3Transport: RTP/AVP;unicast;client_port=8000-8001;server_port=9000-9001;ssrc=1234ABCDSession: 12345678

Play  Play request

Play  Play request   Will cause one or all media streams to play . You can stack playback requests by sending multiple playback requests .URL It can be aggregation URL（ Play all media streams ） Or a single media stream URL（ Play this stream only ）. You can specify the range . If no range is specified , The stream will play from the beginning , And play to the end , Or if the stream pauses , Then playback resumes at the pause point .

C->S: PLAY rtsp://example.com/media.mp4 RTSP/1.0CSeq: 4Range: npt=5-20Session: 12345678

S->C: RTSP/1.0 200 OKCSeq: 4Session: 12345678RTP-Info: url=rtsp://example.com/media.mp4/streamid=0;seq=9810092;rtptime=3450012

PAUSE  Pause request

PAUSE  Pause request   Temporarily stop one or all media streams , Therefore, it can be restored later by playing the request . The request contains an aggregate or media stream URL.PAUSE The range parameter in the request specifies when to pause . When the range parameter is omitted , Pauses occur immediately and indefinitely .

C->S: PAUSE rtsp://example.com/media.mp4 RTSP/1.0CSeq: 5Session: 12345678

S->C: RTSP/1.0 200 OKCSeq: 5Session: 12345678

RTMP

RTMP(Real Time Messaging Protocol) Real time messaging protocol is Adobe Systems Company as Flash Audio between player and server 、 Open application layer protocol for video and data transmission development .

RTMP The transport layer is TCP agreement , However, this kind of reliable guarantee will also cause some problems , That is to say, the previous packets have not been delivered to the destination , The following data cannot be transferred . Fortunately, , The current network bandwidth can basically meet RTMP Requirements for protocol transmission of ordinary quality video , And the general delay is at 1-3 Seconds .

RTMP The basic unit of transmitted data is Message, But actually the smallest unit of transmission is Chunk（ Message block ）, because RTMP In order to improve the transmission speed , When transmitting data , Will be able to Message Split it up , Form smaller blocks , These blocks are Chunk.

RTMP Expand

RTMP It's a protocol family , Include RTMP Basic agreement and RTMPT/RTMPS/RTMPE And so on ：

Working in TCP Clear text protocol above , The use of port 1935;

RTMPE stay RTMP On the basis of the increase of encryption function ;

RTMPT Packaged in HTTP In request , Traversable firewall ;

RTMPS similar RTMPT, But it uses HTTPS Connect ;

RTMP agreement (Real Time Messaging Protocol) Be being Flash For objects , video , Audio transmission . This agreement is based on TCP Protocol or polling HTTP The agreement above .

RTMP Protocol is like a container for packets , These data can be either AMF Formatted data , It can also be FLV Medium vision / Audio data . A single connection can transmit multiple network streams over different channels . The packets in these channels are transmitted in fixed size packets .

HLS

HTTP Live Streaming（HLS） It's apple (Apple Inc.) Based on HTTP Streaming media transport protocol of , Live and on demand streaming , Mainly used in iOS System , by iOS equipment （ Such as iPhone、iPad） Provide audio and video live and on demand programs .HLS on demand , Basically, it's a common segment HTTP on demand , Difference is that , It has very small segments .

Compared with the common live streaming protocols , for example RTMP agreement 、RTSP agreement 、MMS Agreements, etc ,HLS The biggest difference of live broadcast is ,

「 Obtained by the live client , Not a complete data flow .HLS The protocol stores the live data stream as continuous on the server side 、 Very short and long media files （MPEG-TS Format ）, And the client is constantly downloading and playing these small files , Because the server will always generate new small files from the latest live data , In this way, the client continuously plays the files obtained from the server in order , It's live .」

thus it can be seen , Basically, we can think ,HLS Live broadcast is realized by on-demand Technology . Due to data passing HTTP Protocol transfer , So you don't have to think about firewalls or agents at all , And the length of segmented files is very short , The client can quickly select and switch the bit rate , To adapt to different bandwidth conditions . however HLS This technical feature of , It determines that its delay is always higher than that of the normal live streaming protocol .　

HLS Provide a m3u8 Address ,Apple Of Safari The browser can be opened directly m3u8 Address , for example

Hong Kong TV :http://live.hkstv.hk.lxdns.com/live/hks/playlist.m3u8

summary

RTP RTCP RTSP  Differences and connections

RTP Streaming data 、RTCP Yes RTP Control , Sync 、RTSP launch / Stop streaming media

RTP and RTCP Sisterhood ,RTSP have access to RTP To transmit data , But it can be used with theout binding TCP/UDP

RTSP、RTMP、HLS Can do live and on demand , They are three different application layer protocols

RTSP、RTMP、HLS  Differences and connections

RTSP、RTMP、HLS Can do live and on demand , They are three different application layer protocols

HLS  Big delay , Suitable for video on demand

RTSP The real-time performance is the best , But the implementation is complex , Suitable for video chat and video surveillance

RTMP Strong in browser support , load flash The plug-in can play directly , Very hot , Instead, play in the browser rtsp It's very difficult

About live broadcasting

Live applications ,RTMP and HLS Basically, it can cover all clients to watch .

・RTSP The main advantage is that the transport layer can use udp/rtp, Good real-time , The delay can be controlled to 1s Inside ,・RTMP The main advantage is that the delay is relatively low (1-3s), and RTMP The support is perfect , Can do flash Play RTMP Flow for a long time , There used to be a lot SIP Video conferencing for has been RTMP replaced .・HLS The advantage of is direct use http Protocol request stream data , You can freely switch between versions with different rates , Seamless playback , The disadvantage is that the delay is relatively large (10s above ).

Each of the three protocols has its own advantages , Mainly depends on the scene selection , Larger companies, of course , Some private agreements can be made to provide better and more scenario compliant services .

author  ——  A kind of い empty ゛ Source ：

http://www.cnblogs.com/ailumiyana/