The origin of PTS, DTS and duration of audio and video packages

Audio and video package pts,dts,duration The origin of .

pts: Broadcast timestamp

dts: Decoding timestamp

duration: The broadcast time

How to deeply understand these concepts , What exactly do they represent ? Understand through specific examples and data .

package pts:
For video packages pts, It's not linear , But there will be fluctuations , for example .
Before the following 12 Video compression package .
packet count:1 ,stream_index:0,packet_vcount:1,packet_acount:0,pts:133200
packet count:2 ,stream_index:0,packet_vcount:2,packet_acount:0,pts:147600
packet count:3 ,stream_index:0,packet_vcount:3,packet_acount:0,pts:140400
packet count:10 ,stream_index:0,packet_vcount:4,packet_acount:6,pts:136800
packet count:11 ,stream_index:0,packet_vcount:5,packet_acount:6,pts:144000
packet count:12 ,stream_index:0,packet_vcount:6,packet_acount:6,pts:162000
packet count:13 ,stream_index:0,packet_vcount:7,packet_acount:6,pts:154800
packet count:22 ,stream_index:0,packet_vcount:8,packet_acount:14,pts:151200
packet count:23 ,stream_index:0,packet_vcount:9,packet_acount:14,pts:158400
packet count:24 ,stream_index:0,packet_vcount:10,packet_acount:14,pts:176400
packet count:25 ,stream_index:0,packet_vcount:11,packet_acount:14,pts:169200
packet count:26 ,stream_index:0,packet_vcount:12,packet_acount:14,pts:165600
Its floating law :14400,-7200,-3600,7200,18000,-7200,-3600,7200,18000,-7200,-3600
Actually , Video package duration All are 3600.

For audio packages pts, Is linearly increasing , for example :
following 14 Audio compression package , Every bag pts More than the previous bag 1920, Of every audio package duraton yes 1920
packet count:4 ,stream_index:1,packet_vcount:3,packet_acount:1,pts:131280
packet count:5 ,stream_index:1,packet_vcount:3,packet_acount:2,pts:133200
packet count:6 ,stream_index:1,packet_vcount:3,packet_acount:3,pts:135120
packet count:7 ,stream_index:1,packet_vcount:3,packet_acount:4,pts:137040
packet count:8 ,stream_index:1,packet_vcount:3,packet_acount:5,pts:138960
packet count:9 ,stream_index:1,packet_vcount:3,packet_acount:6,pts:140880
packet count:14 ,stream_index:1,packet_vcount:7,packet_acount:7,pts:142800
packet count:15 ,stream_index:1,packet_vcount:7,packet_acount:8,pts:144720
packet count:16 ,stream_index:1,packet_vcount:7,packet_acount:9,pts:146640
packet count:17 ,stream_index:1,packet_vcount:7,packet_acount:10,pts:148560
packet count:18 ,stream_index:1,packet_vcount:7,packet_acount:11,pts:150480
packet count:19 ,stream_index:1,packet_vcount:7,packet_acount:12,pts:152400
packet count:20 ,stream_index:1,packet_vcount:7,packet_acount:13,pts:154320
packet count:21 ,stream_index:1,packet_vcount:7,packet_acount:14,pts:156240
Sampling rate of audio stream .

And No 4 An audio package pts:131280 Biti 3 Video packets pts:140400 Small , But the first 9 An audio package pts140880 Biti 3 A video package is big .
The first 10 Video packets pts:136800 Biti 3 Video packets pts:140400 Still small , Biti 9 An audio package pts It's also small , But the first 11 Video packets pts It grows up again .

For compressed packages , Should see dts. We see that video packets are also linearly increasing , Each time long 3600, It looks good like this !!
Audio package dts Is equal to pts
packet count:1 ,stream_index:0,packet_vcount:1,packet_acount:0,dts:126000,duration:3600
packet count:2 ,stream_index:0,packet_vcount:2,packet_acount:0,dts:129600,duration:3600
packet count:3 ,stream_index:0,packet_vcount:3,packet_acount:0,dts:133200,duration:3600
packet count:4 ,stream_index:1,packet_vcount:3,packet_acount:1,dts:131280,duration:1920
packet count:5 ,stream_index:1,packet_vcount:3,packet_acount:2,dts:133200,duration:1920
packet count:6 ,stream_index:1,packet_vcount:3,packet_acount:3,dts:135120,duration:1920
packet count:7 ,stream_index:1,packet_vcount:3,packet_acount:4,dts:137040,duration:1920
packet count:8 ,stream_index:1,packet_vcount:3,packet_acount:5,dts:138960,duration:1920
packet count:9 ,stream_index:1,packet_vcount:3,packet_acount:6,dts:140880,duration:1920
packet count:10 ,stream_index:0,packet_vcount:4,packet_acount:6,dts:136800,duration:3600
packet count:11 ,stream_index:0,packet_vcount:5,packet_acount:6,dts:140400,duration:3600
packet count:12 ,stream_index:0,packet_vcount:6,packet_acount:6,dts:144000,duration:3600
packet count:13 ,stream_index:0,packet_vcount:7,packet_acount:6,dts:147600,duration:3600
packet count:14 ,stream_index:1,packet_vcount:7,packet_acount:7,dts:142800,duration:1920
packet count:15 ,stream_index:1,packet_vcount:7,packet_acount:8,dts:144720,duration:1920
packet count:16 ,stream_index:1,packet_vcount:7,packet_acount:9,dts:146640,duration:1920
packet count:17 ,stream_index:1,packet_vcount:7,packet_acount:10,dts:148560,duration:1920
packet count:18 ,stream_index:1,packet_vcount:7,packet_acount:11,dts:150480,duration:1920
packet count:19 ,stream_index:1,packet_vcount:7,packet_acount:12,dts:152400,duration:1920
packet count:20 ,stream_index:1,packet_vcount:7,packet_acount:13,dts:154320,duration:1920
packet count:21 ,stream_index:1,packet_vcount:7,packet_acount:14,dts:156240,duration:1920

Video bag duration=3600 The origin of
Video package r_frame_rate, Get from video stream information (ctx There is )m_fmt_ctx->streams[m_video_stream_index]->r_frame_rate=(25,1)
Video package time_base, Get from video stream information (ctx There is )m_fmt_ctx->streams[m_video_stream_index]->time_base=(1,90000)
Then the time of one frame 1/25 = 0.04s = 40ms
Convert to video packet time base pts = 0.04 * 90000 = 40*90=3600

Audio package duration=1920 The origin of
Sampling rate of audio packets (sample_rate): By audio stream information (ctx There is )m_fmt_ctx->streams[m_audio_stream_index]->codecpar->sample_rate obtain 48000
The size of the audio package (frame_size): By audio stream information (ctx There is )m_fmt_ctx->streams[m_audio_stream_index]->codecpar->frame_size obtain 1024
Audio package time_base: By audio stream information (ctx There is )m_fmt_ctx->streams[m_audio_stream_index]->time_base obtain (1,90000)
Then the time of an audio packet : 1024/48000=0.0213 second
Use time base to express time : 1024/48000*90000=1920

pts When is it used ? This needs packet The compressed packet is decoded into frame after , Solved frame Is in accordance with the pts They are arranged in order , Each one frame Play the video in sequence .
A section of test data is attached for reference .
packet count:1 ,stream_index:0,packet_vcount:1,packet_acount:0,pts:133200,dts:126000,duration:3600,size:25709
packet count:2 ,stream_index:0,packet_vcount:2,packet_acount:0,pts:147600,dts:129600,duration:3600,size:9433
packet count:3 ,stream_index:0,packet_vcount:3,packet_acount:0,pts:140400,dts:133200,duration:3600,size:3912
frame count:1 ,frametype:v,frame_vcount:1,frame_acount:0,pts:133200,pkt_size:25709
packet count:4 ,stream_index:1,packet_vcount:3,packet_acount:1,pts:131280,dts:131280,duration:1920,size:69
frame count:2 ,frametype:a,frame_vcount:1,frame_acount:1,pts:131280,pkt_size:69
packet count:5 ,stream_index:1,packet_vcount:3,packet_acount:2,pts:133200,dts:133200,duration:1920,size:425
frame count:3 ,frametype:a,frame_vcount:1,frame_acount:2,pts:133200,pkt_size:425
packet count:6 ,stream_index:1,packet_vcount:3,packet_acount:3,pts:135120,dts:135120,duration:1920,size:830
frame count:4 ,frametype:a,frame_vcount:1,frame_acount:3,pts:135120,pkt_size:830
packet count:7 ,stream_index:1,packet_vcount:3,packet_acount:4,pts:137040,dts:137040,duration:1920,size:427
frame count:5 ,frametype:a,frame_vcount:1,frame_acount:4,pts:137040,pkt_size:427
packet count:8 ,stream_index:1,packet_vcount:3,packet_acount:5,pts:138960,dts:138960,duration:1920,size:424
frame count:6 ,frametype:a,frame_vcount:1,frame_acount:5,pts:138960,pkt_size:424
packet count:9 ,stream_index:1,packet_vcount:3,packet_acount:6,pts:140880,dts:140880,duration:1920,size:410
frame count:7 ,frametype:a,frame_vcount:1,frame_acount:6,pts:140880,pkt_size:410
packet count:10 ,stream_index:0,packet_vcount:4,packet_acount:6,pts:136800,dts:136800,duration:3600,size:2727
frame count:8 ,frametype:v,frame_vcount:2,frame_acount:6,pts:136800,pkt_size:2727
packet count:11 ,stream_index:0,packet_vcount:5,packet_acount:6,pts:144000,dts:140400,duration:3600,size:2576
frame count:9 ,frametype:v,frame_vcount:3,frame_acount:6,pts:140400,pkt_size:3912
packet count:12 ,stream_index:0,packet_vcount:6,packet_acount:6,pts:162000,dts:144000,duration:3600,size:8391
frame count:10 ,frametype:v,frame_vcount:4,frame_acount:6,pts:144000,pkt_size:2576
packet count:13 ,stream_index:0,packet_vcount:7,packet_acount:6,pts:154800,dts:147600,duration:3600,size:3486
frame count:11 ,frametype:v,frame_vcount:5,frame_acount:6,pts:147600,pkt_size:9433
packet count:14 ,stream_index:1,packet_vcount:7,packet_acount:7,pts:142800,dts:142800,duration:1920,size:388
frame count:12 ,frametype:a,frame_vcount:5,frame_acount:7,pts:142800,pkt_size:388