Implementation of packaging video into MP4 format and storing it in TF Card

One 、MP4 File format analysis

1、 General introduction to video files

(1) The essence of video files ： Record the compressed video frame and restore it by the player Decode and play ( The key is to find the video I frame ,P frame )

(2) The key of video file ： Record information efficiently 、 Compatibility

(3) Information about video files ： Index information 、 Effective information

2、MP4 General introduction to format

(1)MP4 from MP3( pure audio) Upgrade , contain video and audio ,

(2)MP4 yes h.264 The most mainstream packaging format

(3)MP4 Inside the document ** Adopt network byte order （ Big end model ） **

3、MP4 Learning route

(1) Study MP4 Organizational form and box analysis

(2) Migration and use mp4v2 Open source library to package MP4

(3) Further study MP4 Unpack play and mp4v2 Source code

(4) Program by yourself MP4 The packaging 、 Unpack 、 Segmentation, etc .

4、 The official start of the MP4 Learning of organizational form

Reference learning （ Required reading ）：https://www.cnblogs.com/ranson7zop/p/7889272.html

  MP4 The video file encapsulation format is based on QuickTime Container format defined , So reference QuickTime The format definition of MP4 The file format is very helpful .MP4 File format is a very open container , It can be used to describe almost all media structures ,MP4 The media description in the file is separate from the media data , And the organization of media data is also very free , Not necessarily in chronological order , Even media data can directly reference other files . meanwhile ,MP4 Also supports streaming media ( Download and play through the network ）.MP4 At present, it is widely used in packaging h.264 Video and AAC Audio , Is the representative of HD video .

(1) Whole MP4 The document consists of several different box form , Both packaging and unpacking are based on box Unit

(2)MP4 There is and only one ftyp box（file type box）, The box Located throughout MP4 At the beginning of

Two 、MP4Info Tool use

The software and data links used ：

 link ：https://pan.baidu.com/s/1DFbZPI4ch_y33RUAE6whiA 
 Extraction code ：pu15 
-- From Baidu network disk super member V5 The share of 

First step ： Open the software

The second step ： Use the software to open a MP4 Video file , The following will happen ：

The information in the right half of the figure is analyzed box obtain .

The software cannot open too large video files , Only some small video files can be opened for analysis .

3、 ... and 、mp4v2 Transplant and play the actual battle

  mp4v2 The library is dedicated to processing mp4 Open source projects for containers , Its use c++ To write , And provide c Language interface .

1、 download mp4v2

(1)https://launchpad.net/ubuntu/+source/mp4v2/2.0.0~dfsg0-6

(2) decompression , And create _install Directory as the installation directory

2、 Configure and compile

(1) Export environment variables , Make it possible to find the cross compilation chain ：
sudo PATH=$PATH:/opt/hisi-linux/x86-arm/arm-hisiv300-linux/target/bin CC=arm-hisiv300-linux-gcc CXX=arm-hisiv300-linux-gcc 

 To configure ：
./configure --host=arm-hisiv300-linux --prefix=/home/aston/sambashare/mp4v2-2.0.0/_install  --disable-option-checking --disable-debug --disable-optimize --disable-fvisibility --disable-gch --disable-largefile --disable-util --disable-dependency-tracking --disable-libtool-lock

 notes ：_install   This folder needs to be created manually , The final compiled files will be stored in this folder 
(2)make
(3)make install
(4) Check all necessary documents 

3、 Deploy

(1) Generated lib Add to mpp lib Inside

(2) Generated include Add the following file to Hisilicon SDK Medium mpp/include In the middle

(3) Generated lib/ Dynamic library of （* so *） Add pieces to the development board lib Under the table of contents （ Mount to the directory of the host through the development board ）

   If there is insufficient storage space , You can put some unused files ( Such as usb Network card driver ) Delete or put it on the development board /mnt Directory .

cp ~/sambashare/mp4v2-2.0.0/_install/lib/* ./

cp ~/sambashare/mp4v2-2.0.0/_install/include/* ./ -rf

4、 compile sample

(1) With the sample The folder replaces the original sample, Or give the folder a new name sample_mp4

 link ：https://pan.baidu.com/s/1LMNPxBzWeI-94L8zKCVhpA 
 Extraction code ：gu90 
-- From Baidu network disk super member V5 The share of 

(2)make

6、 Get ready TF card

Reference learning （ Understand expansion ）：https://blog.csdn.net/li_wen01/article/details/79929730
(1)TF The card is formatted as FAT32 file system , If it fails, try low-level formatting software such as SDFormat And so on

(2) After starting up, it will TF Mount the card

 mount -t vfat /dev/mmcblk0p1 /usr/mmc

   If the memory card is too big , Such as 16G It may not work , Unrecognized partition , Can't work . I used one 8G It's a success .

(3) Check it first TF Cards are available , Such as creating a file on the computer , Read and write on the development board .

7、 Run and test

(1) Run the above step make Compile generated sample_venc Executable file , Use rtsp Way test , Confirm that there is an image

(2) To terminate the program , Take out TF Connect the card reader to the computer , View through the computer's video playback software

notes ： In the process of functional test ,rtsp Not working properly , but mp4 The video was packaged successfully , Analysis and guess the reason may be that too much information is printed during video packaging rtsp Normal operation of .

Four 、MP4 Package source code analysis

Source code ： That is, the network disk link above , take mpp All documents in the document pass sourceinsight Software establishment engineering for analysis .

Reference learning ：https://blog.csdn.net/weixin_42462202/article/details/90108485

HI_S32 SAMPLE_COMM_VENC_MP4(VENC_STREAM_S *stStream)// This function is only responsible for packaging one frame of data at a time 
{
    
	static int nRecordFlag = 0x00;// these static The value of the variable is initialized only after the first program run 
	static int recording = 0x1;   // Every subsequent call , Will use the last retained value 
	static int spsflag = 0;
	static int ppsflag = 0;
	static MP4TrackId video = 0;
	static MP4FileHandle hMP4File = NULL;

	static char recordfish = 0x1;
	
	int j = 0;
	int len = 0;
	char *pData = NULL;
	char isSyncSample = 0;
	
	if(recordfish == 0x00){
    
		return 0;
	}

	if(hMP4File == NULL){
    // On the first entry 
		hMP4File = MP4CreateEx("/usr/mmc/test.mp4",0, 1, 1, 0, 0, 0, 0);// Create a mp4 file 
		if (hMP4File == MP4_INVALID_FILE_HANDLE)	{
    
			printf("open file fialed.\n");
			return -1;
		}
	
		MP4SetTimeScale(hMP4File, 90000);// Set timestamp 
	}

	//recording： Means recording , Are packaged , A sequence consists of 1sps+1pps+sei+1I frame +29P frame 
	if(recording && stStream->u32Seq > 30){
    	// Before discarding 30 frame , You can also not discard it , Is to ensure that from I Frame start 
		if(stStream->u32PackCount >= 3){
    //sps,pps,sei, from I Frame start encoding , Ensure that the file can be played at the beginning 
			nRecordFlag = 1;
		}
	
		if(nRecordFlag){
    
			for(j = 0;j < stStream->u32PackCount;j++){
    
				len 	= stStream->pstPack[j].u32Len - stStream->pstPack[j].u32Offset;
				pData	= (stStream->pstPack[j].pu8Addr + stStream->pstPack[j].u32Offset);						
	
				if(stStream->pstPack[j].DataType.enH264EType == H264E_NALU_SPS){
    
					if(spsflag == 0x00){
    
						spsflag = 0x1;
						// Write sps
						printf("Write sps =================\n");	
	
						// add to track, It is equivalent to adding a flow  
						video = MP4AddH264VideoTrack(hMP4File, 90000, 90000 / 30, 1280, 720,
																pData[4+1], //sps[1] AVCProfileIndication
																pData[4+2], //sps[2] profile_compat
																pData[4+3], //sps[3] AVCLevelIndication
																3); // 4 bytes length before each NAL unit
							MP4SetVideoProfileLevel(hMP4File, 0x7F);// Level of video 
							MP4AddH264SequenceParameterSet(hMP4File, video, pData+4, len-4); // add to sps To mp4 file  
					}
					
					continue;
				}
				
				if(stStream->pstPack[j].DataType.enH264EType == H264E_NALU_PPS){
    
					if(ppsflag == 0x00){
    
						ppsflag = 0x1;
						// add to pps 
						printf("Write pps -------------------\n");										
						MP4AddH264PictureParameterSet(hMP4File, video, pData+4, len-4);// add to pps To mp4 file 
					}
					
					continue;
				}
	
				isSyncSample = (stStream->pstPack[j].DataType.enH264EType == H264E_NALU_ISLICE)	?  (1) : (0);
				pData[0] = (len - 4) >> 24;// Build network byte order , Minus four , The separator is removed 0001 The length of 
				pData[1] = (len - 4) >> 16;
				pData[2] = (len - 4) >> 8;
				pData[3] = len - 4; 							
	
				printf("Write date type = %d isSyncSample = %d\n",stStream->pstPack[j].DataType.enH264EType,isSyncSample);								
	
				MP4WriteSample(hMP4File, video, pData, len , MP4_INVALID_DURATION, 0, isSyncSample);// write in nalu
	
				
			}					
		}
	}
	
	if((recording && stStream->u32Seq > 900)){
    // Control file duration 
		recording = 0x00;
		printf("Close mp4 file\n");						
		MP4Close(hMP4File, 0);// close mp4 file , In the writing mp4 This function must be called after the file , Otherwise mp4 The file will be corrupted 
		hMP4File = NULL;
		video = 0;
		recordfish = 0x00;
	}

}

5、 ... and 、mp4v2 combination MP4Info Learning analysis

1、 Ideas

(1) modify MP4 Package source code , use MP4Info Recorded view MP4 details

(2) Further depth ： modify mp4v2 Details in the source code , Compile implementation , REPACK MP4 Video view analysis

2、 practice 1： Get rid of sps

   Pass the test , Get rid of sps Video after VLC Players and other video players （ Tencent, etc ）, Can't play .

3、 practice 2： Get rid of pps

   Pass the test , Get rid of pps Video after VLC Players and other video players （ Tencent, etc ）, Can't play .

The top-down sequence in the figure is complete MP4 video , Get rid of sps, Get rid of pps.

6、 ... and 、 Add network telnet debugging

1、 Why add telnet debugging

(1) The embedded linux The user interface of the system is commandline, Essentially by busybox Provide

(2)busybox The only command line is 1 individual , Once the front desk is occupied, you can't do other operations

(3) The solution is 2 individual ： One is to establish multiple commandline, One is to open other user interfaces .

2、telnet The principle of debugging

(1) Run ahead in the development board telnetd( This software is already included in the system provided by Hisilicon )

telnet： Client program , It is used by the development board to log in to other servers remotely
telnetd： Server program , Is a Daemons , It is used by other clients to log in to the development board

(2) Remote access telnet Of client Connect server, Build a user interface

(3) This is a very traditional and typical way of remote login ··· Actually used

3、 stay HI3518E Development board telnet Remote login debugging practice

(1) Command line execution telnetd &, then Windows open SecureCRT( Other software with the same function can also , Such as MobaXterm) To configure SSH to 192.168.1.10, That is, the development board network interface


(2) stay CRT Interface input ：

 user name ssid：root
password： Because my development board has no password , Directly enter , Get into 

(3) problem ： In case of continuous restart , take etc/profile Add ins in move to /etc/init.d Medium rcS In file . This is because it opens telnet The service will load many times profile（ Unable to mount again after the previous mount ）

profile file :

You can move these contents to /etc/init.d Medium rcS In file .

(4) Can be telnetd & Join in rcS Boot in the default load

7、 ... and 、 Hayes proc File system debugging interface ( Now most of them are sys File system replaces )

1、proc How the file system works

Reference learning ：
https://www.cnblogs.com/lidabo/p/5628020.html
https://blog.csdn.net/weixin_29202687/article/details/116963940

2、 Hayes proc File system debugging documentation

 link ：https://pan.baidu.com/s/1zDDRvTl3gx4GhHE4kgx-zA 
 Extraction code ：pfp8 
-- From Baidu network disk super member V5 The share of 

3、 Additional commissioning experience documentation

 link ：https://pan.baidu.com/s/1Eu3LmkO3-y3YDsEARnikbw 
 Extraction code ：oifi 
-- From Baidu network disk super member V5 The share of 

notes ： This article refers to 《 Mr. Zhu, Internet of things lecture hall 》 Course notes , Combined with their own actual development experience and other technical articles on the Internet , Comprehensive sorting results in . If there is any infringement , Contact deletion ！ Level co., LTD. , Welcome to communicate in the comments area .