How to make your big file upload stable and fast?

With the significant improvement of the network environment and the significant reduction of network charges , People's interaction on the Internet is no longer limited to the traditional text content , picture 、 voice 、 Videos have joined the ranks of interaction .

Compared with the text content , Multimedia has more advantages in ease of use and emotional expression , Of course, relative text , The volume is also much larger . For example, a text message has less than 100 byte , But a video message may exceed 100 MB. This leads to multimedia transmission over the network 、 Real time touch and other aspects will be more difficult than text content .

In a website or program , In addition to the pictures of user communication 、 Video and other interactive messages , There are also more and more large files in other forms . Ensuring the stability of uploading large files is a key point that requires long-term investment and breakthrough . today , Let's take a look at the means to ensure that users have a good upload experience .

How to make resource delivery fast and stable

You may notice , Use QQ when , Pictures and videos sent to you by friends will be loaded for a period of time before they are displayed , This is because the program does not send files to you , Instead, it sends the file path , The file will not be loaded until we access the path . This is the mainstream way of large file communication , namely ：

• Upload file to file server

• Push file path

• Received file path

• Load the file

This is because the transfer of large files takes too long , Will affect the immediacy of the message , So it won't transfer files directly .

that , If we want the picture 、 video 、 Multimedia messages such as voice are sent quickly and stably , You can start from “ Upload multimedia files to the file server ” Start with this step .

Upload access point

The first way to optimize the upload file is to optimize the upload access point .

Upload request to file server , The server responded to the association , The uploader continues to transfer data , Until the upload is complete . The shorter the time of this process, the faster the Internet speed will feel .

However, the composition of fixed line broadband operators in our country is complex , The high latency and instability of cross operator network access has always been a difficult problem . More video for transmitting data 、 For multimedia files such as pictures , Upload files across operators , many times RTT（Round-Trip-Time, Round trip delay ） The experience will be worse .

At present, the mainstream approach to this network situation is , Provide different upload access points for different network operators IP, Then through the operator DNS analysis , So that users can quickly upload files through the upload access point of the operator . At the same time, the back-end image upload storage service will be deployed in the multi line machine room , Easily and quickly submit file streams to the storage tier , This avoids the cross network overhead from the access point to the storage service , And solve the problem that users of other operators need to cross the network when downloading pictures .

however , Although multiple operator access points IP It can solve the problem of cross operator network access , But it will be more complicated in management and use . Therefore, most service providers will take advantage of multi line BGP（Border Gateway Protocol, Border gateway protocol ） Computer room trusteeship , So you can only provide IP , Let access users choose the best route .

Upload in blocks

Remove the optimization of the upload access point , Block upload can also improve the upload performance and success rate of multimedia files .

Upload in blocks , It refers to uploading the client , The file to be uploaded is divided into multiple data blocks according to certain rules , And mark the serial number according to the rules , Then upload these data blocks separately , After the server receives it , Reassemble multiple data blocks into a complete file according to the serial number .

For the picture 、 For larger files like video , Block upload allows the client to upload after the block is completed , Use concurrency to make full use of network bandwidth and upload multiple blocks at the same time , Effectively improve the upload efficiency . Besides , In some network environments 、 When the signal is unstable , In a block wise manner , When retrying after upload failure , Just retransmit a failed block , You don't have to upload the whole file again , Significantly improve the success rate of retransmission .

In block upload , Block size is also an important issue . If the block is too large , Few pieces , The upload concurrency is not enough , It may reduce the upload efficiency , Moreover, the cost of retransmission after failure of each large block will be relatively high . But if the blocks are too small , There are many films , The number of concurrent connections is too much , Multiple connections reduce the overall throughput of the network , The overhead of splitting and merging blocks at the customer's upload end and service receiving end also increases accordingly , And the extra traffic during transmission will be more .

therefore , How to choose an appropriate block size in different network environments is very important . Generally speaking , In a better network environment , For example WiFi、4G Next , The corresponding block size should be set larger ; And in the 2G、3G In the case of weak network , The block can be set smaller .

Breakpoint continuation

Because the upload of large files takes a long time , If the user is disconnected from the network 、 Equipment restart and other reasons , You need to pause the upload task when the upload is not completed . Of course , Users certainly don't want the next upload to start from scratch . Therefore, the function of how to upload from the pause position becomes necessary . This is what we often say “ Breakpoint continuation ”

Breakpoint continuation is achieved by assigning a unique task ID to each upload , When uploading each block, in addition to carrying its own serial number , You also need to bring this task id , The server temporarily stores the blocks of the same task ID received , Even if the upload is suspended for some reason , These temporary blocks will not be cleaned up immediately , But keep it for a certain time .

When the uploader needs to continue uploading , The previous task ID will continue to be carried at the same time as the request . After receiving it, the client will check that the server has been partitioned , If it does not expire, continue to upload from the last disconnected block location , Otherwise, it will be uploaded again .

Breakpoint continuation function when uploading large files , It can solve the pain point of having to start over again due to network errors in the process of transferring large files , It is obvious to improve the user upload experience .

How to quickly have these upload advantages

If you want to complete the above large file upload optimization , Architecture needs to be considered 、 Design 、 Operation and maintenance, etc , Not only the optimization cost is high , The transformation time is relatively long . Want to have these advantages mentioned above , Just choose to shoot cloud storage again .

Cloud storage allows large files to pass through CDN Upload nearby 、 Upload to speed up transmission 、 Block upload and breakpoint continuation , You can also use image processing 、 Video processing 、 File compression and decompression process files . When accessing files , Also through CDN Use large file segmentation cache 、 Video drag 、 Link optimization and other functions for network wide distribution . Ensure the transmission of massive data in enterprises 、 Handle 、 Storage 、 Distribution and disaster recovery requirements .

Recommended reading

CSS The margin of the box collapsed

Popular scripts kill and SaaS The indissoluble bond of