当前位置：网站首页>Talking about PHP file fragment upload from a requirement improvement

Talking about PHP file fragment upload from a requirement improvement

2020-11-09 15:25:00 【gcudwork】

Want to make APP Same thing as WeChat , Can run small programs smoothly ？ | Experience will send you to Xinjiang 、 Huawei 、 Cherry keyboard ！>>>

Let's talk about the preface of this article , I have encountered the need to upload large files, such as video and video material compression package , The demand side even demands G Level file upload

I am the undertaker of the project , The process of the original project is uploading the files to the server ffmpeg Watermark the video , Screenshot , Generate preview video , Then upload it on the server oss, This way is easy to encounter client network fluctuations , Server upload restrictions ,php Upload restrictions , Server memory impact , Often it's time-out, and you can't get hundreds of them M In the video , And upload to oss It will take up the upload bandwidth of the server ,ffmpeg Processing video also reduces the efficiency of the server to a very low level

So I made some improvements in succession : Use alicloud's media processing service to replace the screenshot of the server , Watermark add and generate preview video , One of the intermediate processes is visible [https://my.oschina.net/u/3470006/blog/3104362], Use , Use again oss The client of api Upload directly to oss, In addition to the need for server signature, there is no need for the server to and oss It's interactive , No need to consume a lot of bandwidth and high occupancy cpu, Cloud service is a good place for us to meet various needs

A few days ago, I saw a question saying [ How to solve the problem of memory overflow and timeout when uploading large files , Upload large files, such as hundreds of megabytes , Overflow and timeout problems can occur , How to solve this kind of problem ], The simplest solution is to change php To configure , hold upload_max_filesize,max_execution_time and max_input_time Reform , So you can solve the problem simply and directly

Here's the answer to split , Seeing this, I remember that I haven't done too much upload myself , It took some time to try , In fact, it's quite simple

html

<!DOCTYPE html>
<html lang="zh">
<head>
    <meta charset="UTF-8">
    <title> File fragment upload </title>
</head>
<body>
<input type="file" id="File">
<button id="Upload"> Upload </button>
<p id="Status"></p>
<script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/jquery.min.js"></script>
<script>
    //2M The fragmentation of 
    let BlockSize=2*1048576;
    let Url="Upload.php";
    // It is used to display the current status information to the user 
    let StatusText=$("#Status");
    // Using sequential queues to send fragment files 
    function QueueSendFile(Url,File,BlockSize,Index,MaxNumber){
        if(Index<MaxNumber){
            let Data=new FormData();
            Data.append("File",File.slice(Index*BlockSize,(Index+1)*BlockSize),File.name);
            StatusText.text(` file size ${File.size} byte , Slice size ${BlockSize} byte , in total ${MaxNumber} Fragmentation , Currently uploading ${Index+1} Block and slice `);
            $.ajax(Url,{
                data:Data,
                type:"post",
                // Don't deal with the data and don't specify the type 
                processData:false,
                contentType:false,
                success:function (Result) {
                    console.log(Result)
                    if(Result.Status===1){
                        StatusText.text(` file size ${File.size} byte , Slice size ${BlockSize} byte , in total ${MaxNumber} Fragmentation , The first ${Index+1} Block and fragment upload completed `);
                        Index++;
                        QueueSendFile(Url,File,BlockSize,Index,MaxNumber);

                    }
                    else if(Result.Status===0){
                        Index=MaxNumber;
                        StatusText.text(" There is an error :"+Result.Message);
                    }
                    else{
                        Index=MaxNumber;
                        StatusText.text(" The server returned incorrect data ");
                    }
                },
                error:function (){
                    Index=MaxNumber;
                    StatusText.text(" Request error ");
                }
            });
        }
        else{
            StatusText.text(" File upload complete ");
        }
    }
    $("#Upload").click(function (){
        let File=$("#File")[0].files;
        if(File.length>0){
            File=File[0];
            QueueSendFile(Url,File,BlockSize,0,Math.ceil(File.size/BlockSize));
        }
        else{
            StatusText.text(" You have to select a file ");
        }
    });
</script>
</body>
</html>

Upload.php

<?php
function return_data($Message,$IsSuccess=true){
    header('content-type:application/json');
    exit(json_encode(['Status'=>$IsSuccess?1:0,'Message'=>$Message]));
}
function error_return($Message){
    return_data($Message,false);
}
function success_return($Message=''){
    return_data($Message);
}
// The correct user authentication mechanism should be used , Here, user authentication is not the main content, so set it manually 
$UserId=1;
if(isset($_FILES['File'])){
    $File=$_FILES['File'];
    // Check to create a saved folder , In order to ensure efficiency, the basic folder should be created at one time 
    $BasePath=__DIR__.DIRECTORY_SEPARATOR.'Files'.DIRECTORY_SEPARATOR;
    if(!file_exists($BasePath)){
        mkdir($BasePath);
    }
    $UserPath=$BasePath.$UserId.DIRECTORY_SEPARATOR;
    if(!file_exists($UserPath)){
        mkdir($BasePath);
    }
    // The solution I'm using here is to append the contents of the file each time 
    $SavePath=$UserPath.$File['name'];
    if(!file_exists($SavePath)){
        file_put_contents($SavePath,'');
    }
    file_put_contents($SavePath,file_get_contents($File['tmp_name']),FILE_APPEND);
    success_return();
}
else{
    error_return(' Parameter error ');
}

In fact, before writing this article, I have tried other segmentation schemes :
Can be out of order at the same time upload
Each time the front end sends the partition index and the total size ,php Create a folder for uploaded files , The fragment file is put into it according to the partition index , Check whether the total size and total size of the file are the same each time , In the same way, the operation of merging fragment files is executed , But this one has a drawback , With the increase of fragment files, the size of each calculation will be slower and slower , There are more than two thousand pieces, and the total processing time is more than 100ms( There's a reason why my computer is slow )

Can be out of order, but does not support simultaneous upload
It is similar to the above plan , That is, the total size of the previous segment has been changed to the total number , Check whether the number of folder files is consistent with the total number of files , It's much faster to just get the number of files

Here are a few other questions :
How to deal with the time-consuming operation of merging and slicing ?
After the file upload is completed, separate requests can be made to perform the fragment merge operation , If the time is not too long, it can be completed synchronously , For a long time and there are many such operations, it is necessary to use timed tasks to perform merging and slicing operations , More timely, you can use the task queue to achieve , The front end can simply use polling to find out whether the operation is complete
How to ensure the correctness of uploaded files ?
Calculate the hash value of the file before uploading , When merging or uploading the last fragment, the hash value is passed to php,php To compare the hash of the entire file

版权声明
本文为[gcudwork]所创，转载请带上原文链接，感谢

边栏推荐

猜你喜欢

随机推荐