QT audio and video development 46 video transmission UDP version

5 Line code , Can let wechat small program on the shelf to their own APP in | Register and send it to Dajiang 、 Huawei 、 Cherry keyboard ！>>>

One 、 Preface

It was written in the last article that TCP Transmit video , The pros and cons are obvious , The advantage is no packet loss , The disadvantage is that it's slow , Replace the back with UDP signal communication , It's a lot faster , Less 3 The second handshake , And basically no packet loss in LAN , Even if you lose your bag occasionally , For a second 25-30 Here's a picture of , Occasionally a picture is missing , Basically, I can't see , So ignore , But if it is tested on WAN or Internet, such as Alibaba cloud platform ,UDP horrible , There are a lot of lost bags , After all, there's a lot of packet data .

Qt Network communication class , We usually use three ：QTcpSocket Client class 、QTcpServer Server class 、QUdpSocket Communication class , Why not QUdpServer class ？ Actually UDP It's connectionless communication , It takes up very little resources , It can be a client or a server , If you want to be a server, you need to specify the port to call bind The method can . This program also supports TCP Patterns and UDP Pattern , Actually test it , Still recommended TCP Pattern ,UDP Mode because no connection in a short time to send a large number of packets found packet loss , And the size of the bag is limited , yes 65507 byte , about 64K, therefore UDP The resolution of the real-time transmission image in the mode cannot be too large , Actually measured 640*480 The video file is still very good ,720P Basically, it's a bit tragic , There's a lot to lose , Maybe we need to improve the protocol later .

The pictures agreed in this procedure and the agreement are base64 Code transmission , When received, will base64 Decode string to generate image ,QByteArray Built-in class toBase64 Method into base64 Encoded string ,QByteArray::fromBase64 Methods will base64 String to data . After many experiments, the statistics show , The speed of encoding and decoding can also be , among 720P Picture coding 25ms-30ms、 decode 15ms-20ms,1080P Picture coding 35ms-40ms、 decode 25ms-30ms. In general, one second transmission 25-30 Picture and decode 25-30 A picture , There is no problem , Just left CPU Encoding and decoding , If there are many channels open , It's still a waste CPU Of , But dealing with some simple application scenarios is like a fish in water and no pressure .

Communication protocol ：

1. use TCP Long connection and UDP Optional Protocol , Default communication port 6000.
2. Use custom xml Communication protocol .
3. All transmissions plus 20 Bytes header ：IIMAGE:0000000000000,IIMAGE: For fixing the head , Followed by 13 Bytes of Length of content （ contain 20 Head length ） character string .
4. The following protocol Part omits the header byte .
5. In the data returned by the server uuid Is corresponding to the received message uuid.
6. Each time the server returns, it brings the current time , Can be used for client timing .

 Client sends heartbeat 
<?xml version="1.0" encoding="UTF-8"?>
<ImageClient Uuid="8AF12208-0356-434C-8A49-69A2170D9B5A" Flag="SHJC00000001">
    <ClientHeart/>
</ImageClient>

 The server received a heartbeat back 
<?xml version="1.0" encoding="UTF-8"?>
<ImageServer Uuid="8AF12208-0356-434C-8A49-69A2170D9B5A" NowTime="2019-12-05 16:37:47">
    Ok
</ImageServer>

 Client sends pictures 
<?xml version="1.0" encoding="UTF-8"?>
<ImageClient Uuid="66BCB44A-B567-48ED-8889-36B8FA6C4363" Flag="SHJC00000001">
    <ClientImage> picture base64 Encoded string /9j/4AAQSkZJRgABAQEAYABgAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCAJAAtADASIAAhEBAxEB/8QAHwAAAQUBAQEB...nvWsQRlXA61mTjmtWcdazLgcmrQ0U2plSMKjpDE7UtFFAwxRRRQAUuKWigQlFFFLcD//2Q==</ClientImage>
</ImageClient>

 The server receives the picture and returns 
<?xml version="1.0" encoding="UTF-8"?>
<ImageServer Uuid="66BCB44A-B567-48ED-8889-36B8FA6C4363" NowTime="2019-12-05 16:38:47">
    Ack
</ImageServer>

Two 、 Functional characteristics

1. Multithreading sends and receives image data and analyzes image data , The main interface is not stuck .
2. Support at the same time TCP and UDP Two modes , Encapsulates the TCP Pattern and UDP The client class and server class of the pattern .
3. Image transmission client supports sending to multiple servers at the same time , It can be used as an application scenario that a teacher machine can send to multiple student machines on the same screen .
4. At the same time, multiple clients can send pictures to the server at the same time , Each connection of the server will automatically open up a thread to send and receive and analyze image data .
5. Customize label Control slot mechanism to draw pictures , The main interface is not stuck .
6. It has its own heartbeat mechanism to judge offline , Auto reconnect server , Timeout can be set .
7. Each message has a unique message ID uuid, The server will return the corresponding uuid The message said that it had received , The client can judge that the server parsing is successful according to the returned message , Don't send it again , This ensures that the outgoing data server receives and parses it successfully .
8. Each message has a unique image ID flag, amount to ID Number , According to this identification, determine which interface to display by parsing .
9. Picture with base64 Send in string format , The receiver received base64 After decoding the image data of string, regenerate the image .
10. All data are sent and received by signals , Easy to view output .
11. All provide singleton classes , When there is only one convenience, there is no need to use it directly new.
12. Use custom xml agreement , You can freely expand other attribute fields, such as the content with pictures .

3、 ... and 、 design sketch

Four 、 Related sites

1. Domestic site ：https://gitee.com/feiyangqingyun/QWidgetDemo
2. International sites ：https://github.com/feiyangqingyun/QWidgetDemo
3. Personal home page ：https://blog.csdn.net/feiyangqingyun
4. Zhihu Homepage ：https://www.zhihu.com/people/feiyangqingyun/
5. Experience address ：https://blog.csdn.net/feiyangqingyun/article/details/97565652

5、 ... and 、 Core code

#include "udpimageclient.h"
#include "devicefun.h"

QScopedPointer<UdpImageClient> UdpImageClient::self;
UdpImageClient *UdpImageClient::Instance()
{
    if (self.isNull()) {
        static QMutex mutex;
        QMutexLocker locker(&mutex);
        if (self.isNull()) {
            self.reset(new UdpImageClient);
        }
    }

    return self.data();
}

UdpImageClient::UdpImageClient(QObject *parent) : QThread(parent)
{
    // If it is an external network, please adjust the size of this value , The Internet needs to be turned down 
    packageSize = 10000;
    flag = "SHJC00000001";
    serverIP = "127.0.0.1";
    serverPort = 6000;

    stopped = false;

    //UDP Communication object 
    udpSocket = new QUdpSocket(this);
    connect(udpSocket, SIGNAL(readyRead()), this, SLOT(readData()));

    // The timer parses the received data , You can adjust the interval by yourself 
    timerData = new QTimer(this);
    connect(timerData, SIGNAL(timeout()), this, SLOT(checkData()));
    timerData->setInterval(100);

    // Start the timer after the binding signal is started 
    connect(this, SIGNAL(started()), this, SLOT(started()));
    // Binding send data slot 
    connect(this, SIGNAL(readyWrite(QString)), this, SLOT(sendImage(QString)));
}

UdpImageClient::~UdpImageClient()
{
    this->stop();
}

void UdpImageClient::run()
{
    while (!stopped) {
        // Here we use threads to handle , In fact, it can be done by timer , After all tcp Of write It's asynchronous , The operating system automatically schedules 
        // For the later expansion , For example, you need to judge whether the transmission is successful or not , Need to synchronize , So the thread changed to handle 
        // Picture data into base64 Encoded data also takes time , The main time is transcoding 
        // Take out the data and send , We need locks here , Avoid inserting data 
        if (images.count() > 0) {
            QMutexLocker locker(&mutexImage);
            QImage image = images.takeFirst();
            QString imageData = DeviceFun::getImageData(image);
            emit readyWrite(imageData);
        }

        // Have a little rest , otherwise CPU Will be occupied all the time 
        msleep(1);
    }

    stopped = false;
}

void UdpImageClient::readData()
{
    QHostAddress host;
    quint16 port;
    QByteArray data;

    while (udpSocket->hasPendingDatagrams()) {
        data.resize(udpSocket->pendingDatagramSize());
        udpSocket->readDatagram(data.data(), data.size(), &host, &port);

        // The received data is stored in buffer It needs to be locked 
        QMutexLocker locker(&mutexData);
        buffer.append(data);
        emit receiveData(data);
    }
}

void UdpImageClient::checkData()
{
    if (buffer.length() == 0) {
        return;
    }

    // Take out data processing need to lock , Prevent data from being inserted at this time 
    QMutexLocker locker(&mutexData);
    QDomDocument dom;
    if (!DeviceFun::getReceiveXmlData(buffer, dom, "IIMAGE:", 11, true)) {
        return;
    }

    // Take out the nodes one by one and judge the data 
    QDomElement element = dom.documentElement();
    if (element.tagName() == "ImageServer") {
        QString uuid = element.attribute("Uuid");
        QDomNode childNode = element.firstChild();
        QString name = childNode.nodeName();
        QString value = element.text();
        //qDebug() << uuid << name << value;
        // Here you can add your own processing according to the received data 
    }
}

void UdpImageClient::started()
{
    if (!timerData->isActive()) {
        timerData->start();
    }
}

void UdpImageClient::stop()
{
    buffer.clear();
    images.clear();
    stopped = true;
    this->wait();
    udpSocket->disconnectFromHost();

    if (timerData->isActive()) {
        timerData->stop();
    }
}

void UdpImageClient::setPackageSize(int packageSize)
{
    if (packageSize <= 65507) {
        this->packageSize = packageSize;
    }
}

void UdpImageClient::setFlag(const QString &flag)
{
    this->flag = flag;
}

void UdpImageClient::setServerIP(const QString &serverIP)
{
    this->serverIP = serverIP;
}

void UdpImageClient::setServerPort(int serverPort)
{
    this->serverPort = serverPort;
}

void UdpImageClient::writeData(const QString &body)
{
    // structure xml character string 
    QStringList list;
    list.append(QString("<ImageClient Uuid=\"%1\" Flag=\"%2\">").arg(DeviceFun::getUuid()).arg(flag));
    list.append(body);
    list.append("</ImageClient>");

    // Call the general method to compose the complete data according to the protocol 
    QString data = DeviceFun::getSendXmlData(list.join(""), "IIMAGE:");
    QByteArray buffer = data.toUtf8();

    //udp Can only send 65507 Bytes of data =64K  If it exceeds, it will fail to send 
    // So we need manual subcontracting , The packet on the Internet should be smaller 
    if (packageSize == 65500) {
        udpSocket->writeDatagram(buffer, QHostAddress(serverIP), serverPort);
    } else {
        int len = buffer.length();
        int count = len / packageSize + 1;
        for (int i = 0; i < count; i++) {
            QByteArray temp = buffer.mid(i * packageSize, packageSize);
            udpSocket->writeDatagram(temp, QHostAddress(serverIP), serverPort);
        }
    }

    emit sendData(buffer);
}

void UdpImageClient::sendImage(const QString &body)
{
    writeData(QString("<ClientImage>%1</ClientImage>").arg(body));
}

void UdpImageClient::append(const QImage &image)
{
    // We need locks here , Avoid data being pulled out 
    QMutexLocker locker(&mutexImage);
    // Limit the maximum number of messages in the queue , Avoid crazy insertion when offline 
    if (this->isRunning() && images.count() < 10) {
        images << image;
    }
}

void UdpImageClient::clear()
{
    QMutexLocker locker(&mutexImage);
    images.clear();
}