当前位置:网站首页>QT audio and video development 46 video transmission UDP version
QT audio and video development 46 video transmission UDP version
2020-11-06 22:06:00 【Flying clouds】
One 、 Preface
It was written in the last article that TCP Transmit video , The pros and cons are obvious , The advantage is no packet loss , The disadvantage is that it's slow , Replace the back with UDP signal communication , It's a lot faster , Less 3 The second handshake , And basically no packet loss in LAN , Even if you lose your bag occasionally , For a second 25-30 Here's a picture of , Occasionally a picture is missing , Basically, I can't see , So ignore , But if it is tested on WAN or Internet, such as Alibaba cloud platform ,UDP horrible , There are a lot of lost bags , After all, there's a lot of packet data .
Qt Network communication class , We usually use three :QTcpSocket Client class 、QTcpServer Server class 、QUdpSocket Communication class , Why not QUdpServer class ? Actually UDP It's connectionless communication , It takes up very little resources , It can be a client or a server , If you want to be a server, you need to specify the port to call bind The method can . This program also supports TCP Patterns and UDP Pattern , Actually test it , Still recommended TCP Pattern ,UDP Mode because no connection in a short time to send a large number of packets found packet loss , And the size of the bag is limited , yes 65507 byte , about 64K, therefore UDP The resolution of the real-time transmission image in the mode cannot be too large , Actually measured 640*480 The video file is still very good ,720P Basically, it's a bit tragic , There's a lot to lose , Maybe we need to improve the protocol later .
The pictures agreed in this procedure and the agreement are base64 Code transmission , When received, will base64 Decode string to generate image ,QByteArray Built-in class toBase64 Method into base64 Encoded string ,QByteArray::fromBase64 Methods will base64 String to data . After many experiments, the statistics show , The speed of encoding and decoding can also be , among 720P Picture coding 25ms-30ms、 decode 15ms-20ms,1080P Picture coding 35ms-40ms、 decode 25ms-30ms. In general, one second transmission 25-30 Picture and decode 25-30 A picture , There is no problem , Just left CPU Encoding and decoding , If there are many channels open , It's still a waste CPU Of , But dealing with some simple application scenarios is like a fish in water and no pressure .
Communication protocol :
- use TCP Long connection and UDP Optional Protocol , Default communication port 6000.
- Use custom xml Communication protocol .
- All transmissions plus 20 Bytes header :IIMAGE:0000000000000,IIMAGE: For fixing the head , Followed by 13 Bytes of Length of content ( contain 20 Head length ) character string .
- The following protocol Part omits the header byte .
- In the data returned by the server uuid Is corresponding to the received message uuid.
- Each time the server returns, it brings the current time , Can be used for client timing .
Client sends heartbeat
<?xml version="1.0" encoding="UTF-8"?>
<ImageClient Uuid="8AF12208-0356-434C-8A49-69A2170D9B5A" Flag="SHJC00000001">
<ClientHeart/>
</ImageClient>
The server received a heartbeat back
<?xml version="1.0" encoding="UTF-8"?>
<ImageServer Uuid="8AF12208-0356-434C-8A49-69A2170D9B5A" NowTime="2019-12-05 16:37:47">
Ok
</ImageServer>
Client sends pictures
<?xml version="1.0" encoding="UTF-8"?>
<ImageClient Uuid="66BCB44A-B567-48ED-8889-36B8FA6C4363" Flag="SHJC00000001">
<ClientImage> picture base64 Encoded string /9j/4AAQSkZJRgABAQEAYABgAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCAJAAtADASIAAhEBAxEB/8QAHwAAAQUBAQEB...nvWsQRlXA61mTjmtWcdazLgcmrQ0U2plSMKjpDE7UtFFAwxRRRQAUuKWigQlFFFLcD//2Q==</ClientImage>
</ImageClient>
The server receives the picture and returns
<?xml version="1.0" encoding="UTF-8"?>
<ImageServer Uuid="66BCB44A-B567-48ED-8889-36B8FA6C4363" NowTime="2019-12-05 16:38:47">
Ack
</ImageServer>
Two 、 Functional characteristics
- Multithreading sends and receives image data and analyzes image data , The main interface is not stuck .
- Support at the same time TCP and UDP Two modes , Encapsulates the TCP Pattern and UDP The client class and server class of the pattern .
- Image transmission client supports sending to multiple servers at the same time , It can be used as an application scenario that a teacher machine can send to multiple student machines on the same screen .
- At the same time, multiple clients can send pictures to the server at the same time , Each connection of the server will automatically open up a thread to send and receive and analyze image data .
- Customize label Control slot mechanism to draw pictures , The main interface is not stuck .
- It has its own heartbeat mechanism to judge offline , Auto reconnect server , Timeout can be set .
- Each message has a unique message ID uuid, The server will return the corresponding uuid The message said that it had received , The client can judge that the server parsing is successful according to the returned message , Don't send it again , This ensures that the outgoing data server receives and parses it successfully .
- Each message has a unique image ID flag, amount to ID Number , According to this identification, determine which interface to display by parsing .
- Picture with base64 Send in string format , The receiver received base64 After decoding the image data of string, regenerate the image .
- All data are sent and received by signals , Easy to view output .
- All provide singleton classes , When there is only one convenience, there is no need to use it directly new.
- Use custom xml agreement , You can freely expand other attribute fields, such as the content with pictures .
3、 ... and 、 design sketch
Four 、 Related sites
- Domestic site :https://gitee.com/feiyangqingyun/QWidgetDemo
- International sites :https://github.com/feiyangqingyun/QWidgetDemo
- Personal home page :https://blog.csdn.net/feiyangqingyun
- Zhihu Homepage :https://www.zhihu.com/people/feiyangqingyun/
- Experience address :https://blog.csdn.net/feiyangqingyun/article/details/97565652
5、 ... and 、 Core code
#include "udpimageclient.h"
#include "devicefun.h"
QScopedPointer<UdpImageClient> UdpImageClient::self;
UdpImageClient *UdpImageClient::Instance()
{
if (self.isNull()) {
static QMutex mutex;
QMutexLocker locker(&mutex);
if (self.isNull()) {
self.reset(new UdpImageClient);
}
}
return self.data();
}
UdpImageClient::UdpImageClient(QObject *parent) : QThread(parent)
{
// If it is an external network, please adjust the size of this value , The Internet needs to be turned down
packageSize = 10000;
flag = "SHJC00000001";
serverIP = "127.0.0.1";
serverPort = 6000;
stopped = false;
//UDP Communication object
udpSocket = new QUdpSocket(this);
connect(udpSocket, SIGNAL(readyRead()), this, SLOT(readData()));
// The timer parses the received data , You can adjust the interval by yourself
timerData = new QTimer(this);
connect(timerData, SIGNAL(timeout()), this, SLOT(checkData()));
timerData->setInterval(100);
// Start the timer after the binding signal is started
connect(this, SIGNAL(started()), this, SLOT(started()));
// Binding send data slot
connect(this, SIGNAL(readyWrite(QString)), this, SLOT(sendImage(QString)));
}
UdpImageClient::~UdpImageClient()
{
this->stop();
}
void UdpImageClient::run()
{
while (!stopped) {
// Here we use threads to handle , In fact, it can be done by timer , After all tcp Of write It's asynchronous , The operating system automatically schedules
// For the later expansion , For example, you need to judge whether the transmission is successful or not , Need to synchronize , So the thread changed to handle
// Picture data into base64 Encoded data also takes time , The main time is transcoding
// Take out the data and send , We need locks here , Avoid inserting data
if (images.count() > 0) {
QMutexLocker locker(&mutexImage);
QImage image = images.takeFirst();
QString imageData = DeviceFun::getImageData(image);
emit readyWrite(imageData);
}
// Have a little rest , otherwise CPU Will be occupied all the time
msleep(1);
}
stopped = false;
}
void UdpImageClient::readData()
{
QHostAddress host;
quint16 port;
QByteArray data;
while (udpSocket->hasPendingDatagrams()) {
data.resize(udpSocket->pendingDatagramSize());
udpSocket->readDatagram(data.data(), data.size(), &host, &port);
// The received data is stored in buffer It needs to be locked
QMutexLocker locker(&mutexData);
buffer.append(data);
emit receiveData(data);
}
}
void UdpImageClient::checkData()
{
if (buffer.length() == 0) {
return;
}
// Take out data processing need to lock , Prevent data from being inserted at this time
QMutexLocker locker(&mutexData);
QDomDocument dom;
if (!DeviceFun::getReceiveXmlData(buffer, dom, "IIMAGE:", 11, true)) {
return;
}
// Take out the nodes one by one and judge the data
QDomElement element = dom.documentElement();
if (element.tagName() == "ImageServer") {
QString uuid = element.attribute("Uuid");
QDomNode childNode = element.firstChild();
QString name = childNode.nodeName();
QString value = element.text();
//qDebug() << uuid << name << value;
// Here you can add your own processing according to the received data
}
}
void UdpImageClient::started()
{
if (!timerData->isActive()) {
timerData->start();
}
}
void UdpImageClient::stop()
{
buffer.clear();
images.clear();
stopped = true;
this->wait();
udpSocket->disconnectFromHost();
if (timerData->isActive()) {
timerData->stop();
}
}
void UdpImageClient::setPackageSize(int packageSize)
{
if (packageSize <= 65507) {
this->packageSize = packageSize;
}
}
void UdpImageClient::setFlag(const QString &flag)
{
this->flag = flag;
}
void UdpImageClient::setServerIP(const QString &serverIP)
{
this->serverIP = serverIP;
}
void UdpImageClient::setServerPort(int serverPort)
{
this->serverPort = serverPort;
}
void UdpImageClient::writeData(const QString &body)
{
// structure xml character string
QStringList list;
list.append(QString("<ImageClient Uuid=\"%1\" Flag=\"%2\">").arg(DeviceFun::getUuid()).arg(flag));
list.append(body);
list.append("</ImageClient>");
// Call the general method to compose the complete data according to the protocol
QString data = DeviceFun::getSendXmlData(list.join(""), "IIMAGE:");
QByteArray buffer = data.toUtf8();
//udp Can only send 65507 Bytes of data =64K If it exceeds, it will fail to send
// So we need manual subcontracting , The packet on the Internet should be smaller
if (packageSize == 65500) {
udpSocket->writeDatagram(buffer, QHostAddress(serverIP), serverPort);
} else {
int len = buffer.length();
int count = len / packageSize + 1;
for (int i = 0; i < count; i++) {
QByteArray temp = buffer.mid(i * packageSize, packageSize);
udpSocket->writeDatagram(temp, QHostAddress(serverIP), serverPort);
}
}
emit sendData(buffer);
}
void UdpImageClient::sendImage(const QString &body)
{
writeData(QString("<ClientImage>%1</ClientImage>").arg(body));
}
void UdpImageClient::append(const QImage &image)
{
// We need locks here , Avoid data being pulled out
QMutexLocker locker(&mutexImage);
// Limit the maximum number of messages in the queue , Avoid crazy insertion when offline
if (this->isRunning() && images.count() < 10) {
images << image;
}
}
void UdpImageClient::clear()
{
QMutexLocker locker(&mutexImage);
images.clear();
}
版权声明
本文为[Flying clouds]所创,转载请带上原文链接,感谢
边栏推荐
- 移动端像素适配方案
- ORA-02292: 违反完整约束条件 (MIDBJDEV2.SYS_C0020757) - 已找到子记录
- Stickinengine architecture 12 communication protocol
- [forward] how to view UserData in Lua
- Event monitoring problem
- This project allows you to quickly learn about a programming language in a few minutes
- Configuration of AP hotspot on xunwei-imx6ull development board
- list转换map(根据key来拆分list,相同key的value为一个list)
- Application insights application insights use application maps to build request link views
- Cloudquery v1.2.0 release
猜你喜欢
Stickinengine architecture 11 message queue
ES6 learning notes (2): teach you to play with class inheritance and class objects
Why is the LS command stuck when there are too many files?
An article takes you to understand CSS gradient knowledge
2020-08-19:TCP是通过什么机制保障可靠性的?
How much disk space does a file of 1 byte actually occupy
September 9, 2020: naked writing algorithm: two threads print numbers 1-100 in turn.
An article taught you to use HTML5 SVG tags
Points to be considered when deleting mapping field of index in ES
Utility class functions (continuous update)
随机推荐
August 14, 2020: what are the execution engines for data tasks?
Unexpected element.. required element
Utility class functions (continuous update)
#JVM 类加载机制
Big data processing black Technology: revealing the parallel computing technology of Pb level data warehouse gaussdb (DWS)
2020-08-19: what mechanism does TCP ensure reliability?
What kind of music do you need to make for a complete game?
How much disk space does a new empty file take?
jenkins安装部署过程简记
Unity performance optimization
Windows 10 蓝牙管理页面'添加蓝牙或其他设备'选项点击无响应的解决方案
The native API of the future trend of the front end: web components
window系统 本机查找端口号占用方法
2020-09-03:裸写算法:回形矩阵遍历。
小熊派开发板实践:智慧路灯沙箱实验之真实设备接入
What grammar is it? ]
实验一
An article takes you to understand CSS3 picture border
ES6 learning notes (2): teach you to play with class inheritance and class objects
2020-08-15: under what circumstances should data tasks be optimized?