QT simulates mouse events and realizes clicking, double clicking, moving and dragging

There was a project involving remote desktop control before , You need to send some instructions to the remote computer , Realize simple desktop click 、 Move 、 Dragging and other functions , There was no time to achieve it well , I studied it again today , So this record .

Although I use QT, But the core is still systematic API, So other platforms should be the same .

I don't say much nonsense , Go straight to the code .

---

mouseacts.h：

#ifndef MOUSEACTS_H
#define MOUSEACTS_H

#include <QWidget>
#include "Windows.h"// This is the introduction of Windows Operating system API
#include "WinUser.h"
#include <QDesktopWidget>
#include <QApplication>
#include <QDebug>

#pragma comment(lib, "User32.lib")// This must have , Otherwise, it can't be compiled 

class MouseActs : public QWidget
{
    
    Q_OBJECT
public:
    MouseActs(QWidget *parent = nullptr);
    
public slots:
    void getClickedPos() ;// Get the global coordinate value at the mouse click （ Screen coordinates ）
    static void doPressAct(int,int,int,int,int);// Execute mouse action .
};


#endif // MOUSEACTS_H

---

mouseacts.cpp

#include "mouseacts.h"
MouseActs::MouseActs(QWidget *parent)
    : QWidget (parent)
{
    

}

// Get the global coordinate value at the mouse click （ Screen coordinates ）
void MouseActs::getClickedPos ()
{
    
    GetCursorPos(&pos);
}

// Execute mouse action 
void MouseActs::doPressAct(int x,int y,int x1,int y1,int type_id)
{
    
    SetCursorPos(x,y);// You must first set the initial position of the mouse , Otherwise, it will lead to failure 

    // Get screen size 
    QDesktopWidget * desktop = QApplication::desktop();
    QRect screen_rect = desktop->screenGeometry(0);
    int desktop_width = screen_rect.width();
    int desktop_height = screen_rect.height();
    qDebug()<<"desktop_width"<<desktop_width;
    qDebug()<<"desktop_height"<<desktop_height;

    if(type_id==1)
    {
    
        // Click the left mouse button 
        mouse_event(MOUSEEVENTF_ABSOLUTE|MOUSEEVENTF_LEFTDOWN,x, y, 0, 0);// Press down 
        mouse_event(MOUSEEVENTF_ABSOLUTE|MOUSEEVENTF_LEFTUP,x, y, 0, 0);// Release 
    }
    else if(type_id==2)
    {
    
        // Double click with the left mouse button 
        mouse_event(MOUSEEVENTF_ABSOLUTE|MOUSEEVENTF_LEFTDOWN,x, y, 0, 0);
        mouse_event(MOUSEEVENTF_ABSOLUTE|MOUSEEVENTF_LEFTUP,x, y, 0, 0);
        mouse_event(MOUSEEVENTF_ABSOLUTE|MOUSEEVENTF_LEFTDOWN,x, y, 0, 0);
        mouse_event(MOUSEEVENTF_ABSOLUTE|MOUSEEVENTF_LEFTUP,x, y, 0, 0);

    }else if(type_id==3)
    {
    
        // Right click 
        mouse_event(MOUSEEVENTF_ABSOLUTE|MOUSEEVENTF_RIGHTDOWN,x, y, 0, 0);
        mouse_event(MOUSEEVENTF_ABSOLUTE|MOUSEEVENTF_RIGHTUP,x, y, 0, 0);
    }else if(type_id==4)
    {
    
        // Double click with the right mouse button 
        mouse_event(MOUSEEVENTF_ABSOLUTE|MOUSEEVENTF_RIGHTDOWN,x, y, 0, 0);
        mouse_event(MOUSEEVENTF_ABSOLUTE|MOUSEEVENTF_RIGHTUP,x, y, 0, 0);
        mouse_event(MOUSEEVENTF_ABSOLUTE|MOUSEEVENTF_RIGHTDOWN,x, y, 0, 0);
        mouse_event(MOUSEEVENTF_ABSOLUTE|MOUSEEVENTF_RIGHTUP,x, y, 0, 0);
    }else if(type_id==5)
    {
    
        // Mouse movement 
        mouse_event(MOUSEEVENTF_ABSOLUTE|MOUSEEVENTF_MOVE ,
        			x * 65536 / desktop_width,y * 65536 / desktop_height,0,GetMessageExtraInfo());

    }else if(type_id==6)
    {
    
        // The mouse to drag and drop 
        mouse_event(MOUSEEVENTF_ABSOLUTE|MOUSEEVENTF_LEFTDOWN,x, y, 0, 0);
        mouse_event(MOUSEEVENTF_ABSOLUTE|MOUSEEVENTF_MOVE ,
        			x1 * 65536 / desktop_width,y1 * 65536 / desktop_height,0,GetMessageExtraInfo());
        mouse_event(MOUSEEVENTF_ABSOLUTE|MOUSEEVENTF_LEFTUP,x1, y1, 0, 0);

        qDebug()<<"--"<<x<<","<<y<<"to"<<x1<<","<<y1;
    }
}

---

In this way, direct external calls are ok

void MainWidget::on_btn_test_1_clicked()
{
    
    MouseActs macts;
    macts.doPressAct(ui->lineEdit->text().toInt(),ui->lineEdit_2->text().toInt(),0,0,1);
}

void MainWidget::on_btn_test_2_clicked()
{
    
    MouseActs::doPressAct(ui->lineEdit->text().toInt(),ui->lineEdit_2->text().toInt(),0,0,2);
}

void MainWidget::on_btn_test_3_clicked()
{
    
    MouseActs::doPressAct(ui->lineEdit->text().toInt(),ui->lineEdit_2->text().toInt(),0,0,3);
}

void MainWidget::on_btn_test_4_clicked()
{
    
    MouseActs::doPressAct(ui->lineEdit->text().toInt(),ui->lineEdit_2->text().toInt(),0,0,4);
}

void MainWidget::on_btn_test_5_clicked()
{
    
    MouseActs::doPressAct(ui->lineEdit->text().toInt(),ui->lineEdit_2->text().toInt(),0,0,5);
}

void MainWidget::on_btn_test_6_clicked()
{
    
    MouseActs::doPressAct(ui->lineEdit->text().toInt(),ui->lineEdit_2->text().toInt(),1000,1000,6);
}

The above codes are valid through personal testing , There is reference to this article , If you are interested, you can have a look at . Reference 1
The main functions are doPressAct This interface , according to type_id To judge the code , To achieve single double click with left and right keys , Move and drag

well , Next, let's talk about the three interfaces involved .

1.GetCursorPos(&pos); Get mouse position , I didn't use this , But actually qt in , Call directly this->pos() That's all right. .
2.SetCursorPos(x,y); Set mouse position , If you move, you can use this directly
3.mouse_event Mouse events , Core implementation interface

Focus on the analysis of mouse_event Interface

[System.Runtime.InteropServices.DllImport(“user32”)] private static
extern int mouse_event(int dwFlags, int dx, int dy, int cButtons, int
dwExtraInfo);

Parameters significance dwFlags Long, One of the symbols in the following table or a combination of them dx,dy
Long, according to MOUSEEVENTF_ABSOLUTE sign , Appoint x,y The absolute or relative position of the direction cButtons Long, Not used
dwExtraInfo Long, Not used

dwFlags constant significance

const int MOUSEEVENTF_MOVE = 0x0001; Move the mouse
const int MOUSEEVENTF_LEFTDOWN = 0x0002; Simulate left mouse button press
const int MOUSEEVENTF_LEFTUP = 0x0004; Simulate the left mouse button lifting
const int MOUSEEVENTF_RIGHTDOWN = 0x0008; Simulate right mouse button press
const int MOUSEEVENTF_RIGHTUP = 0x0010; Simulate right mouse button lifting
const int MOUSEEVENTF_MIDDLEDOWN = 0x0020; Simulate middle mouse button press
const int MOUSEEVENTF_MIDDLEUP = 0x0040; Simulate middle mouse button lifting
const int MOUSEEVENTF_ABSOLUTE = 0x8000; Indicate whether absolute coordinates are used

According to dwFlags sign , as well as dx、dy Set up , It can basically realize the movement of the mouse 、 Click on 、 double-click 、 Right click 、 Middle click and other functions , For details, please refer to my code .

Here we will focus on the mobile situation , That is to say MOUSEEVENTF_MOVE The situation of ！！
This one. x、y The coordinates seem to be different from the click , Require special treatment , For details, please refer to my code
Here is an explanation given on the Internet , Although I don't understand it very well

In the event of simulating mouse movement , The value of flag bit is different , The meaning of entering coordinates is also different .
In a nutshell , add to MOUSEEVENTF_ABSOLUTE The flag bit indicates that the mouse moves through absolute coordinates , At this time, the coordinates should be transformed .
The cursor is divided into 65535 Small pieces , It can be converted as follows ：

Copy the code as follows :

double fx = x *(65535.0f / fScreenWidth);
double fy = y *(65535.0f / fScreenHeight);
If not used MOUSEEVENTF_ABSOLUTE Sign a , Then the coordinate is the displacement relative to the previous coordinate .

Above , The input coordinates have to undergo some conversion operations , But first we need to know fScreenWidth and fScreenHeight The width and height of , That is, the width and height of the screen size

Different monitors naturally have different resolutions , I got it through this ：

	// Get screen size 
    QDesktopWidget * desktop = QApplication::desktop();
    QRect screen_rect = desktop->screenGeometry(0);//0 It refers to the first display , If there are multiple monitors, you need to get another number 
    int desktop_width = screen_rect.width();
    int desktop_height = screen_rect.height();
    qDebug()<<"desktop_width"<<desktop_width;
    qDebug()<<"desktop_height"<<desktop_height;

thus , Simply simulate mouse event control . Of course , This is all done in one step , Like some dynamic effects of moving and dragging , It may be realized by timer or other means , But that's basically it API The function of .

You can also tell me any mistakes and additions in the comment area , thank you .