Detailed explanation of multi-mode input event distribution mechanism

One 、 Overview of multimode input

Multimode input service is designed to support developers to provide users with a variety of human-computer interaction methods , On the basis of continuous improvement and support for traditional input , Multimode input will also play OpenAtom OpenHarmony（ hereinafter referred to as “OpenHarmony”） The distributed advantage of , Improve the cross device interaction experience , Facing new scenes 、 New businesses provide system level support .

Multimode input Input The components are OpenHarmony System level input event management framework ; Connect a variety of input devices to the South , Gather multiple input events ( Key 、 touch ), Through unification / After standardized treatment , Distribute to consumers ( system service 、 application ).

The docking of southbound input devices includes many types of input devices , Such as ： A touch screen 、 mouse 、 keyboard 、 Touch Pad 、 The remote control .

Two 、 Input event and device status data flow introduction

First , Let's look at the key data flow under the multi-mode input architecture , It is convenient for users to have an in-depth understanding from initiating an interactive request , To the system 、 The whole process of application giving an interactive response .

As shown in the figure below , There are two types of key data flows ：

●  Input device status data flow ：

The input device status data describes the status changes of the input device and its device attribute information , Include ： Device insertion 、 Remove state 、 Unique identification of equipment 、 Equipment name 、 Input mode supported by the device .

Enter device status data , Through the kernel device driver, it is reported to the input device status management module of the multi-mode input server . In the input device management module, the global input device status is managed and maintained , At the same time, the device status will be encapsulated as a listening interface and provided to the upper business module to monitor the status of the system input peripherals .

●  Interactive input event data flow ：

Interactive input event data is used to describe the keyboard 、 mouse 、 Touch screen input events ; Keyboard events include ： Key code 、 Key timestamp 、 Information such as the device to which the key belongs ; Mouse events include ： mouse X/Y coordinate 、 Mouse button ( Such as ： Mouse left | in | Right ) Events, etc. ; Touch events include ： Time stamp 、 Touch the location X/Y Coordinates, etc .

The input event data is reported by the device driver to the input event receiving module to complete the forwarding of input events from kernel space to user space , Then the input event preprocessing module is given to complete the standardized processing of input events ( Key KeyCode Mapping standardization, etc ), Finally, the input event distribution module completes the event distribution with the system preset distribution mechanism and principles .

Refer to the following data flow diagram , We can clearly understand that after the user initiates an interactive request through the input device , Enter the whole process of event reporting and distribution .

chart 1 Input event and device status data flow diagram

notes ： Data flow diagram arrow schematic description

 

Enter the event pre-processing instructions of the event distribution module ：

1. The input event distribution process will first pass through the input event interception module , When an interceptor is registered , Entering the event will terminate and continue to report , The corresponding interceptor will intercept all input events . This event interception feature currently mainly supports barrier free mode .

2. When there is no interceptor registration , The input event will be reported to the input event listening module , System level applications ( Such as ： System settings 、 desktop ) By listening to input events , Support system level features ( Such as ： The status bar is hidden / Disappear, etc ).

3. The event monitoring module will not prevent the event from continuing to report ; Support event monitoring at the same time , Input events will continue to be reported .

4. Key events will be reported to the subscription key distribution module for processing , Distribute to the corresponding application for processing , The event distribution process ends .

5. Other touch screen events and mouse events will not go through the subscription key distribution module , It will continue to be reported to the application window for processing .

3、 ... and 、 Multimode input event distribution principle

1. mouse / Touch screen event distribution principles

mouse / Which target does the touch screen coordinate point to , The input event is distributed to the corresponding target .

mouse / Touch screen event distribution special scenario description ：

1. If there is no button on the mouse to press , Which target is the current mouse pointing to , Mouse input events are distributed to coordinate locked targets .

2. If a button on the mouse is pressed , Take the target locked by the mouse coordinate when the first button is pressed as the distribution target , Until all the buttons are lifted .

3. Touch screen input , Press the locked target with the first finger as the input event distribution target , Until all fingers are raised .

2. Key event distribution principle

The key event distribution takes the focus in the current user visual interface as the distribution target , Which target is the focus of the current interface , Key events are distributed to the corresponding target .

Four 、OpenHarmony 3.1 Interface description added in version

In order to better support upper layer applications and system services, real-time detection and processing of input device hot plug status change events , stay OpenHarmony 3.1 The new version is open JS API Interface , It can support passing JS API Monitor the hot plug event of the device . At the same time, it provides the way to obtain the unique identification of the hot plug input device by registering the callback interface . Input device hot plug monitoring interface and inputDevice.getDevice The detailed information of the hot plug device can be obtained by interface coordination , Include ： Enter device name 、 Input types supported by the device ( keyboard | Touch screen | mouse | Game handle ) etc. .

1. New interface description of multi-mode input subsystem

Input peripheral hot plug monitoring interface ：function on(type: "change", listener: Callback<DeviceListener>): void; Input peripheral to cancel listening interface ：function off(type: “change”, listener?: CallbackDeviceListener): void;

2. Add interface parameter description

【DeviceListener】

【ChangeType】 

3. Input device hot plug interface application scenario

Soft keyboard adaptive display ： In the text editing scenario , The input method monitors the hot plug operation of the physical keyboard input device , It can adaptively decide whether the soft keyboard is displayed . When there is a physical keyboard device , The soft keyboard does not need to be displayed , The user input operation is completed through the physical keyboard . When there is no physical keyboard , Input method pop-up soft keyboard , The user completes the input operation through the soft keyboard .

4. Example of hot plug interface of input device

After having a preliminary understanding of the mouse hot plug monitoring interface , Let's learn more about , How to use input device hot plug interface in actual development ：

1. Import module first

import inputDevice from '@ohos.multimodalInput.inputDevice';

2) The input device hot plug event monitoring is realized through the monitoring interface ：

//  Input method in the soft keyboard display logic by subscribing to the status of the physical keyboard ： Insert / Pull out //  according to isPhysicalkeyboardExist The value of determines whether the soft keyboard pops up ...let isPhysicalkeyboardExist = false;inputDevice.on("change", (callback) => { console.log("type: " + callback.type + ", deviceId: " + callback.deviceId); inputDevice.getDevice(callback.deviceId, (ret) => { console.log("The keyboard type of the device is: " + ret); if (ret == keyboard.ALPHABETIC_KEYBOARD && callback.type == 'add') { //  The hot plug device is keyboard  isPhysicalkeyboardExist = true; } else if (ret == keyboard.ALPHABETIC_KEYBOARD && callback.type == 'remove') { isPhysicalkeyboardExist = false; } });});...

 

3) By canceling the listening interface, the input device hot plug event cancels listening ：

listener: function(data) { console.log("type: " + data.type + ", deviceId: " + data.deviceId);}//  Cancel alone listener Listening in .inputDevice.off("change", this.listener);//  Cancel all listening inputDevice.off("change");//  After canceling listening , The soft keyboard pops up by default 

notes ： Cancel hot plug event listening interface , Enter the reference listener Optional ; When entering the reference belt listener On behalf of canceling a specific listening callback . When entering the reference, do not bring listener, It means to cancel all listening callbacks .

Through the above introduction , I'm sure you're right OpenHarmony We have a comprehensive understanding of the input event processing and distribution mechanism of the multi-mode input subsystem . At the same time, we are also interested in OpenHarmony 3.1 The new version of the input device hot plug monitoring interface is introduced in detail , More about the multimode input subsystem for developers API Welcome to the interface Gitee Learn more about ：

https://gitee.com/openharmony/docs/blob/master/zh-cn/application-dev/reference/apis/js-apis-inputdevice.md

 

I believe you have the above basic knowledge as a cushion in the subsequent development , In the subsequent development, it can be more flexible , Develop applications with better interactive experience . Looking forward to working with developers to build the ultimate user experience . Last , We look forward to working together to build , Can be found in OpenHarmony Community (https://gitee.com/openharmony) Discuss with each other .