当前位置:网站首页>Unity embeddedbrowser browser plug-in event communication
Unity embeddedbrowser browser plug-in event communication
2022-07-03 13:33:00 【foenix66】
Unity EmbeddedBrowser Browser plug-in event communication
Embedded Browser The browser plug-in is based on Chrome Embedded browser in the kernel , Embedded in many development languages and frameworks , Provide convenient web browsing support for developers .
Embedded Browser The browser works by running the browser process in the background , Analyze the website through the communication of the background browser , synthesis Texture Map frame , Pass to Unity, meanwhile Unity You can call the interface , Interact with browser processes .
Embedded This integration method is different from the mobile terminal WebView Integration mode (WebView Is to open up a piece of screen space for WebView Use , The caller and WebView There is not much correlation between ), Provide more free integration channels , Web pages can be displayed in 3D space and solid surface , More flexible use .
Simulate mouse input
Usually , Whether it's Screen Canvas Or three-dimensional Mesh Superficial Embedded page , The plug-in has provided a seamless input method , Provided internally MapPointerToBrowser It has helped users convert 3D coordinates to Browser Plane coordinates .
If we want to customize the input , Such as remote control , Send control coordinates and mouse button communication through the remote host , Drive non user hardware input , The usual approach is to introduce Windows DLL, The requirements can be achieved by simulating the mouse
#region DLLs
[DllImport("user32.dll")]
private static extern int SetCursorPos(int x, int y); // Set cursor position
[DllImport("user32.dll")]
private static extern bool GetCursorPos(ref int x, ref int y); // Get cursor position
[DllImport("user32.dll")]
static extern void mouse_event(MouseEventFlag flags, int dx, int dy, uint data, System.UIntPtr extraInfo); // Mouse events
// Method parameter description
// VOID mouse_event(
// DWORD dwFlags, // motion and click options
// DWORD dx, // horizontal position or change
// DWORD dy, // vertical position or change
// DWORD dwData, // wheel movement
// ULONG_PTR dwExtraInfo // application-defined information
// );
public enum MouseEventFlag : uint
{
Move = 0x0001,
LeftDown = 0x0002,
LeftUp = 0x0004,
RightDown = 0x0008,
RightUp = 0x0010,
MiddleDown = 0x0020,
MiddleUp = 0x0040,
XDown = 0x0080,
XUp = 0x0100,
Wheel = 0x0800,
VirtualDesk = 0x4000,
Absolute = 0x8000
}
#endregion
Calling code
IEnumerator MouseClick(int x, int y)
{
SetCursorPos(x, y);
mouse_event(MouseEventFlag.LeftDown, 0, 0, 0, System.UIntPtr.Zero);
yield return new WaitForSeconds(0.1f);
mouse_event(MouseEventFlag.LeftUp, 0, 0, 0, System.UIntPtr.Zero);
}
When debugging and using this method , Because the window resolution is not 1:1 Full screen causes coordinate mapping error 、 The window is not in the front desk 、 Interfere with normal mouse operation and so on , Use is very unfriendly .
call Embedded Interface direct input
View the source code BrowserInput.cs, You can see that the plug-in handles the input event code
private void HandleMouseInput() {
var handler = browser.UIHandler;
var mousePos = handler.MousePosition;
var currentButtons = handler.MouseButtons;
var mouseScroll = handler.MouseScroll;
if (mousePos != prevPos) {
BrowserNative.zfb_mouseMove(browser.browserId, mousePos.x, 1 - mousePos.y);
}
// ...
}
Core code
BrowserNative.zfb_mouseMove(browser.browserId, mousePos.x, 1 - mousePos.y);
EmbeddedBrowser.Browser Is virtual to Unity The use of Browser object , The actual browser object is BrowserNative A series of definitions DLL Interface .
browserId Is an array index , Tag a list of all actual browser instances
EmbeddedBrowser.Browser Add... When creating allBrowsers list
unsafeBrowserId = newId;
allBrowsers[unsafeBrowserId] = this;
Take a ,BrowserNative Contains a series of browser interfaces , Directly calling these interfaces can extend the functions defined by the plug-in .
Code excerpt
/** * Reports the mouse's current location. * x and y are in the range [0,1]. (0, 0) is top-left, (1, 1) is bottom-right */
public delegate void Calltype_zfb_mouseMove(int id, float x, float y);
public static Calltype_zfb_mouseMove zfb_mouseMove;
public delegate void Calltype_zfb_mouseButton(int id, MouseButton button, bool down, int clickCount);
public static Calltype_zfb_mouseButton zfb_mouseButton;
/** Reports a mouse scroll. One "tick" of a scroll wheel is generally around 120 units. */
public delegate void Calltype_zfb_mouseScroll(int id, int deltaX, int deltaY);
public static Calltype_zfb_mouseScroll zfb_mouseScroll;
/** * Report a key down/up event. Repeated "virtual" keystrokes are simulated by repeating the down event without * an interveneing up event. */
public delegate void Calltype_zfb_keyEvent(int id, bool down, int windowsKeyCode);
public static Calltype_zfb_keyEvent zfb_keyEvent;
/** * Report a typed character. This typically interleaves with calls to zfb_keyEvent */
public delegate void Calltype_zfb_characterEvent(int id, int character, int windowsKeyCode);
public static Calltype_zfb_characterEvent zfb_characterEvent;
/** Register a function to call when console.log etc. is called in the browser. */
public delegate void Calltype_zfb_registerConsoleCallback(int id, ConsoleFunc callback);
public static Calltype_zfb_registerConsoleCallback zfb_registerConsoleCallback;
public delegate void Calltype_zfb_evalJS(int id, string script, string scriptURL);
public static Calltype_zfb_evalJS zfb_evalJS;
call BrowserNative Interface , The only mark you need is browserId, This is in EmbeddedBrowser.Browser Is a protected variable
/** Handle to the native browser. */
[NonSerialized]
internal protected int browserId;
We add a public attribute to it
/** Handle to the native browser. */
[NonSerialized]
internal protected int browserId;
public int BrowserId {
get {
return browserId; } } // Increase public attribute exposure browserId
after , We can call it directly BrowserNative Interface simulates mouse input
public void UpdateMouseEvent(EventType eventType, ViewRect rect)
{
float x = rect.x / rect.width;
float y = rect.y / rect.height;
Vector2 pos = MapPointerToBrowser(x, y, viewRect);
if (eventType == EventType.MouseDown)
{
ZenFulcrum.EmbeddedBrowser.BrowserNative.zfb_mouseMove(browser.BrowserId, pos.x, pos.y); // Be careful :x,y Coordinate for 0-1 Range
ZenFulcrum.EmbeddedBrowser.BrowserNative.zfb_mouseButton(browser.BrowserId, ZenFulcrum.EmbeddedBrowser.BrowserNative.MouseButton.MBT_LEFT, true, 1);
}
else if (eventType == EventType.MouseUp)
{
ZenFulcrum.EmbeddedBrowser.BrowserNative.zfb_mouseButton(browser.BrowserId, ZenFulcrum.EmbeddedBrowser.BrowserNative.MouseButton.MBT_LEFT, false, 1);
}
else if (eventType == EventType.MouseMove)
{
ZenFulcrum.EmbeddedBrowser.BrowserNative.zfb_mouseMove(browser.BrowserId, pos.x, pos.y); // Be careful :x,y Coordinate for 0-1 Range
}
}
边栏推荐
- rxjs Observable filter Operator 的实现原理介绍
- Flink SQL knows why (17): Zeppelin, a sharp tool for developing Flink SQL
- Heap structure and heap sort heapify
- MapReduce implements matrix multiplication - implementation code
- regular expression
- IBEM 数学公式检测数据集
- 服务器硬盘冷迁移后网卡无法启动问题
- 静态链表(数组的下标代替指针)
- Fabric. JS three methods of changing pictures (including changing pictures in the group and caching)
- json序列化时案例总结
猜你喜欢
Flink SQL knows why (7): haven't you even seen the ETL and group AGG scenarios that are most suitable for Flink SQL?
Kivy教程之 盒子布局 BoxLayout将子项排列在垂直或水平框中(教程含源码)
使用Tensorflow进行完整的深度神经网络CNN训练完成图片识别案例2
rxjs Observable filter Operator 的实现原理介绍
Today's sleep quality record 77 points
CVPR 2022 | interpretation of 6 excellent papers selected by meituan technical team
这本数学书AI圈都在转,资深ML研究员历时7年之作,免费电子版可看
Libuv库 - 设计概述(中文版)
使用tensorflow进行完整的DNN深度神经网络CNN训练完成图片识别案例
35道MySQL面试必问题图解,这样也太好理解了吧
随机推荐
Flink SQL knows why (7): haven't you even seen the ETL and group AGG scenarios that are most suitable for Flink SQL?
Realize the recognition and training of CNN images, and process the cifar10 data set and other methods through the tensorflow framework
常见的几种最优化方法Matlab原理和深度分析
Server coding bug
顺序表(C语言实现)
网上开户哪家证券公司佣金最低,我要开户,网上客户经理开户安全吗
JS convert pseudo array to array
Servlet
35道MySQL面试必问题图解,这样也太好理解了吧
File uploading and email sending
Can newly graduated European college students get an offer from a major Internet company in the United States?
Anan's doubts
Road construction issues
研发团队资源成本优化实践
已解决(机器学习中查看数据信息报错)AttributeError: target_names
[how to solve FAT32 when the computer is inserted into the U disk or the memory card display cannot be formatted]
双链笔记 RemNote 综合评测:快速输入、PDF 阅读、间隔重复/记忆
AI scores 81 in high scores. Netizens: AI model can't avoid "internal examination"!
CVPR 2022 | 美团技术团队精选6篇优秀论文解读
When updating mysql, the condition is a query