当前位置:网站首页>Libuv thread
Libuv thread
2022-07-06 09:29:00 【What are you afraid of? The truth is infinite】
Threads
wait a minute ! Why should we talk about threads ? The event loop (event loop) It should not be used for web Programming method ?( If you are right about event loop, Not very well understood , You can see here ). Oh , No no . Threads are still an important means for processors to complete tasks . Threads may therefore come in handy , Although it will make you have to deal with all kinds of primitive synchronization problems .
Threads will be used internally , Used to fake the illusion of asynchrony when executing system calls .libuv Threads can also make programs execute a blocked task asynchronously . The method is to generate a large number of new threads , Then collect the results returned by thread execution .
At present, there are two dominant thread libraries :windows Thread implementation and POSIX Of pthread.libuv The thread of API And pthread Of API It is very close in usage and semantics .
It is worth noting that ,libuv The thread module of is self-contained . such as , Other functional modules need to rely on event loop And callback principle , But threads are not like this . They are not constrained , It will block when necessary , A signal error is generated by the return value , And as the following example demonstrates , Don't need to event loop In the implementation of .
Because the thread API On different system platforms , Syntax and semantics are not very similar , The degree of support varies . in consideration of libuv Cross-platform features ,libuv Supported threads API The number is very limited .
Finally, I want to emphasize one sentence : Only one main thread , There is only one on the main thread event loop. There will be no other threads that interact with the main thread .( Unless used uv_async_send).
Core thread operations
The following example is not very complicated , You can use uv_thread_create()
Start a thread , Reuse uv_thread_join()
Wait for it to end .
thread-create/main.c
int main() {
int tracklen = 10;
uv_thread_t hare_id;
uv_thread_t tortoise_id;
uv_thread_create(&hare_id, hare, &tracklen);
uv_thread_create(&tortoise_id, tortoise, &tracklen);
uv_thread_join(&hare_id);
uv_thread_join(&tortoise_id);
return 0;
}
TIP
stay Unix On uv_thread_t It's just pthread_t Another name for , But this is just a concrete implementation , Don't rely too much on it , Think this is always true .
uv_thread_t The second parameter of points to the address of the function to be executed . The last parameter is used to pass custom parameters . Final , function hare Will execute in the new thread , Scheduled by the operating system .
thread-create/main.c
int main() {
int tracklen = 10;
uv_thread_t hare_id;
uv_thread_t tortoise_id;
uv_thread_create(&hare_id, hare, &tracklen);
uv_thread_create(&tortoise_id, tortoise, &tracklen);
uv_thread_join(&hare_id);
uv_thread_join(&tortoise_id);
return 0;
}
uv_thread_join
Unlike pthread_join
like that , Allow the thread to return a value to the parent thread through the second parameter . Want to pass values , Inter thread communication must be used Inter-thread communication.
Synchronization Primitives
Because the focus of this tutorial is not on threads , So I only listed libuv API Some magical places in . You can read the rest by yourself pthreads Manuals .
Mutexes
libuv The mutex function on and pthread There is a one-to-one mapping on . If the pthread Upper mutex I don't know much. I can see
here :POSIX Threads Programming | LLNL HPC Tutorials
libuv mutex functions
UV_EXTERN int uv_mutex_init(uv_mutex_t* handle);
UV_EXTERN void uv_mutex_destroy(uv_mutex_t* handle);
UV_EXTERN void uv_mutex_lock(uv_mutex_t* handle);
UV_EXTERN int uv_mutex_trylock(uv_mutex_t* handle);
UV_EXTERN void uv_mutex_unlock(uv_mutex_t* handle)
uv_mutex_init And uv_mutex_trylock After successful execution , return 0, Or when it's wrong , Return error code .
If libuv Debugging mode is turned on when compiling ,uv_mutex_destroy(), uv_mutex_lock() and uv_mutex_unlock() Will be called where there is an error abort() interrupt . Allied ,uv_mutex_trylock() It will also interrupt when an error occurs , Instead of going back EAGAIN and EBUSY.
Recursively calling mutex functions is supported on some system platforms , But you can't rely too much on . Because, for example, in BSD Recursively calling the mutex function on the will return an error , For example, when you are going to use the mutex function to lock a locked critical area again , Will go wrong . such as , Like the following example .
uv_mutex_lock(a_mutex);
uv_thread_create(thread_id, entry, (void *)a_mutex);
uv_mutex_lock(a_mutex);
It can be used to wait for other threads to initialize some variables and then release a_mutex lock , But the second call uv_mutex_lock(), In debugging mode, the program will crash , Or it returns an error .
NOTE
stay linux Recursive locking is supported in , But in libuv Of API Not implemented in .
Lock
Read write locking is a more granular implementation mechanism . Two reader threads can read data from the shared area at the same time . When the reader occupies the read-write lock in read mode , The writer can no longer possess it . When the writer holds the lock in write mode , No other writer or reader can possess it . Read / write locks are very common in database operations , Here is a toy like example :
ocks/main.c - simple rwlocks
#include <stdio.h>
#include <uv.h>
uv_barrier_t blocker;
uv_rwlock_t numlock;
int shared_num;
void reader(void *n)
{
int num = *(int *)n;
int i;
for (i = 0; i < 20; i++) {
uv_rwlock_rdlock(&numlock);
printf("Reader %d: acquired lock\n", num);
printf("Reader %d: shared num = %d\n", num, shared_num);
uv_rwlock_rdunlock(&numlock);
printf("Reader %d: released lock\n", num);
}
uv_barrier_wait(&blocker);
}
void writer(void *n)
{
int num = *(int *)n;
int i;
for (i = 0; i < 20; i++) {
uv_rwlock_wrlock(&numlock);
printf("Writer %d: acquired lock\n", num);
shared_num++;
printf("Writer %d: incremented shared num = %d\n", num, shared_num);
uv_rwlock_wrunlock(&numlock);
printf("Writer %d: released lock\n", num);
}
uv_barrier_wait(&blocker);
}
int main()
{
uv_barrier_init(&blocker, 4);
shared_num = 0;
uv_rwlock_init(&numlock);
uv_thread_t threads[3];
int thread_nums[] = {1, 2, 1};
uv_thread_create(&threads[0], reader, &thread_nums[0]);
uv_thread_create(&threads[1], reader, &thread_nums[1]);
uv_thread_create(&threads[2], writer, &thread_nums[2]);
uv_barrier_wait(&blocker);
uv_barrier_destroy(&blocker);
uv_rwlock_destroy(&numlock);
return 0;
}
Try to execute the above procedure , See how many times readers will execute synchronously . When there are multiple writers , The scheduler will give them high priority . therefore , If you join two readers , You will see that all readers tend to end before readers get the chance to lock .
In the example above , We also used barriers . So the main thread waits until all threads have ended , Finally, recycle the barrier and lock .
Others:
libuv Also supports Semaphore , Condition variables, and barrier , and API How to use and pthread The usage in is very similar .( If you are not familiar with the above three nouns , You can see here ).
/* Initialize guard */
static uv_once_t once_only = UV_ONCE_INIT;
int i = 0;
void increment() {
i++;
}
void thread1() {
/* ... work */
uv_once(once_only, increment);
}
void thread2() {
/* ... work */
uv_once(once_only, increment);
}
int main() {
/* ... spawn threads */
}
When all threads have finished executing ,i == 1
.
stay libuv Of v0.11.11 In the version , Launched uv_key_t Structure and operation Thread local storage TLS Of API, The method of use is the same as pthread similar .
libuv work queue
uv_queue_work() Is a convenient function , It enables an application to run tasks on different threads , When the task is done , The callback function will be triggered . It seems very simple , But its real attraction is that it enables any third-party library to event-loop By . When using event-loop When , The most important thing is not to let loop Thread blocking , Or high execution cpu Occupied program , Because it will make loop Slow down ,loop event The high-efficiency characteristics of can't be brought into full play .
However , Many programs with blocking features ( For example, the most common I/O) Use open new threads to respond to new requests ( The most classic ‘ A customer , One thread ‘ Model ). Use event-loop You can provide another way to implement .libuv Provides a good abstraction , So that you can use it well .
Here is a good example , Inspiration comes from <<nodejs is cancer>>. We are going to execute fibonacci The sequence , And sleep for a while , But it will block and cpu Tasks that take a long time are assigned to different threads , So that it won't block event loop Other tasks on the Internet .
queue-work/main.c - lazy fibonacci
void fib(uv_work_t *req) {
int n = *(int *) req->data;
if (random() % 2)
sleep(1);
else
sleep(3);
long fib = fib_(n);
fprintf(stderr, "%dth fibonacci is %lu\n", n, fib);
}
void after_fib(uv_work_t *req, int status) {
fprintf(stderr, "Done calculating %dth fibonacci\n", *(int *) req->data);
}
The task function is very simple , Also not running on threads .uv_work_t
Is the key clue , You can go through void *data
Transfer any data , Use it to complete the communication task between threads . But be sure , When you change something when multiple threads are running , Be able to use appropriate locks .
The trigger is uv_queue_work
:
queue-work/main.c
int main() {
loop = uv_default_loop();
int data[FIB_UNTIL];
uv_work_t req[FIB_UNTIL];
int i;
for (i = 0; i < FIB_UNTIL; i++) {
data[i] = i;
req[i].data = (void *) &data[i];
uv_queue_work(loop, &req[i], fib, after_fib);
}
return uv_run(loop, UV_RUN_DEFAULT);
}
Thread functions fbi() Will run in different threads , Pass in uv_work_t Structural parameters , once fib() The function returns ,after_fib() Will be event loop Thread call in , Then it is passed into the same structure .
To encapsulate blocked libraries , A common pattern is to use Utilities — An Introduction to libuv To exchange data .
from libuv 0.9.4 After the version , Added function uv_cancel(). It can be used to cancel tasks in the work queue . Only tasks that have not yet started can be cancelled , If the task has been started or completed ,uv_cancel() The call fails .
When the user wants to terminate the program ,uv_cancel() It can be used to clean up tasks waiting to be executed in the task queue . for example , A music player can sort songs by singer's name , If the user wants to exit the program at this time ,uv_cancel() You can exit quickly , Instead of waiting for the execution of the task queue , Quit again .
Let's make some changes to the above program , For demonstration uv_cancel() Usage of . First, let's register a function to handle interrupts .
queue-cancel/main.c
int main() {
loop = uv_default_loop();
int data[FIB_UNTIL];
int i;
for (i = 0; i < FIB_UNTIL; i++) {
data[i] = i;
fib_reqs[i].data = (void *) &data[i];
uv_queue_work(loop, &fib_reqs[i], fib, after_fib);
}
uv_signal_t sig;
uv_signal_init(loop, &sig);
uv_signal_start(&sig, signal_handler, SIGINT);
return uv_run(loop, UV_RUN_DEFAULT);
}
When the user passes Ctrl+C
When the signal is triggered ,uv_cancel()
Reclaim all tasks in the task queue , If the task has been started or completed ,uv_cancel()
return 0.
queue-cancel/main.c
void signal_handler(uv_signal_t *req, int signum)
{
printf("Signal received!\n");
int i;
for (i = 0; i < FIB_UNTIL; i++) {
uv_cancel((uv_req_t*) &fib_reqs[i]);
}
uv_signal_stop(req);
}
For tasks that have been successfully cancelled , Parameters of his callback function status
Will be set to UV_ECANCELED
.
queue-cancel/main.c
void after_fib(uv_work_t *req, int status) {
if (status == UV_ECANCELED)
fprintf(stderr, "Calculation of %d cancelled.\n", *(int *) req->data);
}
uv_cancel()
Function can also be used in uv_fs_t
and uv_getaddrinfo_t
On the request . For a series of file system operation functions ,uv_fs_t.errorno
Will also be set to UV_ECANCELED
.
Tip
A well-designed program , It should be able to terminate a long-time task that has already started running .
Such a worker could periodically check for a variable that only the main process sets to signal termination.
Inter-thread communication
A lot of times , You want running threads to send messages to each other . For example, you are running a long-lasting task ( May use uv_queue_work), But you need to monitor its progress in the main thread . Here's a simple example , Demonstrates a download management program to show users the progress of each download thread .
progress/main.c
uv_loop_t *loop;
uv_async_t async;
int main() {
loop = uv_default_loop();
uv_work_t req;
int size = 10240;
req.data = (void*) &size;
uv_async_init(loop, &async, print_progress);
uv_queue_work(loop, &req, fake_download, after);
return uv_run(loop, UV_RUN_DEFAULT);
}
Because asynchronous thread communication is based on event-loop Of , So although all threads can be the sender , But only in event-loop The thread on can be the receiver ( Or say event-loop It's the receiver ). In the above code , When the asynchronous monitor receives the signal ,libuv Will fire the callback function (print_progress).
WARNING
It should be noted : Because the message is sent asynchronously , When uv_async_send After being called in another thread , The callback function may be called immediately , It may also be called at a later time .libuv It is also possible to call more than once uv_async_send, But the callback function was called only once . The only guarantee is : Thread calling uv_async_send The callback function can then be called at least once . If you have no uncalled uv_async_send, Then the callback function will not be called . If you call twice ( above ) Of uv_async_send, and libuv There is no chance to run the callback function for the time being , be libuv It may be called many times uv_async_send Then call the callback function only once , Your callback function will never be called twice in one event ( Or many times ).
progress/main.c
void fake_download(uv_work_t *req) {
int size = *((int*) req->data);
int downloaded = 0;
double percentage;
while (downloaded < size) {
percentage = downloaded*100.0/size;
async.data = (void*) &percentage;
uv_async_send(&async);
sleep(1);
downloaded += (200+random())%1000; // can only download max 1000bytes/sec,
// but at least a 200;
}
}
In the above download function , We modified the progress display , Use uv_async_send
Send progress information . Remember :uv_async_send
It is also non blocking , It will return to .
progress/main.c
void print_progress(uv_async_t *handle) {
double percentage = *((double*) handle->data);
fprintf(stderr, "Downloaded %.2f%%\n", percentage);
}
function print_progress
It's standard libuv Pattern , Extract data from the monitor .
Finally, the most important thing is to recycle the monitor .
progress/main.c
void after(uv_work_t *req, int status) {
fprintf(stderr, "Download complete\n");
uv_close((uv_handle_t*) &async, NULL);
}
At the end of the example , Let's say data
Abuse of domain ,bnoordhuis Point out the use of data
There may be thread safety problems in the domain ,uv_async_send()
In fact, it just woke up event-loop. Mutexes or read-write locks can be used to ensure the correctness of the execution sequence .
Note
Mutexes and read-write locks do not work correctly in signal processing functions , however uv_async_send Sure .
One needs to use uv_async_send The scene of , When calling a library that requires thread interaction . for example , Let's take an example node.js in V8 An example of an engine , Both context and object are related to v8 The engine thread is bound , From another thread directly to v8 Requesting data will result in uncertain results . however , Considering many now nodejs All modules of are bound to third-party libraries , It can be like the following , Solve this problem :
1. stay node in , Third party libraries will be established javascript Callback function for , So that when the callback function is called , Can return more information .
var lib = require('lib');
lib.on_progress(function() {
console.log("Progress");
});
lib.do();
// do other stuff
2.lib.do It should be non blocking , But the third-party library is blocked , So you need to call uv_queue_work function .
3. To complete a task in another thread, you want to call progress Callback function for , But not directly with v8 signal communication , So we need to uv_async_send function .
4. In the main thread (v8 Threads ) Asynchronous callback function called in , Will be in v8 With the cooperation of javscript Callback function for .( in other words , The main thread will call the callback function , And to provide v8 analysis javascript The function of , So that they can complete the task ).
边栏推荐
- Global and Chinese market of bank smart cards 2022-2028: Research Report on technology, participants, trends, market size and share
- CSP student queue
- [oc]- < getting started with UI> -- common controls - prompt dialog box and wait for the prompt (circle)
- Selenium+Pytest自动化测试框架实战
- O & M, let go of monitoring - let go of yourself
- 068.查找插入位置--二分查找
- Full stack development of quartz distributed timed task scheduling cluster
- LeetCode41——First Missing Positive——hashing in place & swap
- [oc]- < getting started with UI> -- learning common controls
- [Yu Yue education] reference materials of complex variable function and integral transformation of Shenyang University of Technology
猜你喜欢
Pytest's collection use case rules and running specified use cases
Research and implementation of hospital management inpatient system based on b/s (attached: source code paper SQL file)
Redis core configuration
Multivariate cluster analysis
Pytest参数化你不知道的一些使用技巧 /你不知道的pytest
The five basic data structures of redis are in-depth and application scenarios
Redis之哨兵模式
postman之参数化详解
一改测试步骤代码就全写 为什么不试试用 Yaml实现数据驱动?
Mapreduce实例(四):自然排序
随机推荐
Redis cluster
Global and Chinese market of linear regulators 2022-2028: Research Report on technology, participants, trends, market size and share
AcWing 2456. 记事本
Redis之主从复制
【图的三大存储方式】只会用邻接矩阵就out了
Publish and subscribe to redis
Leetcode:608 树节点
The carousel component of ant design calls prev and next methods in TS (typescript) environment
MySQL数据库优化的几种方式(笔面试必问)
Kratos战神微服务框架(二)
leetcode-14. Longest common prefix JS longitudinal scanning method
[oc]- < getting started with UI> -- learning common controls
Redis之Geospatial
为拿 Offer,“闭关修炼,相信努力必成大器
Redis' performance indicators and monitoring methods
Global and Chinese markets for modular storage area network (SAN) solutions 2022-2028: Research Report on technology, participants, trends, market size and share
英雄联盟轮播图自动轮播
Activiti7工作流的使用
Design and implementation of online shopping system based on Web (attached: source code paper SQL file)
Advance Computer Network Review(1)——FatTree