Scheme of printing statistical information in log

Summary

Currently maintained c++ service , There will be lock competition when printing logs . First use brpc Positioning lock competition link , Then put forward a centralized improvement plan , And their advantages and disadvantages .

contention Flame chart

Use brpc Print in the pressure measurement environment contention Flame chart .

The circle on the left is the access data of the processing service , The receiving volume will be recorded ,os type , Business party type and other statistical parameters ;

The two smaller circles on the right are two sending downstream , It will record the statistical parameters such as the quantity sent ;

They all use the same lock , When statistics need to be written , Will compete for this lock ;

The size of the data next to the edge of the connection box , You can see that the main lock competition comes from the business function processing the received data . The reason is that our service receives data item by item , But sending data is batch , So the quantity is not equal .

The improved scheme

Each statistical index corresponds to a lock

benefits ： The scope of the lock is controlled with minimum force

Disadvantage ： When there are many places that need to write a large amount of statistical information , This kind of scene is not suitable

spinlocks

benefits ： Not going to happen cs

Disadvantage ： When the critical zone is long , Yes cpu The waste of cannot be ignored

Use atomic

benefits ： The basic types are x86 The architecture is unlocked

Disadvantage ：atomic Although variables are lighter than lock assignment , But it is heavier than ordinary parameter replication . When there are many statistical parameters , This part of the time cost is also huge

contrast atomic And the time cost of locking

atomic

#include<iostream>
#include<atomic>
#include<thread>

using namespace std;

atomic<int> a(0);

void sum() {
    for(int i=0; i!=10000000; i++) {
        a++;
    }
}

int main() {
    // cout << a.is_lock_free() << endl;
    //  Start four threads 
    thread t1(sum);
    thread t2(sum);
    thread t3(sum);
    thread t4(sum);
    t1.join();
    t2.join();
    t3.join();
    t4.join();
    int ret = a.load();
    cout << ret << endl;
    return 0;
}

mutex

#include<iostream>
#include<thread>
#include<mutex>

using namespace std;

mutex m;
int a = 0;
int b = 0;

void sum() {
    for(int i=0; i!=10000000; i++) {
        m.lock();
        a++;
        m.unlock();
    }
}

int main() {
    //  Start four threads 
    thread t1(sum);
    thread t2(sum);
    thread t3(sum);
    thread t4(sum);
    t1.join();
    t2.join();
    t3.join();
    t4.join();
    cout << a << endl;
    return 0;
}

function

[email protected] cpptest % time ./atomic 
40000000
./atomic  3.41s user 0.01s system 395% cpu 0.865 total
[email protected] cpptest % time ./mutex
40000000
./mutex  2.13s user 4.70s system 317% cpu 2.153 total

Visible use atomic after , The overhead of system calls is significantly reduced , Improved performance

Dual version switching

The idea of the plan is Save statistics using dual versions . When it is necessary to record statistical parameters , First get the current version index, You can get which version to write . Open a new thread , Switch every other time interval index, At the same time, print the previous statistical parameters

Code

#include<iostream>
#include<atomic>
#include<vector>
#include<thread>
#include<unistd.h>

using namespace std;

struct Info {
    int a{0};
    void clear() {a=0;}
};

atomic<int> Index(0); //  Point to the current version 
vector<Info*> vec_infos; //  Save dual version information 

void print_info(int idx) { //  Print idx Statistical parameters of version 
    cout << vec_infos.at(idx)->a << endl;
}

void clear_info(int idx) { //  take idx The statistical parameters of the version are cleared 
    vec_infos.at(idx)->a = 0;
}

void printInfo() {
    int i=0; //  Ensure that the function can exit 
    int idx=0;
    while(i!=10) {
        sleep(1);
        int next_idx = (idx+1) % 2;
        clear_info(next_idx); //  To be used Info Zero clearing 
        Index.store(next_idx); //  Switch index, The statistics written later will be put into another Info in 
        print_info(idx); //  Statistics before printing 
        idx = next_idx; //  Point to the new index
        i++;
    }
}

void writeInfo() { //  Write statistical parameters 
    while(true) {
        int idx = Index.load(); //  Get current index
        vec_infos.at(idx)->a++;
    }
}

int main() {
    Info* info1 = new Info();
    Info* info2 = new Info();
    vec_infos.push_back(info1);
    vec_infos.push_back(info2); //  Dual version statistical parameter initialization 
    thread t1(printInfo); //  Start the print thread 
    thread t2(writeInfo); //  Start the thread that simulates writing statistical parameters 

    t1.join();
    return 0;
}

This code is not rigorous , When writing statistical parameters , You may have finished printing , Cause data loss . First of all, the probability is not great ,atomic Of store The time cost of operation is greater than that of ordinary variable assignment ; The second is for statistical information , Even if a small amount of data is lost , No big problem ; Then we can put intervel The time is divided into two parts , stay store Operation and print_info Wait a while before , Ensure that the data of previous versions are written .