Technology selection rust post analysis

introduction

Recently used Rust Wrote a consumer enterprise GitHub Comment on the microservice of events . This article records your choice Rust Write instead of Python Some thoughts on , Take a little time .

What microservices need to do is very simple ： Expose a receiver post Requested http Interface . Every time I receive payload When , If it's news that interests me , Then extract the corresponding information , Send to another server .

I anticipate using Rust Will need 4 God （ Effective working hours per day 4 Hours ,4*4=16 Hours ）, And if you use Python Will need 2 God .

It actually cost 2 God , The approximate time distribution is as follows ,

1. 1.5 Days to complete Rust Code and unit tests
2. 0.5 Days to kubenetes and GitHub It's all through .

The architecture of the completed microservice is as follows ,

Use Rust Points that require additional thinking

This is for me Rust Prepared by 2.5 A project , The first is  https:// github.com/Celthi/symbo l-service

Use Rust In the process of writing code , Encountered some possibilities Rust Unique problems that require a little time

Global variable

web When the module receives the message , It should be stored in the message queue . and web A module is a function that receives specific parameters , After it extracts the message , How to access the message queue ？

Message queues cannot be passed in from parameters , Then you can only use global variables .

Global variables use lazy_static This library . But now create a variable channel, Similar to the following code

let (tx, rx) = mpsc::channel();

    thread::spawn(move || {
        let val = String::from("hi");
        tx.send(val).unwrap();
    });

    let received = rx.recv().unwrap();
    println!("Got: {}", received);
}

I thought channel The sender and receiver must be variable , Because sending a message must change channel Queues . So we need variable variables that can be shared by multiple threads globally . Shared multithreading because web Module and processing module are not in the same thread .

Shared variable , It's time to Mutex, Then I spent time studying how to do , The final generated code is as follows ,（ It can be used , But it's a little ugly

lazy_static! {
    static ref CHANNEL: (
        Arc<Mutex<Option<Sender<task::Task>>>>,
        Arc<Mutex<Option<Receiver<toil_task::ToilTask>>>>
    ) = (Arc::new(Mutex::new(None)), Arc::new(Mutex::new(None)));
    static ref CONFIG: config_env::ConfigEnv = (|| {
        match config_env::ConfigEnv::new() {
            Ok(c) => c,
            Err(e) => {
                eprintln!("{}", e);
                process::exit(1);
            }
        }
    })();
}

Here we also put the environment variables into the global variables , Convenient for all places . And also use closed packets to read environment variables . Why use closures , Because I want to deal with the situation that the environment variable does not exist , immediate withdrawal .

As for why Option？ That's because I can't find it in lazy_static How to create channel. Now it seems that you can create .

The solution above , Ugly but it works . I can already accept rc<Mutex<Option<Sender<task::Task>>>>, After all, there are few places to use , You can wrap it up a little bit .

Asynchronous processing

From the above architecture diagram , We can go down one way , Process an event message synchronously . But I chose to create two asynchronous event loops to handle (event loop)

first event loop yes web The event loop in the framework . It is responsible for obtaining http request , Put it in the message queue .

the second event loop It's the event cycle of handling news . It is responsible for taking messages out of the message queue , Then process .

I took poem web frame . This is because I am in a Rust Wechat group of , There is... In the group poem The author and often hear poem The discussion of the .

The second one that takes a little time here event loop, I was in the beginning poem Of main Start a thread in the function , Used in threads loop To cycle through getting messages from the message queue .

/// !!!! Error code 
#[tokio::main]
async fn poem_main() -> Result<(), std::io::Error> {
    if std::env::var_os("RUST_LOG").is_none() {
        std::env::set_var("RUST_LOG", "poem=debug");
    }
    tracing_subscriber::fmt::init();
    std::thread::spawn(|| {
       loop {
         tokio::spawn(async  {
           println!("");
         });
       }
    });
    let app = Route::new().at("/webhook", post(hello)).with(Tracing);
    Server::new(TcpListener::bind("0.0.0.0:31440"))
        .run(app)
        .await
}

But I encountered the following error report

thread '<unnamed>' panicked at 'there is no reactor running, must be called from the context of a Tokio 1.x runtime', src/bin/ocr_agent.rs:185:8

therefore Google Once again , Find out （ Actually, I didn't find ） I haven't found any useful news in a short time , therefore ,

Change the code as follows ,

#[tokio::main]
async fn web_main() -> Result<(), std::io::Error> {
    if std::env::var_os("RUST_LOG").is_none() {
        std::env::set_var("RUST_LOG", "poem=debug");
    }
    tracing_subscriber::fmt::init();
    tokio::spawn(async  {
       loop {
          tokio::spawn(async  {
             println!("");
          });
       }
    });
    let app = Route::new().at("/ocr_webhook", post(hello)).with(Tracing);
    Server::new(TcpListener::bind("0.0.0.0:31430"))
        .run(app)
        .await
}

It's working , But after a short test , It is strange to find such a program bug： The first message will never be consumed , And the news coming later is normal .

After a short search , I have seen some problems encountered by some people on the Internet , But they are different from me . Because my goal is to write this micro service in two days , So I skipped this question at that time . As of now, I haven't confirmed the reason .

So I came back to the first question , By reading tokio Of runtime That part of the document , I found that I could use two tokio runtime instance. Isn't this the design I want ？

So the second event loop , I created a new one directly tokio runtime instance. So the correct and desired code is as follows ,

fn main() {
    ensure_config();
    init_channels();
    let mut v = vec![];
    let j = thread::spawn(|| {
        poem_main();
    });
    v.push(j);
    let j = thread::spawn(|| {
        message_main();
    });
    v.push(j);
    for t in v {
        t.join().unwrap();
    }
}

This problem has been solved .

Others also encountered some problems , But those are basically search , Just look at the sample code , I won't repeat it here .

Estimate from zero

If you estimate from zero, use Rust How much time will it take , Then here is my estimate . Be careful ： Unit day refers to 4 More than hours of investment .

• Will use Poem web frame 1 God
• Will use Tokio Asynchronous framework 3 God
• Will use lazy_static1 God
• journal x God
• Business processing time n God

The above is based on the meeting Rust On the basis of . And learn Rust The investment varies from person to person , Don't think about it . therefore , Use rust It takes time to build a microservice from scratch

5 God +n God +x God .n Days is the time needed for business processing , Varies by business .x I don't know how many days it takes

The business of the micro service I wrote is to process messages , Send it to another server . It's very simple , It only needs 1 God . add poem,tokio I've been able to output all the logs to stdout and stderr（ because kubenetes The default is to save 10M The content of ） So it takes a total of time 2 God .

contrast Python

use Rust I spent two days writing this micro service , If using Python, Then remove Rust The only problem , Writing this micro service only requires 1 God .

But I still choose to use Rust To write the , as a result of ,

• From the perspective of real use ,Rust The experience of writing code is better than Python Not much less . In addition to occasionally correcting simple compilation errors , For example, forget to add mut, Used move Value , Forget to add references, etc .
• Rust Force the handling of situations with errors . Someone will say , Isn't it annoying to force errors , Shouldn't you do the function well first in rapid development ？ actually , According to my years of programming experience , Spend more time in the early stage , Better than troubleshooting later . It's not about using Rust There is no need to troubleshoot later . It's better to troubleshoot . Because you have handled the wrong situation , Then the failure rate is probably in the place you know . If you use Python, Then the common method is to put exception catch live , Then print the call stack , Then it's like nothing happened , Continue operation . Continue to run in order not to interrupt the service , Oneortwo requests failed , It doesn't matter. .
• Python Handle json Very convenient . because github The messages sent are json Format , So just extract the part of interest directly . however Rust Yes serde_json, Handle json It's very convenient , It's not as troublesome as expected . The extra step is to define the structure , Call the parse function , Finally, the interested part can be extracted .
• Rust It's a strong type , But type derivation is more powerful , Many times you don't need to write types . however IDE Tips make it easy to write code .Python 3.10 There are also types ,IDE The prompt is also ok .

Practice has proved Rust Writing is also very convenient , Now we are dealing with companies GitHub 7+ Code base events .

The resource occupation of microservices is top -p <pid>