This paper addresses ：https://www.ebpf.top/post/bumblebee

1. Preface

not long ago ,Solo.io The company announced open source on its official website blog with a name of BumbleBee New projects . The project focuses on simplifying the build eBPF Threshold of tools and optimized use experience , By way of eBPF The program is packaged into OCI Mirror image , Bring and use Docker Construction of a consistent experience 、 Distribute and run eBPF Program .

BumbleBee The purpose is to let us focus on writing eBPF Code , It is responsible for automatic generation and eBPF Program related user space code functions , Including loading eBPF Procedures and will eBPF The data of the program is used as a log 、 Indicators and histograms are displayed .

So why do we need to BumbleBee Project to manage eBPF The procedure ？ This needs to be done from eBPF Talking about the characteristics of Technology .

2. Build and distribute eBPF The challenge of tools

eBPF Technology is called near 50 The biggest change in the operating system in the past years , It's solved Linux The kernel is developed upstream 、 The dilemma of slow merger and distribution functions . eBPF Technology provides the kernel with the ability to customize the kernel without upstream implementation , Currently, it is already observable 、 Network and security have been widely used in many fields , Especially in the cloud native technology trend ,eBPF The ability of technology to play is also becoming more and more important , Such as those in the current high tide Cilium project .

But development 、 Build and distribute eBPF It has always been a high threshold job , The community has launched BCC、BPFTrace Wait for the front-end binding , Greatly reduce the writing and use eBPF The threshold of Technology , But the source code delivery of these tools , Need to run eBPF programmatic Linux Install the supporting compilation environment on the system , It brings a lot of trouble for distribution , At the same time, the problem of kernel adaptation can only be verified at runtime , It is also not conducive to finding and solving problems in advance .

In recent years , To solve problems in different kernel versions eBPF The distribution and operation of the program , The community is based on BTF Technology has launched CO-RE function （“ A compilation , Run anywhere ”）, To a certain extent, through eBPF Binary bytecode distribution , At the same time, it also solves the problem of porting running on different kernels , But how to package and distribute eBPF There is no unified and concise way for binary code . except eBPF Program , At present, we also need to write for loading eBPF Program and for reading eBPF Various codes of data generated by programs , This often involves copying and pasting the source code to solve some problems .

Besides ,libbpf-bootstrap adopt bpftool The tool generates the relevant scaffold code , To some extent, it solves the problem of common code duplication , But for building 、 Distribute and run eBPF The help provided on the program is limited .

3. BumbleBee brief introduction

BumbleBee The project is exactly Solo The company in Enterprise service grid Gloo-Mesh For the convenience of application in the project eBPF Technology is born , It is used to solve the problem of building 、 Distribute and run eBPF Repetitive challenges encountered by the program ,

At present, the project is still in its early stage （ current version 0.0.9）, Provided function scenarios （Network and FileSystem） Co., LTD. , But build on specific template capabilities OCI The idea of mirror image , For us in management eBPF The program provides an efficient and concise implementation , It's worth our attention .

Use BumbleBee Pre dependency of tools ： Running eBPF Your operating system is turned on BTF Support , Compiling eBPF The code also needs to use CO-RE Correlation function , About CO-RE Relevant technologies can be referred to here .

BumbleBee Provided with Docker Consistent experience feeling . The picture below is Docker High level schematic diagram of ,BumbleBee The tool completely refers to this process .

3.1 structure

BumbleBee make " Just right " Of eBPF Tool chain , take eBPF The construction process of the program is automated , Let you focus on the code itself . BumbleBee Of eBPF The code is packaged into a OCI Standard image , This allows distribution in the infrastructure .

The following command can realize eBPF Program probe.c Directly compile and package into image my_probe:v1 .

$ bee build probe.c username/my_probe:v1

3.2 Release

utilize BTF and OCI Packing capacity ,BumbleBee Compiling eBPF The code is portable , And it can be embedded into the existing publishing workflow . By way of eBPF Image of code construction , Push to any match OCI Standard image warehouse , You can publish it to other users .

The following commands realize the function of publishing images to the image warehouse , It can be used directly bee run Run based on image .

#  push 

$ bee push username/my_probe:v1 

#  Pull 

$ bee pull username/my_probe:v1

3.3 function

Use BumbleBee Provided CLI Interface and the image stored in the image warehouse , We can quickly run in other places . BumbleBee Not only built user space code , And you can use eBPF map, To show the log 、 Indicators and histogram information . BumbleBee Used BTF Format self-examination ability , Know which data types need to be displayed .

$ bee run my_probe:v1

Let's pass a complete example , To experience BumbleBee Bring us management eBPF Procedural convenience .

4. Complete experience

4.1 bee install

First, we need a running support BTF Kernel Linux operating system , It is recommended to use ubuntu 2110 edition , The installed kernel has been supported by default BTF. If you choose to use Vagrant To manage virtual machines ,BumbleBee Provided in the warehouse Vagrantfile Files can be used directly . Or you can use mulipass The tool directly launches a ubuntu 2110 Version of the system .

Here, use the script provided by the warehouse to install , Of course, you can also go directly through git clone The way of warehouse .

For a quick experience , Avoid permission problems in some scenarios , It is recommended to use root User to install .

[email protected]:~# curl -sL https://run.solo.io/bee/install | BUMBLEBEE_VERSION=v0.0.9 sh

Attempting to download bee version v0.0.9

Downloading bee-linux-amd64...

Download complete!, validating checksum...

Checksum valid.

bee was successfully installed 

Add the bumblebee CLI to your path with:

  export PATH=$HOME/.bumblebee/bin:$PATH

Now run:

  bee init     # Initialize simple eBPF program to run with bee

Please see visit the bumblebee website for more info:  https://github.com/solo-io/bumblebee

After installation ,bee The main commands of are as follows ：

# bee --help

Usage:

  bee [command]

Available Commands:

  build       Build a BPF program, and save it to an OCI image representation.

  completion  generate the autocompletion script for the specified shell

  describe    Describe a BPF program via it's OCI ref

  help        Help about any command

  init        Initialize a sample BPF program

  list

  login       Log in so you can push images to the remote server.

  pull

  push

  run         Run a BPF program file or OCI image.

  tag

  version

Flags:

  -c, --config stringArray   path to auth configs

      --config-dir string    Directory to bumblebee configuration (default "/root/.bumblebee")

  -h, --help                 help for bee

      --insecure             allow connections to SSL registry without certs

  -p, --password string      registry password

      --plain-http           use plain http and not https

      --storage string       Directory to store OCI images locally (default "/root/.bumblebee/store")

  -u, --username string      registry username

  -v, --verbose              verbose output

Use "bee [command] --help" for more information about a command.

4.2 Bee init Generate eBPF Procedural scaffolding

Bee init Commands can be generated through the problem Wizard mode eBPF Code scaffolding , Function and libbpf-bootstrap Some similar , But it's easier to use the wizard .

$ export PATH=$HOME/.bumblebee/bin:$PATH

# ebpf-test && cd ebpf-test

# bee init

Use the arrow keys to navigate: ↓ ↑ → ←

? What language do you wish to use for the filter: #  step   Choose to write  eBPF  The language of the code 

  ▸ C   #  Currently only supported  C,Rust  May support in the future 

---------------------------------------------  #  step  2  choice  eBPF  Program type 

 INFO  Selected Language: C

Use the arrow keys to navigate: ↓ ↑ → ←

? What type of program to initialize:

  ▸ Network        #  Choose to write  eBPF  The type of program , The current support  Network  and  File System

    File system    #  The generated templates correspond to  tcp_connet  and  open  function 

--------------------------------------------- #  step  3   choice  map  type 

 INFO  Selected Language: C

 INFO  Selected Program Type: Network

Use the arrow keys to navigate: ↓ ↑ → ←

? What type of map should we initialize:

  ▸ RingBuffer

    HashMap

 --------------------------------------------- #  step  4  choice  map  Export type 

 INFO  Selected Language: C

 INFO  Selected Program Type: Network

 INFO  Selected Map Type: HashMap

Use the arrow keys to navigate: ↓ ↑ → ←

? What type of output would you like from your map:

  ▸ print    # map  Presentation of data , Log printing 、 Count or index Export 

    counter

    gauge

 ---------------------------------------------  #  step  5 eBPF  Program save file name 

 INFO  Selected Language: C

 INFO  Selected Program Type: Network

 INFO  Selected Map Type: HashMap

 INFO  Selected Output Type: print

 BPF Program File Location: probe.c

 ---------------------------------------------- #  Finally, complete the whole code generation wizard 

 INFO  Selected Language: C

 INFO  Selected Program Type: Network

 INFO  Selected Map Type: HashMap

 INFO  Selected Output Type: print

 INFO  Selected Output Type: BPF Program File Location probe.c

 SUCCESS  Successfully wrote skeleton BPF program  

# ls -hl

total 4.0K

-rw-rw-r-- 1 ubuntu ubuntu 2.0K Feb 11 11:33 probe.c

adopt init Command generated probe.c The file format is generally as follows ：

#include "vmlinux.h"

#include "bpf/bpf_helpers.h"

#include "bpf/bpf_core_read.h"

#include "bpf/bpf_tracing.h"

#include "solo_types.h"

// 1. Change the license if necessary

char __license[] SEC("license") = "Dual MIT/GPL";

struct event_t {

	// 2. Add ringbuf struct data here.

} __attribute__((packed));

// This is the definition for the global map which both our

// bpf program and user space program can access.

// More info and map types can be found here: https://www.man7.org/linux/man-pages/man2/bpf.2.html

struct {

	__uint(max_entries, 1 << 24);

	__uint(type, BPF_MAP_TYPE_RINGBUF);

	__type(value, struct event_t);

} events SEC(".maps.print");

SEC("kprobe/tcp_v4_connect")

int BPF_KPROBE(tcp_v4_connect, struct sock *sk)

{

	// Init event pointer

	struct event_t *event;

	// Reserve a spot in the ringbuffer for our event

	event = bpf_ringbuf_reserve(&events, sizeof(struct event_t), 0);

	if (!event) {

		return 0;

	}

	// 3. set data for our event,

	// For example:

	// event->pid = bpf_get_current_pid_tgid();

	bpf_ringbuf_submit(event, 0);

	return 0;

}

Based on the generated code template , We need to fill in our own logic , This is not the point , Skip the relevant code first , The complete code can be found in the official Start documentation View in .

4.3 structure eBPF Program

The build process requires Docker Or type Docker The container engine , Need to be installed in advance .

# apt  install docker.io  #  install  docker

# bee build probe.c my_probe:v1

 SUCCESS  Successfully compiled "probe.c" and wrote it to "probe.o"

 SUCCESS  Saved BPF OCI image to my_probe:v1

We don't need to involve in the whole construction process clang And other related compilation commands , Just go through bee build Command input eBPF The name of the program file and the image you want to generate , After compilation ,eBPF Binary bytecode of program probe.o Will be automatically added to the image my_probe:v1 in , We can use bee tag Complete the redefinition of the image warehouse .

4.4 Release eBPF Program

We can go through bee tag and push The subcommand completes the release of the image warehouse .

# bee tag my_probe:v1 dwh0403/my_probe:v1

# bee login

# bee push dwh0403/my_probe:v1

Take a look at the above commands , Is there a feeling of deja vu ？

4.5 function eBPF Program

After building the image , You can directly go through bee run To run the , After operation bee It will start automatically TUI Interface , To show how we write eBPF In program map Content , Automatically generated map Names have some special suffixes for bee TUI User space program to read the corresponding map Display the data in , For example, in the generated code template SEC(".maps.print"), It means that we should map For printing .

# bee run my_probe:v1

 SUCCESS  Fetching program from registry: my_probe:v1

 SUCCESS  Loading BPF program and maps into Kernel

 SUCCESS  Linking BPF functions to associated probe/tracepoint

 INFO  Rendering TUI..

5. summary

thus , We have completed the experience of the whole project function ,bee init The tool can help us generate eBPF The code framework , Although the function is still a little thin , But it is a fast and convenient way to use our specific scenes .

bee build/push/run Wait for orders , Commands that will be compiled 、 Packaging mirroring 、 The steps of publishing and running images have been greatly simplified , Very easy to use , Greatly reduces the build 、 Release and run eBPF The repetition cost of the program , I have to praise the author's ideas .

Because through bee The generated tools are based on specific scenarios , Functional richness is still limited , For writing eBPF Programs and user space programs with rich functions are not applicable , But its construction 、 The overall idea of release and operation （ Even some basic functions ） But we can use it directly or use it for reference .

6. Reference material

BumbleBee: Silky smooth construction 、 Deliver and run eBPF More articles on the program

  1. Large scale promotion and concentration ,CDN How to ensure the smooth e-commerce experience ？

    brief introduction : Not long ago , Alibaba cloud technology group airborne CSDN Online Summit , Interpretation of the core technology competitiveness . among , Zeng Fuhua, senior technology expert of Alibaba cloud, shared < double 11: CDN How to ensure the smooth promotion of e-commerce > Issues . It is said that : A thousand days of military training , use ...

  2. How to make Caffeine Cache Smooth as silk ？

    One . About Caffeine Cache In the recommendation service , Although a small number of requests are allowed to return to the default list due to calculation timeout and other reasons . But in terms of operational indicators , Higher " Completion rate " It means that the more complete the algorithm effect is , It also means higher business ...

  3. Smooth as silk from Windows transfer SQLServer Database to Linux

    I read an old rookie article <MSSQL On Linux Backup and restore > After the article , Very satisfied , But I can't help asking :" This article is about MSSQL stay Linux Backup and restore on the system , If I had been Windo ...

  4. Vessel schedule • Docker The Internet • Continuous delivery • Run applications dynamically Diversification of deployment

    < Hero alliance > Online service operation and maintenance - InfoQ https://www.infoq.cn/article/running-online-services-riot/ Chapter one Jane Medium I am a Jo ...

  5. Microsoft Build The conference released a large number of development tools and services ！ code 、 Collaboration 、 Release , Silky smooth

    Microsoft Build 2020 The developer conference has come to a successful conclusion , For two days in a row 48 Hours of live broadcast , Developers from all over the world go to a feast , The scene is quite spectacular . In this annual party , Microsoft is also full of sincerity , It's a chain of products ...

  6. I want Android to APP Silky smooth ？

    With the increasing market share of Android phones , Just look for a few people on the street 80% It's all Android phones ! So many people use Android phones , I don't know if you've ever felt APP Carton . crash ? Whether the application is not responding . Flash back ? This article is for you ...

  7. ios Slide smoothly （ As smooth as silk ） rolling

    stay ios html->body->list( You can't do without one style !) html->body->list <!DOCTYPE html> <html lang=&q ...

  8. 【AMAD】django-silk -- by Django Provides a silky smooth performance measurement

    motivation brief introduction Personal rating motivation Django As a web frame , It's very complicated to make performance measurements , You can't use traditional programs profile Tools . because ,web app Our performance is multidimensional , It's not just code execution efficiency , It also includes network latency ...

  9. Give Way API The unique skill of making parallel calls as smooth as silk

    When the amount of data is large , Will be split by database and table , Share the pressure of reading and writing . After dividing database and table, the problem of query is more troublesome , If you don't query directly according to the partition key , You need to query multiple tables . In some complex business scenarios , Like order search , In addition to orders ...

  10. 【 Programming learning notes 】 How to organize and build multiple files C Language program ！ Programming also has ~

    good Unix Procedural Philosophy First , You need to know this C Program is a  Unix  Command line tools . That means it runs on ( Or it can be ported to ) Those offer Unix C In the operating system of the running environment . When Bell Labs invented Unix after , It is from ...

Random recommendation

  1. install windows Under the Cscope

    http://blog.csdn.net/maxiee/article/details/10034263 Cscope Is a software used to view code in large projects . It's easy to use , Support quick search C Symbol ...

  2. turn ：eclipse as well as step into step over step return The difference between

    First of all, let's talk about step into step over step return The difference between : step into It's a single step , If you encounter a subfunction, enter and continue to step in :(F5) step over It's in a single step , In function ...

  3. Delphi Project documents of project components DPR

    1 2 3 4 5 6 7 8 9 10 11 12 13 program Project1;                         { keyword program, The standard Pascal Source file format } ...

  4. android From entry to proficiency ( 5、 ... and )--- radio broadcast

    1. The radio type : Standard and orderly broadcasting . Standard broadcast is asynchronous broadcast. After the broadcast is sent, all receivers will receive it almost at the same time , There is no order , Efficient , But it can't be truncated . Orderly broadcasting is synchronous broadcasting , Only one receiver can receive this message at the same time , After the execution ...

  5. Two domestic popular open source .net WeChat public platform SDK comparative analysis

    Recently, I am busy with the development of wechat surrounding It's hard to avoid itching. Search for relevant sdk Use it directly to make I really found a lot Here are two things that look good .net Based on C# Language development SDK One is strong and the other is small (1) Senparc.W ...

  6. C51 Detailed explanation of the relationship between pointer type and storage area

    One . Relationship between storage type and storage area data     --->    Addressable on-chip ram bdata   --->     Bit addressable on-chip ram idata    --->    Addressable on-chip ram ...

  7. scala Learning notes —— The operator

    Middle operator ( Binary operators ), The operator lies between two parameters . Operators include letters , such as to, It can also include operator characters , such as 1->10, Equivalent to method call 1.->(10) a identifier b The identifier is one with two ...

  8. php The zoom gif and png The solution of format transparent background turning black

    In the face of gif or png When zooming and other operations are carried out for pictures in format , The pictures with transparent background finally turn black , terms of settlement $img = imagecreatetruecolor(, ); //2. Color $color=imageco ...

  9. haproxy- Code reading - memory management

    haproxy Memory pool Overview Memory pools are classified by type , Each type of memory pool has a name , Use a linked list to record free memory blocks , Each memory block is equal in size , And in accordance with the 16 Byte alignment . haporxy use pool_head Structure record memory pool str ...

  10. Qt Programming Guide 10 QImage Mat QPixmap transformation

      // Example pushButtonOpenPicBig Button clicked Click action trigger void Qt_Window::on_pushButtonOpenPicBig_clicked() { strin ...