How do gateways and chirpstacks in lorawan communicate? UDP? GRPC? MQTT?

One , Preface

Doing standard LoRaWAN In the project , Gateway plays a connecting role , stay MCU Gateway program running on , It is necessary to receive data from nodes , You also need to package and send the node data to the server （chirpstack）.
Today's blog mainly talks about the communication between gateway and server ？
Sounds , The connection between gateway and server is not standard socket Process , Create two file descriptors, one for uplink , One is responsible for the downlink , Just read and write inside ？ Actually not. , There is a lot of knowledge in it !

Two , Several knowledge points

2.1,chirpstack form

2.1.1,ChirpStack Gateway Bridge

Located in packet forwarding and MQTT Between agents . It formats packets into Forwarders （ for example Semtech UDP Packet forwarder Protocol ） Convert to ChirpStack The data format used by the component （GRPC）. It also provides integration with various cloud platforms , for example GCP Cloud IoT Core and Azure IoT Hub.

2.1.2,ChirpStack Network server （NS）

Responsible for managing the status of the network . It has the knowledge of device activation on the network , And it can handle the join request when the device wants to join the network .
When multiple gateways receive data ,ChirpStack The network server will deduplicate these data , And forward it as a payload to ChirpStack application server . When the application server needs to send data back to the device ,ChirpStack The network server will keep these items in the queue , Until it can be sent to one of the gateways .

2.1.3,ChirpStack application server （AS）

And ChirpStack Network server compatible . It provides for managing users , organization , Applications , Gateway and device Web Interface and API. The received uplink data will be forwarded to one or more configured integrators .

2.1.4,Redis

By the Italians Salvatore Sanfilippo（ Net name ：antirez） Developed a memory cache database .Redis Its full name is ：Remote Dictionary Server（ Remote data services ）, The software uses C Language writing ,Redis It's a key-value The storage system , It supports rich data types , Such as ：string、list、set、zset(sorted set)、hash.ChirpStack The application server stores all non persistent data in Redis In data storage .

2.1.5,PostgreSQL

It is an object of free software with very complete features - Relational database management system （ORDBMS）, It was developed by the computer department at the University of California POSTGRES,4.2 Version based object relational database management system .ChirpStack The application server saves the gateway data to PostgreSQL In the database .

2.1.6 ,mosquitto

ChirpStack Application servers utilize MQTT To publish and receive application payloads .Mosquitto It's popular open source MQTT The server .

2.2,GRPC

2.2.1,GRPC What is it ？

yes Goolge The company develops and works on the communication protocol of the application layer ,RPC It refers to the local service calling the program or service on the remote server , Is a client server （C/S） Communication mode .GRPC By Google Develop and open source . Through it , A client consumer service can be like calling a local method , Call the server method on another host .GRPC be relative to RPC, The implementation uses HTTP2.0, Protocol buffer （protobuf） Technology .

2.2.2,GRPC The advantages of ？

1） In the design of micro Services , We usually use HTTP REST（ Design style and development method of network application ） To transmit data , And serialize and deserialize between the two services ; And the traditional RESTful Interface to use JSON、XML Or some other formats as data carriers , Make the service run slowly , High memory usage 、 And the transmission process is not compressed ;grpc What runs inside is protobuf Format , Rely on the client and server to establish socket The connection goes on .

2） because HTTP Complete... In the application layer , The cost of the whole communication is high , Remote procedure calls are directly based on TCP Make a remote call , Data transmission is at the transport layer TCP Layer complete , It is more suitable for scenes requiring high efficiency ,RPC It mainly depends on the establishment between the client and the server Socket Link to , Bottom implementation ratio REST More complicated ;

3） Compared with the regular RPC,GRPC Have better performance ;

2.3,protobuf

2.3.1,protobuf What is it ？

Protobuf The full name is Google Protocol Buffer, It is an efficient and lightweight structured data storage method , Can be used for （ data ） Communication protocol 、 Data storage, etc . It can also be understood as Serialization method of structured data , It provides an efficient serialization and deserialization mechanism , Serialization is to convert an object into binary data and send it to the server , Deserialization is to convert the received binary data into the corresponding object .

Serialization transforms a data structure or object into something that can be stored and transferred （ For example, network transmission ） The format of , At the same time, we should ensure that the serialization result is later （ Maybe in another computing environment ） Can be reconstructed back to the original data structure or object .

2.3.2,protobuf The advantages of

1） It has nothing to do with language 、 It's not about the platform . namely ProtoBuf Support Java、C++、Python multilingual , Support multiple platforms
2） Efficient . I.e. ratio XML smaller （3 ~ 10 times ）、 faster （20 ~ 100 times ）、 It's simpler
3） Extensibility 、 Compatibility is good. . You can update the data structure , Without affecting and destroying the original old program

json It's a text file , Text files are character based .
protobuf It's binary code , Value based .

3、 ... and , Gateway and chirpstack signal communication

3.1, Don't consider MQTT

According to the program block diagram, we can see ,chirpstack_bridge be located NS Before , The function is to convert all data packets from the gateway into suitable chirpstack Inside protobuf Format , However, it is obvious that , We can also implement this function in the gateway program , That is, the gateway sends the received RF Package assembly protobuf Reissue , It means to implement in the gateway program chirpstack_bridge This function , Then this data can be passed directly GRPC Transferred to the chirpstack Of NS Without going through chirpstack_bridge！

The following is my own flow chart ：

3.2, consider MQTT

Still look at the flow chart ,MQTT Of broker be located NS and chirpstack_bridge Between , So we realize mqtt There will be two ways ：

The first one is ：
Gateway implementation chirpstack_bridge function （ Encapsulate data into protobuf） in the future , Put the data pub To chirpstack Of broker, Then carry out the following operations , This implementation is relatively easy .

The second kind :
Gateway not implemented chirpstack_bridge function , adopt udp To chirpstack_bridge, then chirpstack_bridge Again pub To chirpstack Of broker, Then carry out the following operations , This kind of implementation needs to re package the data packets on the gateway side and then pass udp send out , And in chirpstack Internal and multiple mqtt The operation of , More difficult , But the division of labor of each part is clear , There can be interaction between various parts , Good scalability .

The following is my own flow chart ：