Apache apisik v2.14.1 exploratory release to expand into more fields

The last time distance APISIX v2.13 LTS The version has been released for more than two months , Every time in the past APISIX A small version of , Will bring you new functions . However APISIX v2.14.1 Release features , Will keep up with the technological frontier , It has brought us many exploratory new functions , And for APISIX v3 Version of the release of the stone , Welcome to explore these new features .

Next, let's take a look at APISIX What exploratory new features are supported .

be based on WebSocket Of pubsub Proxy framework

stay APISIX v2.14.1 Before the release , Whether it's an agent gRPC The request is normal HTTP request ,APISIX The upstream of are all connected application servers , Unable to meet the needs of diversified scenarios . For example, users need to use other upstream types （ such as Kafka）, It can only be achieved in other ways . But in APISIX v2.14.1 In the version ,APISIX New one based on Websocket Message subscription broker framework , The framework allows clients to pass through APISIX To subscribe to the specified message queue （ The upstream ） The messages in the . Now you can use APISIX Subscribe to Kafka The messages in the .

With Kafka For example , We need the following configuration ：

curl -X PUT 'http://127.0.0.1:9080/apisix/admin/routes/kafka' \    -H 'X-API-KEY: ${api-key}' \    -H 'Content-Type: application/json' \    -d '{    "uri": "/kafka",    "upstream": {        "nodes": {            "kafka-server1:9092": 1,            "kafka-server2:9092": 1,            "kafka-server3:9092": 1        },        "type": "none",        "scheme": "kafka"    }}'

The above example adds a... To the route Kafka Upstream of type , And contains multiple Broker.

You can subscribe to this upstream by referring to the following steps ：

1. First, please go through WebSocket Establishing a connection .
2. obtain Topic One of the Partition Current offset. The following example uses Protobuf To encode related requests and responses ：

message PubSubReq {    int64 sequence = 1;    oneof req {        CmdEmpty cmd_empty                       = 31;        CmdPing cmd_ping                         = 32;        CmdKafkaFetch      cmd_kafka_fetch       = 33;        CmdKafkaListOffset cmd_kafka_list_offset = 34;    };}message PubSubResp {    int64 sequence = 1;    oneof resp {        ErrorResp error_resp                       = 31;        PongResp pong_resp                         = 32;        KafkaFetchResp kafka_fetch_resp            = 33;        KafkaListOffsetResp kafka_list_offset_resp = 34;    };}

Like getting offset Our request is ：

message CmdKafkaListOffset {    string topic = 1;    int32 partition = 2;    int64 timestamp = 3;}

For the specific meanings of each field, please refer to pubsub.proto.

3. After that, each subscription operation can be based on the current offset To get the latest news .

Be careful ： After successfully getting the message , Need to update the current offset, Updated offset Is returned before offset + 1.

Please refer to the source code and test cases for specific operations ：

• https://github.com/apache/apisix/blob/master/t/pubsub/kafka.t
• https://github.com/apache/apisix/blob/master/t/lib/pubsub.lua

Although the current Pubsub The framework provides only the underlying interfaces , But it has already realized two basic requirements ：

• Through the common 80/443 Port exposure Kafka Service capability, , There is no need to encapsulate the application server .
• Allow to add authentication plug-ins , As usual Websocket Frame that way Kafka Add security to your services .

If you encounter problems in actual use , You can submit issue Give feedback to Apache APISIX Community , The community will be based on user feedback , Continue to improve and enhance this feature .

be based on xRPC Framework management is not HTTP Of 7 Layer protocol

APISIX Agents were supported in earlier versions TCP agreement , But in some scenes , Pure TCP The protocol proxy cannot meet the needs of users . Because some functions can only be realized after the application protocol is encoded and decoded , Therefore, the user needs a proxy for a specific application protocol , such as Redis Proxy、Kafka Proxy etc. .

from APISIX v2.14.1 Version start ,APISIX Provides xRPC frame , Allow developers to customize specific application protocols on the framework . be based on xRPC frame ,APISIX It can provide proxy support for several mainstream application protocols . At the same time, users can also support their own private based on the framework TCP Application protocol , Make it similar HTTP The precision granularity and higher order of the protocol proxy 7 Layer control .

at present ,APISIX Already in xRPC The framework implements Redis Agent function of , Support to inject delay according to command and selectively record log content . Even though APISIX Need to be right Redis The protocol encodes and decodes , But in simple SET/GET In the performance test , Use double Worker Process APISIX Acting as agent , Its performance can reach direct connection Redis Of 80%.

You can create a proxy by referring to the following commands Redis Protocol flow routing ：

curl http://127.0.0.1:9080/apisix/admin/stream_routes/1 \-H 'X-API-KEY: ${api-key}' -X PUT -d '{    "upstream": {        "type": "none",        "nodes": {            "127.0.0.1:6379": 1        }    },    "protocol": {        "name": "redis",        "conf": {            "faults": [{                "commands": ["get", "ping"],                "delay": 5            }]        },        logger = {            [                "name": "syslog",                "filter": [                    ["rpc_time", ">=", 1],                ],                "conf": {                    "host": "127.0.0.1",                    "port": 8125,                    "sock_type": "udp",                    "batch_max_size": 1,                    "flush_limit": 1                }            ]        }    }}'

The definition of the above configuration is as follows ：

When the command is GET or ping when , There will be a 5 Second delay . At the same time, after each command is executed , Will determine if it takes longer than 1 second , If it is , Then the corresponding logger object , And send the syslog UDP Log to 127.0.0.1 Of 8125 port .

The control surface supports service discovery

stay v2.14.1 Before the release ,APISIX Service discovery is supported only on the data side . In this case , Every APISIX All instances need to obtain service discovery data , But in the actual application process , The following questions were fed back ：

1. Every APISIX All instances need to pull data from the service discovery system , This complicates the network topology .
2. The service discovery configuration needs to be configured in each APISIX Configure all instances . To change the password , You have to modify the configuration file , Then publish to each APISIX For instance .
3. Currently, many service discovery services are not provided Lua SDK, If you want to use these services to discover systems , You need to directly connect to the... Provided by the server HTTP API （ If there is ）.

So from v2.14.1 Version start ,APISIX The service discovery function will be supported in the control plane . Service discovery will be through APISIX-Seed Realization .

This function is realized through apisix-seed Simultaneous monitoring etcd in Upstream Related resources and corresponding upstream service resources in the service discovery component , Update when the upstream service resources in the service discovery component change etcd Related to Upstream Information .

The specific implementation process is shown in the following figure ：

For now , The scheme of control surface service discovery also has shortcomings , For example, yes. etcd High pressure . therefore APISIX Two service discovery schemes will be retained at the same time , And through more practical applications to prove which scheme will be better .

Preliminary support Istio

In order to adapt to a wider range of application scenarios , from v2.14.1 Version start ,APISIX Will try to be compatible with Istio, With Istio Is the control surface 、APISIX In the form of a data plane , Start exploring in the field of service grid .

because Istio The configuration is through xDS The agreement was issued , So we developed Amesh project , hold Istio Issued xDS convert to APISIX Configuration of . at present ,APISIX Has been able to run through Istio Official Simple Bookstore App demo. In later versions ,APISIX We will also continue to broaden our understanding of xDS Support for , hold Istio and APISIX The ability to become more and more closely integrated .

More plug-ins and features

In addition to the exploratory features mentioned above , This release also provides users with some traditional functions ：

• newly added casdoor plug-in unit , Improve and Casdoor Interactive experience of .
• response-rewrite The plug-in is added for Body Replacement filter for .

More feature updates and Bug Restoration details , Please check the official Releases CHANGELOG.