Redis learning - 05 node JS client operation redis and pipeline pipeline

Use the programming language client operation Redis

At present, all our operations are carried out through Redis Command line client for redis-cli On going .

Developers can also use Redis Graphics management software operation , for example RDM（Redis Desktop Manager）（ charge , Try it for free 14 God ）.

You can also use Redis The program clients supported by each language listed on the official website ：https://redis.io/clients, Among them, the stars represent the meaning of recommendation .

With Node.js Developed Redis Client as an example , The following two are recommended ：

• node-redis： Born early （2010 year ）, Quite stable , Very good performance
• ioredis： Later came （2015 year ）, Function and node-redis almost , Alibaba is using

This article takes ioredis As an example to introduce how to use the program client operation Reids.

ioredis The official introduction

ioredis It's a powerful Redis client , Alibaba, the world's largest online commerce company, and many other excellent companies have used .

• The function is all ready . Support clusters , sentry , flow , Assembly line , Of course, there's support Lua Scripting and Publishing / subscribe （ Support with binary messages ）.
• High performance
• Pleasant API. It's asynchronous API Support callback functions and Promise
• Conversion of command parameters and return values
• Transparent key prefix
• Lua Script abstraction , Allows you to define custom commands .
• Binary data support
• Support TLS
• Support offline queue and readiness check
• Support ES6 type , for example Map and Set
• Support GEO command （Redis 3.2 unstable ）
• Complex error handling strategies
• Support NAT mapping
• Support autopeling Automatic pipeline function

Related links ：

• Github
• API file

Quick start

install

#  Initialize project 
mkdir ioredis-demo
cd ioredis-demo
npm init -y

#  install  ioredis
npm install ioredis

Basic usage

Create an execution file in the project directory index.js：

const Redis = require('ioredis')

// 1.  Establishing a connection 

//  establish  Redis  example 
//  Default local connection  127.0.0.1:6379
const redis = new Redis()

// 2.  operation  Redis  database 

//  Callback function mode 
redis.set('foo', '1', (err, ret) => {
    
  if (err) {
    
    return console.log(' Write failure ')
    // return console.log(' Write failure ', err)
  }
  console.log(' Write successfully ')
  // console.log(' Write successfully ', ret)
})

// Promise  The way 
redis
  .get('foo')
  .then(ret => {
    
    console.log(' To be successful ', ret)
  })
  .catch(err => {
    
    console.log(' Acquisition failure ')
    // return console.log(' Acquisition failure ', err)
  })

// async/await  The way 
async function main() {
    
  try {
    
    const ret = await redis.get('foo')
    console.log(ret)
  } catch (err) {
    
    console.log(' Acquisition failure ')
  }
}

main()

The official sample ：https://github.com/luin/ioredis/tree/master/examples

Remote connection

Modify the configuration

External host connection Redis The instance needs to be limited by two configuration parameters ：

• bind： Specify the network card address that this computer can accept connections , Default 127.0.0.1 and ::1.
• protected-mode： Protected mode ,Redis 3.2 New features added after , Default on .

Be careful ： because bind The specified network card address is acceptable to this computer , Instead of binding an external host that allows connections IP.

bind By default, only local... Is bound IP 127.0.0.1, So the external host cannot connect ,Linux Can pass ifconfig Command to view all network card addresses ：

Take my server for example , This can be configured to allow external hosts to access ：

#  Prefix of each address  `-`, Indicates if the address is not available ,redis  Will not fail to start 
bind 127.0.0.1 -::1 -172.26.26.38

#  You can also listen for all addresses 
bind * -::*

#  The following configuration is the same as listening to all addresses 
# 1.  notes  bind
# bind 127.0.0.1 -::1

# 2.  To configure  0.0.0.0
bind 0.0.0.0

however , Just configure bind There is no guarantee that the external host can connect .

When protection mode （protected-mode） On , If any of the following conditions are satisfied ：

1. The server is not in use bind The instruction shows that it is bound to a set of IP Address
2. Password not configured

The server will still reject external host connections Redis.

therefore Redis The database must have no password set （ Default ） Through external connections , Two configurations need to be modified ：

1. To configure bind, Allow external host connections
2. Turn off protection mode protected-mode no

In addition, check whether the server firewall is open Redis The port number occupied by the service , For example, Alibaba cloud needs to configure security groups separately .

Be careful ： To protect data security , Open remote connections require caution .

restart Redis

To make the configuration effective , Need to stop Redis Service and specify the configuration file to restart ：

redis-cli shutdown
redis-server < Profile path >

Create examples

const Redis = require('ioredis')

//  Remote connection ： Specify the address and port 
const redis = new Redis({
    
  port: 6379, // Redis  Database port 
  host: 'xxx.xxx.xxx.xxx' // Redis  Database address 
})

Safety precautions

By default Redis The server doesn't have a password , If the server passes root User on , And the server has opened the access restriction of the external network of the port , Allow an attacker to remotely log on to Redis in , So vulnerable to malicious attacks , for example ：

• adopt Redis The built-in command writes its own public key to the server , Then you can log in without secret
• Plant malicious scripts in the server , Set suspicious scheduled tasks , For example, mining

for example ：

Therefore, it is recommended to Redis Services create individual users , Start with this user Redis The server , And prohibit remote login , Set up Redis Password etc. .

Redis Pipeling（ Assembly line ）

Official documents ：Using pipelining to speedup Redis queries – Redis

Request/Response The protocol and RTT

Redis It's a TCP The server , Use client-server Models and so-called request/response agreement .

This means that the request is usually completed through the following steps ：

• client （ Such as redis-cli） Send one to the server query, And from socket Read the server response , Usually in a blocking way
• The server processes the command and sends the response back to the client

The client and server are connected through the network , Packets are transferred from the client to the server , It takes time to return from the server to the client to answer , This Round trip time be called RTT（Round Trip Time）.

When the client needs to execute multiple requests at the same time （ for example , Add multiple elements to the same list , Or use multiple key Fill the database ）, It's easy to see how this affects performance . Before the client sends a new command , Always wait for the reply of the previous command .

for example , If RTT Time is 250 millisecond （ When the network speed is very slow ）, Even if the server can handle 100k A request , We can also process at most per second 4 Client requests . Even if the RTT Very short , Handling a large number of write operations is also a big problem .

Fortunately, there is a way to improve this problem ：Redis Pipeling

Redis Pipeling

Achieve one request/response The server , Even if the client has not read the old response , Can also handle new requests .

In this way , You can send multiple commands to the server without waiting for a reply at all , The server will be forced to use memory to queue replies , Finally, read all replies in the next step . In order to reduce RTT Round trip time , And greatly improve Redis The number of operations per second that can be performed in the server .

This is known as pipeling（ Assembly line ）, Is a technology that has been widely used for decades .

pipeline It looks like a business , But it's just Redis Provide an improvement request/response The function of efficiency , It's not atomic , There is no guarantee .

ioredis Pipeline

If you want to send a batch of commands （ for example > 5）, have access to pipeline Queue commands in memory , Then send them to... At one time Redis. such , Performance improved 50％〜300％.

redis.pipeline() Create a Pipeline example . You can look like Redis Instance to call any Redis command . These commands are queued in memory , And by calling exec Method to Redis.

Example

Batch operations are sent to at one time Redis：

const Redis = require('ioredis')

const redis = new Redis()

async function main() {
    
  try {
    
    //  establish  Pipeline  example 
    const pipeline = redis.pipeline()
    //  Batch add data 
    for (let i = 0; i < 100; i++) {
    
      pipeline.set(`${
      i}-foo`, i)
    }
    const ret = await pipeline.exec()
    console.log(ret)
  } catch (err) {
    
    console.log(' operation failed ', err)
  }
}

main()

The official sample

exec Method can accept a callback , Parameters ：

• err Always be null
• results Is the response array corresponding to the queued command , The format of each response is [err, result]

const pipeline = redis.pipeline();
pipeline.set("foo", "bar");
pipeline.del("cc");
pipeline.exec((err, results) => {
    });

You can also chain call ：

redis
  .pipeline()
  .set('foo', 'bar')
  .del('cc')
  .exec((err, results) => {
    })

Each chained command can also accept a callback , This callback will be called when the command is replied ：

redis
  .pipeline()
  .set("foo", "bar")
  .get("foo", (err, result) => {
    
    // result === 'bar'
  })
  .exec((err, result) => {
    
    // result[1][1] === 'bar'
  });

In addition to adding commands to the pipeline queue separately , You can also pass commands and arguments to constructors as arrays ：

redis
  .pipeline([
    ["set", "foo", "bar"],
    ["get", "foo"],
  ])
  .exec(() => {
    
    /* ... */
  });

length Property to display the number of commands in the pipeline ：

const length = redis.pipeline().set("foo", "bar").get("foo").length;
// length === 2

Transaction Business

Most of the time , Business order multi & exec And pipeline Use it together . therefore , Calling multi when , Automatically created by default Pipeline example , So you can use it like a pipe multi：

redis
  .multi() //  The default is to return a  Pipeline  example 
  .set("foo", "bar")
  .get("foo")
  .exec((err, results) => {
    
    // results === [[null, 'OK'], [null, 'bar']]
  });

If there is a syntax error in the command chain of the transaction （ For example, the number of parameters is wrong 、 Wrong command name, etc ）, Will be in ioredis Be identified , Will not execute to Redis Send any command , And return error ：

redis
  .multi()
  .set("foo")
  .set("foo", "new value")
  .exec((err, results) => {
    
    // err:
    // { [ReplyError: EXECABORT Transaction discarded because of previous errors.]
    // name: 'ReplyError',
    // message: 'EXECABORT Transaction discarded because of previous errors.',
    // command: { name: 'exec', args: [] },
    // previousErrors:
    // [ { [ReplyError: ERR wrong number of arguments for 'set' command]
    // name: 'ReplyError',
    // message: 'ERR wrong number of arguments for \'set\' command',
    // command: [Object] } ] }
  });

In terms of interfaces ,multi And pipeline The difference is that , When a callback is specified for each linked command , Pass the queued status to the callback , Not the result of a command ：

redis
  .multi()
  .set("foo", "bar", (err, result) => {
    
    // result === 'QUEUED'
  })
  .exec(/* ... */);

If you want to use without pipeline The business of , Please put { pipeline: false } Pass to multi, Each command will be sent immediately to Redis, Without waiting exec call （ However, it is recommended to use pipeline Increase of efficiency ）：

redis.multi({
     pipeline: false }); //  Return to one  Promise
redis.set("foo", "bar"); //  Note that there are no chained calls 
redis.get("foo");
redis.exec((err, result) => {
    
  // result === [[null, 'OK'], [null, 'bar']]
});

multi The constructor of also accepts a batch of commands ：

redis
  .multi([
    ["set", "foo", "bar"],
    ["get", "foo"],
  ])
  .exec(() => {
    
    /* ... */
  });

pipeline Support inline transactions , This means that you can pipeline The subset of commands in are grouped into one transaction ：

redis
  .pipeline() //  establish  pipeline
  .get("foo")
  .multi() //  Open transaction 
  .set("foo", "bar")
  .get("foo")
  .exec() //  Execute the transaction command 
  .get("foo")
  .exec(); //  perform  pipeline  command 

Error handling

Redis All errors returned by the server are ReplyError Example , Can pass Redis Visit ：

const Redis = require("ioredis");
const redis = new Redis();
// This command causes a reply error since the SET command requires two arguments.
redis.set("foo", (err) => {
    
  err instanceof Redis.ReplyError;
});

This is a ReplyError The error stack of ：

ReplyError: ERR wrong number of arguments for 'set' command
    at ReplyParser._parseResult (/app/node_modules/ioredis/lib/parsers/javascript.js:60:14)
    at ReplyParser.execute (/app/node_modules/ioredis/lib/parsers/javascript.js:178:20)
    at Socket.<anonymous> (/app/node_modules/ioredis/lib/redis/event_handler.js:99:22)
    at Socket.emit (events.js:97:17)
    at readableAddChunk (_stream_readable.js:143:16)
    at Socket.Readable.push (_stream_readable.js:106:10)
    at TCP.onread (net.js:509:20)

By default , The error stack has no meaning , Because the whole stack happens in ioredis The module itself, not the code . therefore , It's not easy to find out where the code went wrong . ioredis Provides an option showFriendlyErrorStack To solve the problem . Enable showFriendlyErrorStack when ,ioredis Will optimize the error stack for you ：

const Redis = require("ioredis");
const redis = new Redis({
     showFriendlyErrorStack: true });
redis.set("foo");

The output will be ：

ReplyError: ERR wrong number of arguments for 'set' command
    at Object.<anonymous> (/app/index.js:3:7)
    at Module._compile (module.js:446:26)
    at Object.Module._extensions..js (module.js:464:10)
    at Module.load (module.js:341:32)
    at Function.Module._load (module.js:296:12)
    at Function.Module.runMain (module.js:487:10)
    at startup (node.js:111:16)
    at node.js:799:3

This time, , The stack tells you that the error occurred on the third line of code . however , Optimizing the error stack can significantly reduce performance . therefore , By default , This option is disabled , Only for debugging purposes . It is not recommended to use this feature in a production environment .