Differences between nodes and sharding in ES cluster

At first I got confused , Think partition is node

node ： Nodes are our hosts one by one , You can also understand it as one by one ES

Fragmentation ： Sharding is to shard the data originally stored on a node , Then store it to different nodes .

Slice is divided into main slice and sub slice , Secondary fragmentation is used for data copying

For example, create an index library here

PUT /test
{
  "settings": {
    "number_of_shards": 3, //  Number of slices 
    "number_of_replicas": 1 //  Copy number 
  },
  "mappings": {
    "properties": {
      // mapping Mapping definition  ...
    }
  }
}

This means that the data of the index database will be stored in three partitions , Which node will the three partitions be distributed to? By default, there are ES control , At the same time, it can also be controlled by the master node

es01、es02、es03 Represents three nodes , The entity asterisk indicates that this node is the primary node , The number on the right is the number of slices , And it shows the distribution of slices on nodes

Let's look at the process of inserting a document , In this way, we can clearly understand the difference between nodes and partitions

The process ：

• 1） Add a new one id=1 Documents

• 2） Master node pair id do hash operation , If you get 2, Should be stored in shard-2

• 3）shard-2 The main partition of is node3 node , Route data to node3

• 4）node3 Save the document

• 5）node3 Synchronize data to shard-2 Copy of replica-2, stay node2 node

• 6） Return the result to coordinating-node node

• 7） Return the result to the user

  annotation ：

• 1）shard It means dividing into pieces

• 2）hash Algorithm will document id Get the sequence number of slices by taking the remainder with the number of slices , Then the document data is stored in the host where the partition is located data Go to the data node

• 3） When the master partition saves data , Data backup should also be carried out for secondary fragmentation

Now you should know the difference between sharding and nodes