Play with mongodb - build a mongodb cluster

1 summary

As the title , Play this time MongoDB Let's start with building clusters , Words MongoDB There are three ways to build clusters , But due to the version update , It is said that in 4.0 The first way after version , That is, the master-slave replication method has been abandoned , Probably because this method is not efficient , Because we are using 5.x edition , So don't spend time explaining the first way , Excerpts from other articles , For your reference . The focus is still on the latter two .

2 Mode one ： Master slave copy

（1） Prepare three machines, one host and two standby

mongod --port 27007 --dbpath /data/master/ --bind_ip 0.0.0.0 --master --oplogSize 50

mongod --port 27008 --dbpath /data/slave1 --bind_ip 0.0.0.0 --slave --source 127.0.0.1:27007 --only ymx

mongod --dbpath /data/slave2 --port 27009 --bind_ip 0.0.0.0 --slave --source 127.0.0.1:27007 --only ymx --slavedelay 30

（2） Options for master-slave replication

• –master ： Master node
• –slave ： From the node
• –source [server:port] ： Is the master node from which to copy <>
• –only [ Library name ] ： For the slave node, only the library of the master node is copied
• –slavedelay [ Number of seconds ]： How long does the slave node delay replicating the master node second
• –autoresync ： The slave data is not up-to-date, but it is automatically resynchronized
• –oplogSize ： The operation log unit of the master node is M

（3） View the cluster status

rs.help();

rs.slaveOk(); // Start the slave query

3 Mode two ：Replica Set（ Replica set ）

（ The picture is from ：https://www.mongodb.com/docs/manual/replication/）

Replica Set The concept translates into a replica set , It is also equivalent to master-slave copy , The master node is responsible for reading and writing data , The slave node is responsible for backing up data , When the primary node goes down , The remaining slave nodes will automatically select the master node for data reading and writing .

（ The picture is from ：https://www.mongodb.com/docs/manual/replication/）

Two roles ：

• Primary： Master node , Responsible for data interaction
• Secondary： From the node , Responsible for data backup

3.1 Overall process

Environmental Science ： a Linux virtual machine , Three MongoDB node （ One master and two subordinates ）

1.  Create the corresponding directory 
2.  Start each node according to the directory and port and other cluster parameters 
3.  Configure the cluster on the master node 
4.  Enter each slave node to observe 
5.  Configuration is successful 

3.2 Specific process

3.2.1 Create the corresponding directory

[email protected]:/data$ sudo mkdir db1
[email protected]:/data$ sudo mkdir db2
[email protected]:/data$ sudo mkdir db3
[email protected]:/data$ ls -al
drwxrwxrwx  6 root root  0 6 month   26 16:28 .
drwxr-xr-x 19 root root  0 6 month   26 07:37 ..
drwxrwxrwx  4 root root  0 6 month   26 21:52 db1
drwxrwxrwx  4 root root  0 6 month   26 21:52 db2
drwxrwxrwx  4 root root  0 6 month   26 21:52 db3
[email protected]:/data$ cd ..
[email protected]:/$ sudo chmod 777 -R data/

3.2.2 Start each node

mongod --port 27011 --dbpath /data/db1 --bind_ip 0.0.0.0 --replSet  ymx/localhost:27012
mongod --port 27012 --dbpath /data/db2 --bind_ip 0.0.0.0 --replSet  ymx/localhost:27013
mongod --port 27013 --dbpath /data/db3 --bind_ip 0.0.0.0 --replSet  ymx/localhost:27011

grammar ：

[ command ] --port [ port ] --dbpath [ Catalog ] --bind_ip [ binding IP] --replSet [ Replica set name ]/[ Host name of adjacent contact : Port number ]
mongod --port 27011 --dbpath /data/db1 --bind_ip 0.0.0.0 --replSet  ymx/localhost:27012

Concept ：

3.2.3 Enter the configuration

grammar ：

var config = {
     
    _id:[ Replica set name ], 
    members:[
        {
    _id:0,host:[ Node hostname ： Port number ]},
        {
    _id:1,host:[ Node hostname ： Port number ]},
        ......
    }
rs.initiate(config);

example ：

[email protected]:~/Desktop/mongodb/bin$ ./mongo 127.0.0.1:27011
......
> use admin
switched to db admin
> 
> var config = {
     
...     _id:"ymx", 
...     members:[
...         {
    _id:0,host:"localhost:27011"},
...         {
    _id:1,host:"localhost:27012"},
...         {
    _id:2,host:"localhost:27013"}]
...     }
> rs.initiate(config);
{
    
        "ok" : 1,
        "$clusterTime" : {
    
                "clusterTime" : Timestamp(1656250256, 1),
                "signature" : {
    
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        },
        "operationTime" : Timestamp(1656250256, 1)
}
ymx:SECONDARY> 
ymx:PRIMARY> 
ymx:PRIMARY> show databases
admin   0.000GB
config  0.000GB
local   0.000GB

3.2.4 Enter other nodes to observe

[email protected]:~/Desktop/mongodb/bin$ ./mongo 127.0.0.1:27012
......
ymx:SECONDARY> 

[email protected]:~/Desktop/mongodb/bin$ ./mongo 127.0.0.1:27013
......
ymx:SECONDARY> 

3.2.5 Configuration is successful

3.3 Be careful

3.3.1 Driver Of URL

mongodb://127.0.0.1:27017,127.0.0.1:27018,127.0.0.1:27019/ems( Library name )?replcaSet=spock( Replica set name )

3.3.2 Access data from nodes

The slave node cannot directly access data , If you need to visit , You can use rs.slaveOk();

4 Mode three ：Sharding（ Fragmentation ）

（ The picture is from ：https://www.mongodb.com/docs/manual/sharding/）

Three roles ：

• Router： route , The client is connected by this , And make the whole cluster look like a single database .
• Config Servers：mongod example , Store the whole ClusterMetadata, Metadata .
• Shard： Used to store actual data blocks , In a real production environment shard server A character can be made up of several machines replica set To undertake , Prevent single point failure of the main engine

4.1 Overall process

Environmental Science ： a Linux virtual machine , One Router、 Two Shard、 Two Config Servers

1.  Create a data directory 
2.  Start according to port and directory Shard node 
3.  Start according to port and directory Config Servers node 
4.  To configure Config Servers Node replica set 
5.  start-up Router node 
6.  utilize mongos Enter the routing configuration partition node 
7.  utilize mongos Configure the shard database and collection as well as the shard key information 

4.2 Specific process

4.2.1 Create a data directory

[email protected]:/$ cd data/

[email protected]:/data$ sudo mkdir shard_test
 Please input a password :
 Verify success 
[email protected]:/data$ cd shard_test/
[email protected]:/data/shard_test$ ls
[email protected]:/data/shard_test$ sudo mkdir db1 db2 config1 config2
[email protected]:/data/shard_test$ ls
config1  config2  db1  db2
[email protected]:/data/shard_test$ cd ..
[email protected]:/data$ sudo chmod 777 -R shard_test/

4.2.2 start-up Shard node

mongod --port 27021 --dbpath /data/shard_test/db1 --bind_ip 0.0.0.0 --shardsvr --replSet "shards"/localhost:27022

mongod --port 27022 --dbpath /data/shard_test/db2 --bind_ip 0.0.0.0 --shardsvr --replSet "shards"/localhost:27021

Get into Shard Node configuration ：

[email protected]:~/Desktop/mongodb/bin$ ./mongo 127.0.0.1:27011
......
> use admin
switched to db admin
> 
> var config = {
     
...     _id:"shards", 
...     members:[
...         {
    _id:0,host:"localhost:27022"},
...         {
    _id:1,host:"localhost:27021"},]
...     }
> rs.initiate(config);
{
    
        "ok" : 1,
        "$clusterTime" : {
    
                "clusterTime" : Timestamp(1656250256, 1),
                "signature" : {
    
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        },
        "operationTime" : Timestamp(1656250256, 1)
}

4.2.3 start-up Config Servers node

mongod --port 27100 --dbpath /data/shard_test/config1 --bind_ip 0.0.0.0 --replSet  "configs"/localhost:27101 --configsvr

mongod --port 27101 --dbpath /data/shard_test/config2 --bind_ip 0.0.0.0 --replSet "configs"/localhost:27100 --configsvr

4.2.4 To configure Config Servers Node replica set

use admin 
var config = {
     
	 	_id:"configs", 
		configsvr: true,
	 	members:[
	   	{
    _id:0,host:"localhost:27100"},
	   	{
    _id:1,host:"localhost:27101"}]
}
rs.initiate(config);

example ：

[email protected]:~/Desktop/mongodb/bin$ ./mongo -port 27999
......
mongos> use admin
switched to db admin
mongos> var config = {
     
	 	_id:"configs", 
		configsvr: true,
	 	members:[
            {
    _id:0,host:"localhost:27100"},
            {
    _id:1,host:"localhost:27101"}
	   	]
	   	}
mongos> rs.initiate(config);

4.2.5 start-up Router node

./mongos --port 27999 --configdb "configs"/localhost:27100,localhost:27101 --bind_ip 0.0.0.0

Be careful : config Is the replica set name above

4.2.6 utilize mongos Enter the routing configuration partition node

./mongo -port 27999

use admin

db.runCommand({
     addshard:"localhost:27021","allowLocal":true });

db.runCommand({
     addshard:"localhost:27022","allowLocal":true });

4.2.7 utilize mongos Configure the shard database and collection as well as the shard key information

• Set partitioned Library :db.runCommand({ enablesharding:“[ Library name ]” });

• Set the set and slice key information in the Library :

  db.runCommand({ shardcollection: “[ Library name ].users”, key: { _id:1}}) // according to id Fragmentation

  db.runCommand({ shardcollection: “[ Library name ].users”, key: { _id: “hashed”}}) // according to hash Fragmentation

5 summary

Summarize the advantages and disadvantages of the three types ：

• Master slave copy ：
  • advantage ： Simple configuration
  • shortcoming ： Not recommended
• Replica Set：
  • advantage ： High availability , Automatic failover
  • shortcoming ： It takes up too much storage space , data redundancy
• Sharding：
  • advantage ： High utilization of storage space
  • shortcoming ： The node type is complex

see you~