Mongodb replica set

Catalog

One 、 Replica set architecture

1、primary

2、secondary

3、Arbiter

Two 、oplog

1、oplog structure

2、 Synchronization delay

3、oplog Import and export of

---

One 、 Replica set architecture

        Reference resources ：Replica Set Members — MongoDB Manual

         mongoDB 3 The replica set in version supports at most 50 Nodes , Seven of these nodes have voting rights .

1、primary

         There can be at most one in the replica set Primary node .

         User oriented , All data write operations work on this node . At the same time, this node also supports read operations , All nodes in the replica set support read operations , But the user's read operation works on by default Primary node .

         Left and right data changes on this node , Will produce oplog（ The operation log ）.Secondary Nodes can pass through Primary Copy oplog Synchronous data .

2、secondary

         Maintenance and Primary Copy of the same data , Read operation is supported . At the same time, it also acts as Primary Candidate node , When Primary In case of failure, you can recommend yourself as Primary, Then each node votes , elect Primary node . Be careful , The weight of 0 The node of cannot be elected Primary node （ The weight of 0 The node of still has the function of reading and voting ）.

        Secondary Nodes can also be hidden . Hidden Secondary, Normal synchronization Primary Node data , But its weight is 0, Cannot be elected Primary node （ You can vote ）, And invisible to users （db.isMaster() Do not show hidden nodes ）. Hidden Secondary Nodes can be used to perform some special tasks （ The report 、 Backup, etc ）, In addition, there will be no other communication traffic .

        Secondary You can also set delay nodes （delay node）, The delay node can delay the backup according to the set time Primary The data of . This is shown below , Delay nodes must be set to hidden , And the weight is 0. The role of delayed nodes may be a little hard to understand , Why delay replication ？ In fact, the data of the delay node is equivalent to Primary Data snapshot of the node at a certain time in the past . All in the replica set Secondary Nodes are all at Primary Data synchronization , But if some mistakes of operators make the data in the whole cluster go wrong , Setting a delay node can restore data to a certain time in the past .

3、Arbiter

         Arbitration node , Don't store data , Can't be Primary node , Mainly used for election from Seconary Election in node Primary node .

Two 、oplog

         All in the replica set Secondary And Primary All are data synchronized （ Except for delay nodes ）, Between them oplog Data synchronization .oplog yes Primary Data operation log on node , Idempotent （ Multiple execution , Consistent result ）,Secondary basis oplog You can reproduce Primary All data operations of the node , Data synchronization . All nodes in the replica set will maintain one locally oplog（lcoal.oplog.rs）, Used to record the status of maintenance data .

        oplog The size constraints are as follows .

1、oplog structure

         have access to mongodump export oplog data , Look at the structure .

         because mongodump Export data as bson Format , You can use bsondump Convert to json Format .

attribute	describe	Value
ts	Time stamp . There are two variables , The former indicates the time when the operation occurred （ Company ： second ）, The latter represents the order of operation implementation in the same time （ The default from the 1 Start ）	(timestamp, count)
h	Uniquely identify the operation ID
t
v	edition
op	i：insert, Database insert operation
	d：delete, Database delete operation
	u：update, Database update operation
	c：command, Database execution command , Set as built
	n：none, Empty operation
ns	Write operations	db_name.collection_name
	Database command execution	db_name.$cmd
	Empty operation	blank
o	Initialize replica set	{"msg":"initiate set"}
	Select the master node	{"msg": "new primary"}
	Add the first slave node	{"msg": "Reconfig Set", "version":2}
	Add a second slave node	{"msg": "Reconfig Set", "version":3}
	Remove a slave node	{"msg": "Reconfig Set", "version":4}
	Create set	{"create": "collection_name" }
	Delete the collection	{"drop": "collection_name"}
	Delete database	{ "dropDatabase" : 1 }
	Insert or delete documents	{"_id":1, "Name":"ABC"}
	Update the document	{ "$set":{ "Name":"MyTest" } }

2、 Synchronization delay

         Because the master node performs all write operations , The slave node can share the load pressure of the master node's read operation . In general business scenarios , Write less and read more , Therefore, synchronizing the data of the slave node and the master node can well reduce the business pressure . And when the master node fails, the data of the slave node can be used to recover the data of the master node .

         When the master node writes data , Will generate one or more oplog journal . Generate from the master node oplog journal , To the slave node oplog Go local and copy oplog This period of logging is the synchronization delay . There are many factors affecting this delay , The Internet , The workload of nodes may be affected .

3、oplog Import and export of

        oplog The import and export of can use mongodump And mongorestore These two tools . For specific parameter explanation, please refer to the introduction on the official website .

         There are two points to note ：

        0、 Use mongodump have access to query Parameter execution json String for data filtering .

        1、mongorestore Only insert is supported , Overwrite operation is not supported . That is, when the data is restored, if the same exists ID Documents , Ignore . Of course, this is right oplog It doesn't work , because oplog Recorded hourly operation log , Every data update operation will generate new oplog Record . When using mongorestore Direct recovery mongodump You need to pay attention to this when exporting database data .

        2、mongorestore Data recovery will also produce oplog journal .

mongodump — MongoDB Manualhttps://www.mongodb.com/docs/v3.4/reference/program/mongodump/mongorestore — MongoDB Manualhttps://www.mongodb.com/docs/v3.4/reference/program/mongorestore/