Some principles of mongodb optimization

The basic chapter （ Can solve the problem of 80% The problem of ）:

1. MongoDB Overview 、 Application scenarios 、 Download mode 、 Connection mode and development history, etc

2. MongoDB data type 、 Key concepts and shell Commonly used instructions

3. MongoDB Various additions to documents 、 to update 、 Delete operation summary

4. MongoDB Summary of various query operations

5. MongoDB Summarize the various operations of the column

6. MongoDB Summary of index operations in

Advanced :

7. MongoDB Summary of aggregation operations

8. MongoDB Import and export of 、 Backup recovery summary

9. MongoDB Summary of user management

10. MongoDB Copy （ Replica set ） summary

11. MongoDB Slice summary

12. MongoDB meet spark（ Integration ）

13. MongoDB Internal storage principle

Other :

14. python3 operation MongoDB Various cases of

15. MongoDB Command summary

1. Query optimization

Make sure your query takes full advantage of the index , use explain Command to check the execution of the query , Add the necessary indexes , Avoid table scanning .

2. Figure out the size of the hot data

Maybe your data set is very large , But it's not that important , What matters is how big your thermal data set is , How big is the data you often visit ( Including frequently accessed data and all index data ).

Use MongoDB, You'd better make sure your hot data is below the memory size of your machine , Make sure the memory can hold all the hot data .

3. Choose the right file system

MongoDB The data file is pre allocated , And in Replication Inside ,Master and Replica Sets Non - Arbiter Nodes will create enough empty files in advance to store operation logs .

These file allocation operations can be very slow on some file systems , Cause the process to be Block. So we should choose those file systems with fast space allocation . The conclusion here is to try not to use ext3, use ext4 perhaps xfs.

4. Choose the right hard disk

The choices here include the disk RAID The choice of , It also includes disks and SSD Comparative selection of .

5. Use less as far as possible in How to query

Especially in shard On , He will let your inquiry go by a shand Last run , If you have to use it as a last resort shard Index up .

Optimize in The way is to in Decompose into single queries one by one . The speed will increase 40-50 times

6. Reasonable design sharding key

The incremental sharding-key： Suitable for fields that can be delimited , such as integer、float、date Type of , The query time is relatively fast

Random sharding-key: It is suitable for the scenario of frequent write operations , And in this case, if it's in a shard It's going to make this shard The load is higher than the others , It's not balanced , So I hope to hash Inquire about key, Distribute writes across multiple locations shard on , Consider compounding key As sharding key, The general principle is fast query , Try to reduce the number of spans shard Inquire about ,balance Less equalization .

mongodb The default is a single record 16M, Especially in the use of GFS When , Be sure to pay attention to shrading-key The design of the .

Unjustified sharding-key There will be , Multiple documents , In a chunks On , meanwhile , because GFS Large files are often stored in , Lead to mongodb Doing it balance You can't get through sharding-key To separate these documents into different shard On , Now mongodb Will constantly report errors [conn27669] Uncaught std::exception: St9bad_alloc, terminating. Finally lead to mongodb Fall down .

terms of settlement ： enlarge chunks size （ Cure the symptoms ）, Well designed sharding-key（ Permanent cure ）.

7.mongodb Can pass profile To monitor data , To optimize .

See if it's on profile function Command db.getProfilingLevel() return level Grade , The value is 0|1|2,

Respectively means ：0 On behalf of closed ,1 For recording slow orders ,2 Represent all

Turn on profile The function command is db.setProfilingLevel(level); #level Grade , The value is the same as above. level by 1 When , The default value of slow command is 100ms, Change to db.setProfilingLevel(level,slowms) Such as db.setProfilingLevel(1,50) This changes to 50 millisecond

adopt db.system.profile.find() View the current monitoring log .