当前位置:网站首页>Chapter 05_ Storage engine
Chapter 05_ Storage engine
2022-07-05 21:08:00 【Passerby a_ passerby】
The first 05 Chapter _ Storage engine
List of articles
- 1. View storage engine
- 2. Set the default storage engine of the system
- 3. Set the storage engine for the table
- 4. Engine introduction
- 4.1 InnoDB engine : Transaction storage engine with foreign key support *
- 4.2 MyISAM engine : The main non transactional storage engine
- 4.3 Archive engine : For data archiving
- 4.4 Blackhole engine : Discard write operation , The read operation will return empty content
- 4.5 CSV engine : When storing data , Separate data items with commas
- 4.6 Memory engine : Table in memory
- 4.7 Federated engine : Access remote tables
- 4.8 Merge engine : Manage multiple MyISAM A collection of tables made up of tables
- 4.9 NDB engine :MySQL Cluster specific storage engine
- 4.10 Engine comparison
- 5. MyISAM and InnoDB
- 6. Alibaba 、 Which one does Taobao use
- Add :
- 2、InnoDB and ACID Model
1. View storage engine
- see mysql What storage engine is provided :
show engines;
show engines \G;
The explicit expression is as follows :
*************************** 1. row ***************************
Engine: InnoDB
Support: DEFAULT Comment: Supports transactions, row-level locking, and foreign keys Transactions: YES XA: YES Savepoints: YES
2. Set the default storage engine of the system
- View the default storage engine :
show variables like '%storage_engine%';
# or
SELECT @@default_storage_engine;
- Modify the default storage engine
If the storage engine of the table is not explicitly specified in the statement creating the table , That will default to InnoDB
As a storage engine for tables .
If we want to change the default storage engine of the table , You can write the command line to start the server like this :
SET DEFAULT_STORAGE_ENGINE=MyISAM;
Or modify my.cnf
file :
default-storage-engine=MyISAM
# Restart the service
systemctl restart mysqld.service
3. Set the storage engine for the table
The storage engine is responsible for extracting and writing the data in the table , We can Different tables set different storage engines
, In other words, different tables can have different physical storage structures , Different extraction and writing methods .
3.1 Specify the storage engine when creating the table
Our previous statements to create tables did not specify the storage engine of the table , Then the default storage engine will be used InnoDB
. If we want to explicitly specify the storage engine of the table , It can be written like this :
CREATE TABLE Table name (
Create table statement ;
) ENGINE = Storage engine name ;
3.2 Modify the storage engine of the table
If the table has been built , We can also use the following statement to modify the storage engine of the table :
ALTER TABLE Table name ENGINE = Storage engine name ;
For example, let's modify engine_demo_table
Table storage engine :
mysql> ALTER TABLE engine_demo_table ENGINE = InnoDB;
Query OK, 0 rows affected (0.05 sec)
Records: 0 Duplicates: 0 Warnings: 0
Now let's check again engine_demo_table
The table structure :
mysql> SHOW CREATE TABLE engine_demo_table\G
*************************** 1. row ***************************
Table: engine_demo_table Create Table: CREATE TABLE `engine_demo_table` (
`i` int(11) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8
1 row in set (0.01 sec)
4. Engine introduction
4.1 InnoDB engine : Transaction storage engine with foreign key support *
MySQL from 3.23.34a Start with InnoDB Storage engine .
Greater than or equal to 5.5 after , By default InnoDB engine
.InnoDB yes MySQL Of
Default transactional engine
, It's designed to handle a lot of short-term (short-lived) Business . It can ensure the complete submission of transactions (Commit) And rollback (Rollback).In addition to adding and querying , It needs to be updated 、 Delete operation , that , Preference should be given to InnoDB Storage engine .
Unless there's a very special reason to use another storage engine , Otherwise, priority should be given to InnoDB engine .
Data file structure :( stay 《 The first 02 Chapter _MySQL Data directory 》 The chapter has been talked about )
Table name .frm Storage table structure (MySQL8.0 when , Merge in table name .ibd in )
Table name .ibd Store data and index
InnoDB yes
Designed for maximum performance in handling huge amounts of data
.- In previous releases , Dictionary data is stored as metadata files 、 Non transaction table, etc . Now these metadata files have been deleted . such as :
.frm
,.par
,.trn
,.isl
,.db.opt
Waiting MySQL8.0 There is no such thing as .
- In previous releases , Dictionary data is stored as metadata files 、 Non transaction table, etc . Now these metadata files have been deleted . such as :
contrast MyISAM Storage engine for ,
InnoDB The processing efficiency of writing is poor
, And it will take up more disk space to save data and indexes .MyISAM Cache index only , Don't cache real data ;InnoDB Not only index but also real data ,
High memory requirements
, And memory size has a decisive impact on performance .
4.2 MyISAM engine : The main non transactional storage engine
MyISAM There are a lot of features , Include full text index 、 Compress 、 Space function (GIS) etc. , but MyISAM
Unsupported transaction 、 Row level lock 、 Foreign keys
, There is no doubt that the defect isCan't recover safely after a crash
.5.5 The previous default storage engine
The advantage is access
Fast
, There is no requirement for transaction integrity or SELECT、INSERT Mainly applicationsThere is additional constant storage for data statistics . so count(*) The query efficiency is very high
Data file structure :( stay 《 The first 02 Chapter _MySQL Data directory 》 The chapter has been talked about )
Table name .frm Storage table structure
Table name .MYD Store the data (MYData)
Table name .MYI Storage index (MYIndex)
Application scenarios : Read only applications or read-only businesses
4.3 Archive engine : For data archiving
- The following table shows ARCHIVE Storage engine functions
features | Support |
---|---|
B Tree index | I won't support it |
Backup / Time to recover ( Implement... In the server , Not in the storage engine ) | Support |
Cluster database supports | I won't support it |
Clustered index | I won't support it |
compressed data | Support |
Data caching | I won't support it |
Encrypt data ( The encryption function is implemented in the server ) | Support |
Foreign key support | I won't support it |
Full text search index | I won't support it |
Geospatial data types support | Support |
Hash index | I won't support it |
The index buffer | I won't support it |
Lock granularity | Row lock |
MVCC | I won't support it |
Storage limits | No restrictions |
transaction | I won't support it |
Update the statistics of the data dictionary | Support |
4.4 Blackhole engine : Discard write operation , The read operation will return empty content
4.5 CSV engine : When storing data , Separate data items with commas
Use cases are as follows
mysql> CREATE TABLE test (i INT NOT NULL, c CHAR(10) NOT NULL) ENGINE = CSV;
Query OK, 0 rows affected (0.06 sec)
mysql> INSERT INTO test VALUES(1,'record one'),(2,'record two');
Query OK, 2 rows affected (0.05 sec)
Records: 2 Duplicates: 0 Warnings: 0
mysql> SELECT * FROM test;
+---+------------+
| i | c |
+---+------------+
| 1 | record one |
| 2 | record two |
+---+------------+
2 rows in set (0.00 sec)
establish CSV The table also creates the corresponding Metafile
, be used for The state of the storage table
and Number of rows in the table
. The name of this file is the same as that of the table , The suffix is CSM
. As shown in the figure
If you check test.CSV
Files in the database directory created by executing the above statement , Its content uses Notepad++ Open as follows :
"1","record one"
"2","record two"
This format can be Microsoft Excel Wait for the spreadsheet application to read , Even write . Use Microsoft Excel Open as shown in the figure
4.6 Memory engine : Table in memory
summary :
Memory The logical medium used is Memory , Fast response
, But when mysqld When the daemon crashes Data will be lost
. in addition , The stored data is required to be in a format with constant data length , such as ,Blob and Text Data of type is not available ( Of variable length ).
Main features :
- Memory meanwhile
Support hash (HASH) Indexes
andB+ Tree index
. - Memory The watch is at least better than MyISAM Table
One order of magnitude fast
. - MEMORY
The size of the table is limited
Of . The size of the table mainly depends on two parameters , Namelymax_rows
andmax_heap_table_size
. among ,max_rows You can specify... At creation time ;max_heap_table_size The default size of is 16MB, It can be expanded as needed . - Data files are stored separately from index files .
- shortcoming : Its data is easy to lose , Short life cycle . Based on this flaw , choice MEMORY You need to be very careful when storing the engine .
Use Memory Storage engine scenario :
The target data is relatively small
, And veryFrequent visits
, Store data in memory , If the data is too large, it will causeout of memory
. You can use the parametermax_heap_table_size
control Memory The size of the table , Limit Memory The maximum size of the table .If
The data is provisional
, andMust be available immediately
obtain , Then you can put it in memory .Stored in Memory If the data in the table suddenly
If you lose it, it doesn't matter much
.
4.7 Federated engine : Access remote tables
- Federated The engine is accessing other MySQL One of the servers
agent
, Although the engine seems to provide a goodCross server flexibility
, But it often brings problems , therefore It is disabled by default .
4.8 Merge engine : Manage multiple MyISAM A collection of tables made up of tables
4.9 NDB engine :MySQL Cluster specific storage engine
It's also called NDB Cluster Storage engine , It is mainly used for MySQL Cluster Distributed cluster
Environmental Science , Be similar to Oracle Of RAC colony .
4.10 Engine comparison
MySQL In the same database , Different storage engines can be selected for different tables . The following table compares the common storage engines .
characteristic | MyISAM | InnoDB | MEMORY | MERGE | NDB |
---|---|---|---|---|---|
Storage limits | Yes | 64TB | Yes | No, | Yes |
Transaction security | Support | ||||
Locking mechanism | Table locks , Even operating on one record locks the entire table , Not suitable for highly concurrent operations | Row lock , Lock only one line during operation , It doesn't affect anything else , Suitable for high concurrency operations | Table locks | Table locks | Row lock |
B Tree index | Support | Support | Support | Support | Support |
Hash index | Support | Support | |||
Full-text index | Support | ||||
Cluster index | Support | ||||
Data caching | Support | Support | Support | ||
The index buffer | Cache index only , Don't cache real data | Not only index but also real data , High memory requirements , And memory size has a decisive impact on performance | Support | Support | Support |
Data can be compressed | Support | ||||
Space use | low | high | N/A | low | low |
Memory usage | low | high | secondary | low | high |
Speed of batch insertion | high | low | high | high | high |
Support foreign keys | Support |
In fact, there is no need to remember these things immediately , The purpose of listing is to let everyone understand that different storage engines support different functions .
In fact, what we use most is InnoDB
and MyISAM
, Sometimes I mention Memory
. among InnoDB yes MySQL The default storage engine .
5. MyISAM and InnoDB
A lot of people InnoDB and MyISAM There are questions about the choice of , Which is better to choose ?
MySQL5.5 The previous default storage engine was MyISAM,5.5 Then it changed to InnoDB.
Contrast item | MyISAM | InnoDB |
---|---|---|
Foreign keys | I won't support it | Support |
Business | I won't support it | Support |
Row table lock | Table locks , Even operating on one record locks the entire table , Not suitable for highly concurrent operations | Row lock , Lock only one line during operation , It doesn't affect anything else , Suitable for high concurrency operations |
cache | Cache index only , Don't cache real data | Not only index but also real data , High memory requirements , And memory size has a decisive impact on performance |
Use the self-contained system table | Y | N |
concerns | performance : Save resources 、 Less consumption 、 Simple business | Business : Write concurrently 、 Business 、 More resources |
Default installation | Y | Y |
By default | N | Y |
6. Alibaba 、 Which one does Taobao use
- Percona by MySQL The database server has been improved , In terms of function and performance MySQL There is a significant improvement .
- This version improves the performance of InnoDB Performance of 、 by DBA Provides some very useful performance diagnostic tools ; In addition, there are more parameters and commands to control the behavior of the server .
- The company has built a new storage engine called
Xtradb
Completely replaceableInnodb
, And better in performance and concurrency - Most of Alibaba mysql The database actually uses percona To modify the prototype of .
Add :
1、InnoDB Advantages of watch
InnoDB Storage engine has many advantages in practical application , For example, the operation is convenient 、 Improve the performance of the database 、 Low maintenance costs . If the server crashes due to hardware or software reasons , Then no additional operation is required after restarting the server .InnoDB Crash recovery automatically finalizes previously submitted content , Then undo the uncommitted process , After restart, continue to execute from the crash point .
InnoDB The storage engine maintains buffer pools in main memory , High frequency data will be processed directly in memory . This caching method can be applied to a variety of information , Speed up the processing process .
On a dedicated server , Up to... In physical memory 80% The part of is applied to the buffer pool . If you need to insert data into different tables , You can set foreign keys to enhance data integrity . Update or delete data , The associated data will be automatically updated or deleted . If you try to insert data from a table , But there is no corresponding data in the main table , The inserted data will be removed automatically . If the data in disk or memory crashes , Before using dirty data , The checksum mechanism will issue a warning . When the primary key of each table is set reasonably , Operations related to these columns are automatically optimized . Insert 、 Update and delete operations are optimized by changing the automatic buffer mechanism . InnoDB It not only supports current reading and writing , It will be Buffer the changed data to the data stream disk .
InnoDB The performance advantage of is not limited to large tables that run queries for a long time . When the same column is queried multiple times , Adaptive hash index will improve the speed of query . Use InnoDB You can compress tables and related indexes , Sure Create or delete indexes without affecting performance and availability
. For large text and BLOB data , Use dynamic line form , This storage layout is more efficient . By inquiring INFORMATION_SCHEMA The tables in the library can monitor the internal work of the storage engine . In the same statement ,InnoDB Tables can be mixed with other storage engine tables . Even some operating systems limit the file size to 2GB,InnoDB Can still handle . When dealing with large amounts of data , InnoDB To two or morethings CPU, For maximum performance
.
2、InnoDB and ACID Model
ACID A model is a set of database design rules , These rules emphasize reliability , Reliability is important for business data and mission critical applications .MySQL Contain similar InnoDB Components of the storage engine , And ACID The models are closely connected , When there is an accident like this , Data will not crash , The result will not be distorted . If you rely on ACID Model , Consistency checking and crash recovery mechanisms may not be used . If you have additional software protection , Extremely reliable hardware or applications can tolerate a small amount of data loss and inconsistency , Can be MySQL Adjust the setting to depend only on the part ACID characteristic , To achieve higher performance . The following interpretation InnoDB Storage engine and ACID Four aspects of the same role of the model .
1. Atomic aspect ACID The atomic aspect of mainly involves InnoDB Business , And MySQL Related features mainly include :
- Auto submit settings .
- COMMIT sentence .
- ROLLBACK sentence .
- operation INFORMATION_SCHEMA Table data in the library .
2. In terms of consistency ACID The consistency of the model mainly involves the internal protection of data from collapse InnoDB Treatment process , And MySQL Related features
It mainly includes :
- InnoDB Double write cache .
- InnoDB Crash recovery .
3. In isolation Isolation is the level applied to transactions , And MySQL Related features mainly include :
- Auto submit settings .
- SET ISOLATION LEVEL sentence .
- InnoDB Low level information of lock .
4. Durability ACID The durability of the model mainly involves the factors that interact with the hardware configuration MySQL Software features . Due to the complexity and diversity of hardware , There are no specific rules to follow in terms of durability . And MySQL Related features are :
- InnoDB Double write cache , adopt innodb_doublewrite Configuration Item Configuration .
- Configuration item innodb_flush_log_at_trx_commit.
- Configuration item sync_binlog.
- Configuration item innodb_file_per_table.
- Write cache of storage device .
- Backup battery cache of storage device .
- function MySQL Operating system of .
- Continuous power supply .
- Backup policy .
- For distributed or hosted applications , The most important thing is the location of hardware equipment and network conditions .
3、InnoDB framework
1. Buffer pool Buffer pool is a part of main memory , Used to cache used table and index data . Buffer pools enable frequently used data to be obtained directly in memory , To speed up .
2. Change cache Change cache is a special data structure , When the affected index page is not in the cache , Change cache caches changes to secondary index pages . The index page is loaded into the cache pool when it is read by other operations , The cached changes are merged . Different from cluster index , Secondary indexes are not unique . When most of the system is idle , The purge operation runs periodically , Brush the updated index page to disk . Update cache merge period , It may greatly reduce the performance of queries . In memory , Updating the cache takes up part of InnoDB Buffer pool . On the disk , The update cache is part of the system tablespace . The data type of update cache is determined by innodb_change_buffering Configuration item management .
3. adaptive hash index Adaptive hash index combines load with enough memory , bring InnoDB Run like an in memory database , There is no need to reduce performance or reliability on transactions . This feature passes through innodb_adaptive_hash_index Options configuration , Or by -- skip-innodb_adaptive_hash_index The command line shuts down when the service starts .
4. Redo log cache The redo log cache stores the data to be put into the redo log . Redo log cache size passed innodb_log_buffer_size Configuration Item Configuration . Redo log cache will periodically flush log files to disk . Large redo log cache enables large transactions to run normally without writing to disk .
5. SYSTEM tablespace System tablespaces include InnoDB The data dictionary 、 Double write cache 、 Update cache and undo logs , It also includes table and index data . Multi table sharing , System tablespaces are considered shared tablespaces .
6. Double write cache The double write cache is located in the system table space , Used to write data pages refreshed from the cache pool . Only after refreshing and writing to the double write cache ,InnoDB Will write the data page to the appropriate location .
7. Undo log An undo log is a collection of transaction related undo records , Contains how to undo the most recent changes to the transaction . If other transactions want to query the original data , You can trace unchanged data from the undo log record . The undo log exists in the undo log fragment , this
These fragments are included in the rollback fragment .
8. The tablespace of one file per table The tablespace of one file per table means that each individual tablespace is created in its own data file , Not in the system tablespace . This function passes innodb_file_per_table Configuration item on . Each table space consists of a separate .ibd Count
According to the document , The file is created in the database directory by default .
9. General tablespace Use CREATE TABLESPACE Syntax to create shared InnoDB Table space . Common tablespaces can be created in MySQL Outside the data directory, it can manage multiple tables and support tables in all row formats .
10. Undo tablespace Undo tablespaces consist of one or more files that contain undo logs . The number of undo tablespaces is determined by innodb_undo_tablespaces Configuration Item Configuration .
11. Temporary table space Both user created temporary tablespaces and disk based internal temporary tables are created in temporary tablespaces .innodb_temp_data_file_path The configuration item defines the relevant path 、 name 、 Size and properties . If the value is null , The default will be innodb_data_home_dir Automatically create the next variable in the specified directory .
12. Redo log Redo logs are disk based data structures , Use... During crash recovery , Used to correct data . During normal operation , The redo log encodes the requested data , These requests will change InnoDB Table data . After an unexpected crash , Incomplete changes are automatically redone during initialization .
边栏推荐
- @Validated基础参数校验、分组参数验证和嵌套参数验证
- 水泥胶黏剂BS 476-4 不燃性测试
- 浅聊我和一些编程语言的缘分
- vant 源码解析 之深层 合并对象 深拷贝
- LeetCode: Distinct Subsequences [115]
- Utils/index TS tool function
- MySQL deep paging optimization with tens of millions of data, and online failure is rejected!
- Clion-MinGW编译后的exe文件添加ico图标
- ViewRootImpl和WindowManagerService笔记
- LeetCode: Distinct Subsequences [115]
猜你喜欢
MySQL deep paging optimization with tens of millions of data, and online failure is rejected!
R language [data management]
事项研发工作流全面优化|Erda 2.2 版本如“七”而至
Open source SPL eliminates tens of thousands of database intermediate tables
leetcode:1755. 最接近目标值的子序列和
Research and development efficiency improvement practice of large insurance groups with 10000 + code base and 3000 + R & D personnel
Why can't Chinese software companies produce products? Abandon the Internet after 00; Open source high-performance API gateway component of station B | weekly email exclusive to VIP members of Menon w
MySQL 千万数据量深分页优化, 拒绝线上故障!
LeetCode_ Hash table_ Difficulties_ 149. Maximum number of points on the line
基于vertx-web-sstore-redis的改造实现vertx http应用的分布式session
随机推荐
Is Kai Niu 2980 useful? Is it safe to open an account
【案例】元素的显示与隐藏的运用--元素遮罩
leetcode:1755. 最接近目标值的子序列和
Sophomore personal development summary
@Validated basic parameter verification, grouping parameter verification and nested parameter verification
js常用方法封装
终端安全能力验证环境搭建和渗透测试记录
Sequence alignment
Aitm 2-0003 horizontal combustion test
ViewRootImpl和WindowManagerService笔记
Viewrootimpl and windowmanagerservice notes
Modifiers of attributes of TS public, private, protect
中国的软件公司为什么做不出产品?00后抛弃互联网;B站开源的高性能API网关组件|码农周刊VIP会员专属邮件周报 Vol.097
LeetCode: Distinct Subsequences [115]
Using webassembly to operate excel on the browser side
使用WebAssembly在浏览器端操作Excel
秋招将临 如何准备算法面试、回答算法面试题
Comparison table of foreign lead American abbreviations
vant 源码解析 event.ts 事件处理 全局函数 addEventListener详解
Five layer network protocol