当前位置:网站首页>Database storage - table partition
Database storage - table partition
2022-07-07 08:43:00 【Blue sky ⊙ white clouds】
With the development of the project , There are more and more single table data in the database , Related operations are getting slower , At this time, how can we improve our relevant operation efficiency ? Many people have heard of Sub database and sub table , But there is another way of partitioning that may be ignored , When the data volume has not reached the level of tens of millions , We may also be able to use partitioning , Let the data of a table be distributed on different files , Of course, we should be clear that our table data is stored on the disk in the form of files , Partition and sub table mean different things , Partitioning refers to distributing the data of a table to different files according to conditions , Before partitioning, it is stored on a file , But it still points to the same table , Just spread the data to different files , But the sub table spreads the data to different tables , Although the structure is the same , But the name of the table has changed . Partitioning helps us reduce the data per operation , To improve performance .
Encounter the problem of such a large amount of data , We can solve it through the following ideas :
1. shunt ( The principle is : Try to reduce the cardinality of data for each operation ):
1.1. With and without 、 Commonly used and infrequently used are separated .
1.2. Data stored in the database : Partition 、 sub-treasury 、 table .
1.3. Data stored in files : Open the file .
1.4. Consider batching .
2. Cache technology : Read more write less cache .
3. Database optimization : Reasonably design the database structure 、 Build index reasonably 、 Database cluster .
4. Processing optimization : Optimize Sql、 Consider using temporary tables 、 In the middle of table .
5. The rational use of NoSql:Mongodb、Redis、HBase etc. .
6. Distributed big data processing scheme :Hadoop、Spark、Storm etc. .
Here we mainly need to know how to partition the table , What are the advantages and disadvantages of partitioning and what are the precautions for partitioning . Here we are mysql Give examples .
Let's first look at the advantages and disadvantages of partitions :
1. advantage :
• Perform logical data segmentation , Split data can have multiple physical file paths
Next, let's look at the partition method :
1.RANGE Partition : Given the column value of a continuous interval .
Create it as follows :
-
CREATE
TABLE tbl_users1 (
-
uuid
INT
NOT
NULL,
-
name
VARCHAR(
20),
-
registerTime
VARCHAR(
100)
-
)
-
PARTITION
BY
RANGE (uuid) (
-
PARTITION p0
VALUES LESS THAN (
5),
-
PARTITION p1
VALUES LESS THAN (
10),
-
PARTITION p2
VALUES LESS THAN (
15),
-
PARTITION p3
VALUES LESS THAN MAXVALUE
-
);
2.LIST Partition :LIST It is the column value that matches a certain value in a discrete value set to select .
Create it as follows :
-
CREATE
TABLE tbl_users2 (
-
uuid
INT
NOT
NULL,
-
name
VARCHAR(
20),
-
registerTime
VARCHAR(
100)
-
)
-
-
PARTITION
BY List (uuid) (
-
PARTITION p0
VALUES
in (
1,
2,
3,
5),
-
PARTITION p1
VALUES
in (
7,
9,
10),
-
PARTITION p2
VALUES
in (
11,
15)
-
);
3.HASH Partition : The return value of user-defined expression hash The partition selected after calculation , The expression evaluates with the column values of the rows to be inserted into the table , This function must produce non negative integer values .
Create it as follows :
-
CREATE
TABLE tbl_users4 (
-
uuid
INT
NOT
NULL,
-
name
VARCHAR(
20),
-
registerTime
VARCHAR(
100)
-
)
-
PARTITION
BY HASH (uuid)
/
/uuid You can add expressions , such as
/
2, perhaps
mod(uuid,
2), Low performance , Each data should be calculated before hash Then insert
-
PARTITIONS
3;
4.KEY Partition : It's like pressing HASH Partition , from MySQL The server provides its own hash function .
Create it as follows :
-
CREATE
TABLE tbl_users5 (
-
uuid
INT
NOT
NULL,
-
name
VARCHAR(
20),
-
registerTime
VARCHAR(
100)
-
)
-
PARTITION
BY LINEAR Key (uuid)
-
PARTITIONS
3;
In a later article, we will specifically introduce partition operations , No more details here . When partitioning, we need to pay attention to the following situations :
1. If... Exists in the table primary key perhaps unique key when , Partitioned columns are part of one of two
边栏推荐
- Three series of BOM elements
- Low success rate of unit test report
- 如何理解分布式架构和微服务架构呢
- PLSQL的安装和配置
- Interpolation lookup (two methods)
- Composer change domestic image
- Teach you how to select PCB board by hand (II)
- 数据分析方法论与前人经验总结2【笔记干货】
- [paper reading] icml2020: can autonomous vehicles identify, recover from, and adapt to distribution shifts?
- Merge sort and non comparison sort
猜你喜欢
leetcode135. Distribute candy
Opencv learning note 4 - expansion / corrosion / open operation / close operation
Through the "last mile" of legal services for the masses, fangzheng Puhua labor and personnel law self-service consulting service platform has been frequently "praised"
Input and output of floating point data (C language)
Novice entry SCM must understand those things
为什么要选择云原生数据库
2-3查找树
Input of mathematical formula of obsidan
All about PDF crack, a complete solution to meet all your PDF needs
Train your dataset with swinunet
随机推荐
opencv之图像分割
Greenplum6.x搭建_环境配置
测试踩坑 - 当已有接口(或数据库表中)新增字段时,都需要注意哪些测试点?
归并排序和非比较排序
Golan idea IntelliJ cannot input Chinese characters
基本数据类型和string类型互相转化
PLSQL的安装和配置
Composer change domestic image
String operation
数据中台落地实施之法
Data type - floating point (C language)
Input and output of floating point data (C language)
Exercise arrangement 2.10, 11
Greenplum6.x重新初始化
Xcit learning notes
Low success rate of unit test report
POJ - 3616 Milking Time(DP+LIS)
2-3查找樹
Golang compilation constraint / conditional compilation (/ / +build < tags>)
Greenplum6.x监控软件搭建