Interview assault 63: how to remove duplication in MySQL?

stay MySQL in , There are two most common methods of weight removal ： Use distinct Or use group by, What's the difference between them ？ Let's take a look at .

1. Create test data

--  Create test table 
drop table if exists pageview;
create table pageview(
    id bigint primary key auto_increment comment ' Since the primary key ',
    aid bigint not null comment ' article ID',
    uid bigint not null comment '（ visit ） user ID',
    createtime datetime default now() comment ' Creation time '
) default charset='utf8mb4';
--  Add test data 
insert into pageview(aid,uid) values(1,1);
insert into pageview(aid,uid) values(1,1);
insert into pageview(aid,uid) values(2,1);
insert into pageview(aid,uid) values(2,2);

The final display effect is as follows ：

2.distinct Use

distinct The basic grammar is as follows ：

SELECT DISTINCT column_name,column_name FROM table_name;

2.1 Separate the heavy ones

We use first distinct Realize single column weight removal , according to aid（ article ID） duplicate removal , The specific implementation is as follows ：

2.2 More than one, more than one

In addition to single train weight removal ,distinct It also supports multiple columns （ Two or more trains ） duplicate removal , We according to the aid（ article ID） and uid（ user ID） Combined weightlessness , The specific implementation is as follows ：

2.3 Aggregate functions + duplicate removal

Use distinct + Aggregate function de duplication , Calculation aid Total number of strips after weight removal , The specific implementation is as follows ：

3.group by Use

group by The basic grammar is as follows ：

SELECT column_name,column_name FROM table_name 
WHERE column_name operator value 
GROUP BY column_name

3.1 Separate the heavy ones

according to aid（ article ID） duplicate removal , The specific implementation is as follows ： And distinct comparison group by More columns can be displayed , and distinct Only the de duplicated columns can be displayed .

3.2 More than one, more than one

according to aid（ article ID） and uid（ user ID） Combined weightlessness , The specific implementation is as follows ：

3.3 Aggregate functions + group by

Count each one aid Total quantity ,SQL The implementation is as follows ： As can be seen from the above results , Use group by and distinct Add count The query semantics of is completely different ,distinct + count It counts the total quantity after weight removal , and group by + count Statistics are the total number of each group of data after grouping .

4.distinct and group by The difference between

Official documents describe distinct When it comes to ： in the majority of cases distinct It's special group by, As shown in the figure below ： Official document address ：https://dev.mysql.com/doc/refman/8.0/en/distinct-optimization.html But there are still some subtle differences between the two , For example, the following .

difference 1： The query result set is different

When using distinct When you go to heavy duty , In the query result set, only the de duplication information , As shown in the figure below ： When you try to add a non de duplication field （ Inquire about ） when ,SQL An error will be reported, as shown in the figure below ： While using group by Sorting can query one or more fields , As shown in the figure below ：

difference 2： Different business scenarios

To count the total quantity after weight removal, you need to use distinct, And statistical grouping details , Or when adding query criteria on the basis of grouping details , You have to use group by 了 . Use distinct Count the total quantity of a column after weight removal ： The number after statistical grouping is greater than 2 The article , Then use group by 了 , As shown in the figure below ：

difference 3： Different performance

If the de duplicated field has an index , that group by and distinct You can use indexes , In this case, their performance is the same ; and When the de duplicated field has no index ,distinct Performance will be higher than group by, Because in MySQL 8.0 Before ,group by There is a hidden function that will sort by default , This will trigger filesort This leads to reduced query performance .

summary

In most scenes distinct It's special group by, But there are subtle differences between the two , For example, they are on the query result set 、 Specific business scenarios used , And the performance is different .

Reference resources & Acknowledgement

zhuanlan.zhihu.com/p/384840662

It's up to you to judge right and wrong , Disdain is to listen to people , Gain or loss is more important than number .

official account ：Java Analysis of the real interview questions

Interview collection ：https://gitee.com/mydb/interview