Advanced SQL skills CTE and recursive query

19.1　 Common table expression

from MySQL 8.x Public table expressions are supported in version （ Referred to as CTE）. The common table expression passes WITH Statements for , It can be divided into non recursive common table expressions and recursive common table expressions .

In regular subqueries , Derived tables cannot be referenced twice , Otherwise it will cause MySQL Performance problems of . If you use CTE For enquiries , Subqueries can only be referenced once , This is also used CTE An important reason for .

19.1.1　 Non recursive CTE

MySQL 8.0 Before , Want to perform complex queries on data tables , With the help of sub query statements , but SQL Poor performance of statements , And the derived table of the subquery cannot be referenced multiple times .CTE The emergence of greatly simplifies the complexity SQL Compiling , Improve the performance of data query .

Non recursive CTE The syntax of is as follows ：

---

WITH        cte_name [(col_name [, col_name] ...)] AS (subquery)        [, cte_name [(col_name [, col_name] ...)] AS (subquery)] …SELECT [(col_name [, col_name] ...)] FROM cte_name;

---

You can compare subqueries with CTE To deepen the query on CTE The understanding of the . for example , stay MySQL Execute the following in the command line SQL Statement to achieve the effect of subquery .

---

mysql> SELECT * FROM      -> (SELECT YEAR(NOW())) AS year;+-------------+| YEAR(NOW()) |+-------------+|        2020 |+-------------+1 row in set (0.00 sec)

---

above SQL Statement uses subquery to obtain the information of the current year .

Use CTE The effect of query is as follows ：

---

mysql> WITH year AS    -> (SELECT YEAR(NOW()))    -> SELECT * FROM year;+-------------+| YEAR(NOW()) |+-------------+|        2020 |+-------------+1 row in set (0.00 sec)

---

Through two kinds of queries SQL Sentence comparison shows , Use CTE Queries enable SQL Semantic clarity . It can also be in CTE Multiple query fields are defined in the statement , for example ：

---

mysql> WITH cte_year_month (year, month) AS    -> (SELECT YEAR(NOW()) AS year, MONTH(NOW()) AS month)    -> SELECT * FROM cte_year_month;+------+-------+| year | month |+------+-------+| 2020 |     1 |+------+-------+1 row in set (0.00 sec)

---

CTE You can reuse the last query result , Multiple CTE They can also be quoted from each other , for example ：

---

mysql> WITH cte1(cte1_year, cte1_month) AS    -> (SELECT YEAR(NOW()) AS cte1_year, MONTH(NOW()) AS cte1_month),    -> cte2(cte2_year, cte2_month) AS    -> (SELECT (cte1_year+1) AS cte2_year, (cte1_month + 1) AS cte2_month FROM cte1)     -> SELECT * FROM cte1 JOIN cte2;+-----------+------------+-----------+------------+| cte1_year | cte1_month | cte2_year | cte2_month |+-----------+------------+-----------+------------+|      2020 |          1 |      2021 |          2 |+-----------+------------+-----------+------------+1 row in set (0.00 sec)

---

above SQL In the sentence , stay cte2 In the definition of cte1.

Be careful ： stay SQL Multiple... Are defined in the statement CTE when , Every CTE You need to separate them with commas .

19.1.2　 recursive CTE

recursive CTE The subquery of can reference itself , recursive CTE The syntax format of is better than non recursive CTE The syntax format of has one more keyword RECURSIVE.

---

WITH RECURSIVE        cte_name [(col_name [, col_name] ...)] AS (subquery)        [, cte_name [(col_name [, col_name] ...)] AS (subquery)] …SELECT [(col_name [, col_name] ...)] FROM cte_name;

---

In recursion CTE in , Subqueries contain two ： One is seed query （ Seed query initializes the query data , And will not reference itself in the query ）, One is recursive query （ Recursive query is based on seed query , Reference your own query according to certain rules ）. Between these two queries, we will pass UNION、UNION ALL perhaps UNION DISTINCT Connect statements .

for example , Using recursion CTE stay MySQL Output... On the command line 1～10 Sequence .

---

mysql> WITH RECURSIVE cte_num(num) AS    -> (    -> SELECT 1    -> UNION ALL    -> SELECT num + 1 FROM cte_num WHERE num < 10     -> )    -> SELECT * FROM cte_num;+------+| num  |+------+|    1 ||    2 ||    3 ||    4 ||    5 ||    6 ||    7 ||    8 ||    9 ||   10 |+------+10 rows in set (0.00 sec)

---

recursive CTE Queries are organized for traversal 、 It is very convenient when there is hierarchical data . for example , Create a regional data table t_area, The data table contains the information of provinces and cities .

---

mysql> CREATE TABLE t_area(    -> id INT NOT NULL,    -> name VARCHAR(30),    -> pid INT    -> );Query OK, 0 rows affected (0.01 sec)

---

towards t_area Insert test data into the data table .

---

mysql> INSERT INTO t_area    -> (id, name, pid)    -> VALUES    -> (1, ' Sichuan Province ', NULL),    -> (2, ' Chengdu ', 1),    -> (3, ' Jinjiang District ', 2),    -> (4, ' Wuhou District ', 2),    -> (5, ' Hebei Province ', NULL),    -> (6, ' Langfang City ', 5),    -> (7, ' Anzi District ', 6);Query OK, 7 rows affected (0.01 sec)Records: 7  Duplicates: 0  Warnings: 0

---

SQL Statement executed successfully , Inquire about t_area Data in the data table .

---

mysql> SELECT * FROM t_area;+----+-----------+------+| id | name      | pid  |+----+-----------+------+|  1 |  Sichuan Province     | NULL ||  2 |  Chengdu     |    1 ||  3 |  Jinjiang District     |    2 ||  4 |  Wuhou District     |    2 ||  5 |  Hebei Province     | NULL ||  6 |  Langfang City     |    5 ||  7 |  Anzi District     |    6 |+----+-----------+------+7 rows in set (0.00 sec)

---

Next , Using recursion CTE Inquire about t_area Hierarchical relationships in data tables .

---

mysql> WITH RECURSIVE area_depth(id, name, path) AS    -> (    -> SELECT id, name, CAST(id AS CHAR(300))    -> FROM t_area WHERE pid IS NULL    -> UNION ALL    -> SELECT a.id, a.name, CONCAT(ad.path, ',', a.id)    -> FROM area_depth AS ad     -> JOIN t_area AS a    -> ON ad.id = a.pid    -> )    -> SELECT * FROM area_depth ORDER BY path;+------+-----------+-------+| id   | name      | path  |+------+-----------+-------+|    1 |  Sichuan Province     | 1     ||    2 |  Chengdu     | 1,2   ||    3 |  Jinjiang District     | 1,2,3 ||    4 |  Wuhou District     | 1,2,4 ||    5 |  Hebei Province     | 5     ||    6 |  Langfang City     | 5,6   ||    7 |  Anzi District     | 5,6,7 |+------+-----------+-------+7 rows in set (0.00 sec)

---

among ,path The column represents the hierarchical relationship of each data queried .

19.1.3　 recursive CTE The limitation of

recursive CTE The query statement of needs to include a condition to terminate the recursive query . When recursive for some reason CTE When the termination condition is not set in the query statement of ,MySQL According to the corresponding configuration information , Automatically terminate the query and throw the corresponding error message . stay MySQL The following two configuration items are provided by default to terminate recursion CTE.

·cte_max_recursion_depth： If you are defining recursion CTE Recursive termination condition is not set , When reach cte_max_recursion_depth After the execution times of parameter settings ,MySQL Will report a mistake .

·max_execution_time： Express SQL The maximum milliseconds of statement execution , When SQL When the execution time of the statement exceeds the value set by this parameter ,MySQL Report errors .

for example , The following recursion without query termination conditions CTE,MySQL Will throw an error message and terminate the query .

---

mysql> WITH RECURSIVE cte_num(num) AS    -> (    -> SELECT 1    -> UNION ALL    -> SELECT n+1 FROM cte_num    -> )    -> SELECT * FROM cte_num;ERROR 3636 (HY000): Recursive query aborted after 1001 iterations. Try increasing @@cte_max_recursion_depth to a larger value.

---

As can be seen from the output , When not for recursion CTE When setting the termination condition ,MySQL By default, it will be in 1001 An error message is thrown during this query , And terminate the query .

see cte_max_recursion_depth The default value of the parameter .

---

mysql> SHOW VARIABLES LIKE 'cte_max%';+-------------------------+-------+| Variable_name           | Value |+-------------------------+-------+| cte_max_recursion_depth | 1000  |+-------------------------+-------+1 row in set (0.03 sec)

---

Results show ,cte_max_recursion_depth The default value of the parameter is 1000, This is also MySQL By default, it will be in 1001 The reason why an error is thrown and the query is terminated during this query .

Next , verification MySQL How is it based on max_execution_time Configuration item terminates recursion CTE. First , To demonstrate max_execution_time Parameter limit , Need to put cte_max_recursion_depth The parameter is set to a large number , Here it is MySQL Session level settings .

---

mysql> SET SESSION cte_max_recursion_depth=999999999;Query OK, 0 rows affected (0.00 sec)mysql> SHOW VARIABLES LIKE 'cte_max%';+-------------------------+-----------+| Variable_name           | Value     |+-------------------------+-----------+| cte_max_recursion_depth | 999999999 |+-------------------------+-----------+1 row in set (0.00 sec)

---

Has succeeded in bringing cte_max_recursion_depth Parameter set to 999999999.

see MySQL in max_execution_time The default value of the parameter .

---

mysql> SHOW VARIABLES LIKE 'max_execution%';+--------------------+-------+| Variable_name      | Value |+--------------------+-------+| max_execution_time | 0     |+--------------------+-------+1 row in set (0.00 sec)

---

stay MySQL in max_execution_time The value of the parameter is the millisecond value , The default is 0, That is, there is no limit . here , stay MySQL The session level will max_execution_time Is set to 1s.

---

mysql> SET SESSION max_execution_time=1000;                       Query OK, 0 rows affected (0.00 sec)mysql> SHOW VARIABLES LIKE 'max_execution%';+--------------------+-------+| Variable_name      | Value |+--------------------+-------+| max_execution_time | 1000  |+--------------------+-------+1 row in set (0.01 sec)

---

Has succeeded in bringing max_execution_time Is set to 1s.

When SQL The execution time of the statement exceeds max_execution_time When setting the value of ,MySQL Report errors .

---

mysql> WITH RECURSIVE cte(n) AS    -> (    -> SELECT 1    -> UNION ALL    -> SELECT n+1 FROM CTE    -> )    -> SELECT * FROM cte;ERROR 3024 (HY000): Query execution was interrupted, maximum statement execution time exceeded

---

MySQL The default mechanism for terminating recursion （cte_max_recursion_depth and max_execution_time Configuration item ）, It effectively prevents the problem of infinite recursion .

Be careful ： although MySQL By default, a mechanism for terminating recursion is provided , But in use MySQL Recursion CTE when , Suggestions are still based on actual needs , in CTE Of SQL The condition of recursive termination is clearly set in the statement . in addition ,CTE Support SELECT/INSERT/UPDATE/DELETE Such statements , This is just a demonstration of SELECT sentence , Readers of other sentences can realize it by themselves , I won't repeat .