当前位置:网站首页>Clickhouse materialized view
Clickhouse materialized view
2022-07-05 03:45:00 【Younger Cheng】
One 、 Concept
clickhouse Materialized view of is a kind of persistence of query results , It has brought us the improvement of query efficiency . There is no difference between user query and table lookup , Its essence is a table , A table that is always pre calculated , The creation process uses a special engine :
notes : Use create grammar , A hidden target will be created to save the view data , Use To Table name , Save to a displayed table , No addition To Table name , The default table name is .inner. Materialized view name ,
CREATE MATERIALIZED VIEW [IF NOT EXISTS] [db.]table_name [TO[db.]name] [ENGINE = engine] [POPULATE] AS SELECT …
“ Query result set ” Has a wide range of , It can be a simple copy of some data in the basic table , It can also be multi table join The result or a subset of it 、 Or the aggregation index of the original data . therefore , The materialized view will not change with the change of the underlying table , So it is also called snapshot
Restrictions on creating materialized views :
1、 The of materialized views must be specified engine For data storage
2、 Use To[db].[table] Grammatical time , Do not use POPULATE,POPULATE All historical data will be loaded and converted , There is an unavailable time when there is a large amount of data ( It is not officially recommended to use when creating materialized views POPULATE, Because the data written in the process of creating materialized views cannot be inserted into materialized views )
3、 Query statement (select) It can contain the following sentences :DISTINCT、GROUP BY、LIMIT etc.
4、 If the definition of materialized view uses TO [db.]name Sub statement , You can uninstall the view of the target table DETACH Reload ATTACH
The difference between materialized view and ordinary view :
Normal view doesn't save data , Only query statements are saved , When querying, you still read data from the original table , You can think of a normal view as a subquery .
Materialized views are Store the query results into disk or memory according to the corresponding engine , Reorganize the data , A new table is generated
Advantages and disadvantages :
advantage : Fast query speed , Write materialized view rules in advance , It is much better than directly querying the original data ( Materialized views play a synchronous role )
shortcoming : The essence is streaming data Usage scenarios of , Adopt cumulative Technology , Historical data should be used for de duplication 、 Analysis of denuclearization . Limited use scenarios , If you add many materialized views to a table , Writing this table will consume a lot of resources , For example, the data bandwidth is full , Sudden increase in storage
# Create table
CREATE TABLE `ts_area_info` (
id UInt32 ,
createDate Date ,
userId UInt32 ,
url String,
income UInt8
) ENGINE=MergeTree()
PARTITION BY toYYYYMM(createDate)
ORDER BY (id,createDate,intHash32(userId))
SAMPLE BY intHash32(userId)
SETTINGS index_granularity = 8192
# Create materialized views
CREATE MATERIALIZED VIEW area_mv
ENGINE SummingMergeTree
PARTITION BY toYYYYMM(createDate)
ORDER BY (id,createDate,intHash32(userId))
AS
select * from ts_area_info;
Newly generated materialized view :
Key fields :
populate: Create a table to synchronize data
final: Go to the latest data
Two 、 Materialized views act as aggregate tables
Realize aggregation when synchronizing data
# Use materialized views to synchronize aggregate tables
1、 Create a schedule
drop table tb_order;
create table tb_order(
id UInt8 ,
createDate Date ,
money UInt64
)
ENGINE =MergeTree()
order by id;
2、 insert data
insert into tb_order values(1,toDate(now()),100),
(2,toDate(now()),100),
(3,toDate(now()),100),
(1,toDate(now()),100),
(2,toDate(now()),200),
(3,toDate(now()),300);
3、 Create materialized views to synchronize data
CREATE MATERIALIZED VIEW order_mv
ENGINE AggregatingMergeTree()
PARTITION BY toYYYYMM(createDate)
ORDER BY (id,createDate)
POPULATE AS
select id,createDate,sumState(money) as ms from tb_order
GROUP BY id,createDate;
4、 Query materialized view
select id,createDate,sumMerge(ms) from order_mv GROUP BY id,createDate;
5、 Reinsert to view synchronized data
insert into tb_order values(1,toDate(now()),100),(2,toDate(now()),100);
insert into tb_order values(1,toDate('2022-06-29'),100),(2,toDate('2022-06-29'),100);
6、 Query order table
select * from tb_order;
notes :
The primary key is not specified when creating the table , Will be used by default order by Use the field of as the primary key
边栏推荐
- MySQL winter vacation self-study 2022 11 (9)
- [positioning in JS]
- 51 independent key basic experiment
- Kuboard
- Talk about the SQL server version of DTM sub transaction barrier function
- Solve the problem that sqlyog does not have a schema Designer
- Yuancosmic ecological panorama [2022 latest]
- PlasticSCM 企业版Crack
- Accuracy problem and solution of BigDecimal
- C # use awaiter
猜你喜欢
Multimedia query
Learning notes of raspberry pie 4B - IO communication (I2C)
Talk about the SQL server version of DTM sub transaction barrier function
[untitled]
花了2晚,拿到了吴恩达@斯坦福大学的机器学习课程证书
[untitled]
UE4 DMX和grandMA2 onPC 3.1.2.5的操作流程
[luat-air105] 4.1 file system FS
Pdf things
Share the newly released web application development framework based on blazor Technology
随机推荐
【web審計-源碼泄露】獲取源碼方法,利用工具
[software reverse analysis tool] disassembly and decompilation tool
Is there any way to change the height of the uinavigationbar in the storyboard without using the UINavigationController?
Redis source code analysis: redis cluster
NPM introduction link symbolic link
Solve the problem that sqlyog does not have a schema Designer
【软件逆向-分析工具】反汇编和反编译工具
特殊版:SpreadJS v15.1 VS SpreadJS v15.0
[Chongqing Guangdong education] 2777t green space planning reference questions of National Open University in autumn 2018
Machine learning experiment report 1 - linear model, decision tree, neural network part
040. (2.9) relieved
Thread Basics
Web components series (VII) -- life cycle of custom components
Logstash、Fluentd、Fluent Bit、Vector? How to choose the appropriate open source log collector
[wp]bmzclub几道题的writeup
天干地支纪年法中为什么是60年一个轮回,而不是120年
Zero foundation uses paddlepaddle to build lenet-5 network
程序员的视力怎么样? | 每日趣闻
SQL performance optimization skills
v-if VS v-show 2.0