当前位置:网站首页>Clickhouse物化视图
Clickhouse物化视图
2022-07-05 03:33:00 【Younger成】
一、概念
clickhouse的物化视图是一种查询结果的持久化,给我们带来了查询效率的提升。用户查询效果和查表没有区别,其本质就是一张表,一张时刻在预计算的表,创建的过程用了一个特殊引擎:
注:使用create语法,会创建一个隐藏的目标来保存视图数据,使用To表名,保存到一张显示的表,没有加To表名,表名默认就是.inner.物化视图名,
CREATE MATERIALIZED VIEW [IF NOT EXISTS] [db.]table_name [TO[db.]name] [ENGINE = engine] [POPULATE] AS SELECT …
“查询结果集”的范围很广泛,可以是基础表中部分数据的一份简单拷贝,也可以是多表join之后产生的结果或其子集、或者原始数据的聚合指标等等。所以,物化视图不会随着基础表的变化而变化,所以也称为快照
创建物化视图的限制:
1、必须指定物化视图的engine用于数据存储
2、使用To[db].[table]语法时,不得使用POPULATE,POPULATE会将历史数据全都加载转换过来,数据量大时存在一个不可用时间(官方并不推荐在创建物化视图的时候使用POPULATE,因为在创建物化视图过程中同时写入的数据不能被插入物化视图)
3、查询语句(select)可以包含下面的句子:DISTINCT、GROUP BY、LIMIT等
4、若物化视图的定义使用了 TO [db.]name 子语句,则可以将目标表的视图卸载DETACH 再装载 ATTACH
物化视图和普通视图的区别:
普通视图不保存数据,保存的仅仅是查询语句,查询的时候还是从原表中读取数据,可以将普通视图理解为是个子查询。
物化视图则是 把查询的结果根据相应的引擎存入到磁盘或者内存中,对数据重新进行了组织,生成了一张新表
优缺点:
优点:查询速度快,提前将物化视图规则写好,比直接查询原数据了很多 (物化视图起到了一个同步的作用)
缺点:本质是流式数据 的使用场景,采用累加式的技术,要用到历史数据做去重、去核这样的分析。使用场景有限,如果一张表加了很多物化视图,在写这张表的时候会消耗很多及其的资源,比如数据带宽占满,存储量突增
#创建表
CREATE TABLE `ts_area_info` (
id UInt32 ,
createDate Date ,
userId UInt32 ,
url String,
income UInt8
) ENGINE=MergeTree()
PARTITION BY toYYYYMM(createDate)
ORDER BY (id,createDate,intHash32(userId))
SAMPLE BY intHash32(userId)
SETTINGS index_granularity = 8192
#创建物化视图
CREATE MATERIALIZED VIEW area_mv
ENGINE SummingMergeTree
PARTITION BY toYYYYMM(createDate)
ORDER BY (id,createDate,intHash32(userId))
AS
select * from ts_area_info;
新生成的物化视图:
关键字段:
populate:建表同步数据
final:去最新的数据
二、物化视图作为聚合表
同步数据的时候实现聚合
#使用物化视图同步聚合表
1、创建明细表
drop table tb_order;
create table tb_order(
id UInt8 ,
createDate Date ,
money UInt64
)
ENGINE =MergeTree()
order by id;
2、插入数据
insert into tb_order values(1,toDate(now()),100),
(2,toDate(now()),100),
(3,toDate(now()),100),
(1,toDate(now()),100),
(2,toDate(now()),200),
(3,toDate(now()),300);
3、创建物化视图同步数据
CREATE MATERIALIZED VIEW order_mv
ENGINE AggregatingMergeTree()
PARTITION BY toYYYYMM(createDate)
ORDER BY (id,createDate)
POPULATE AS
select id,createDate,sumState(money) as ms from tb_order
GROUP BY id,createDate;
4、查询物化视图
select id,createDate,sumMerge(ms) from order_mv GROUP BY id,createDate;
5、重新插入查看同步数据
insert into tb_order values(1,toDate(now()),100),(2,toDate(now()),100);
insert into tb_order values(1,toDate('2022-06-29'),100),(2,toDate('2022-06-29'),100);
6、查询订单表
select * from tb_order;
注:
在建表时没有指定主键,会默认使用order by的字段作为主键
边栏推荐
- Why are there fewer and fewer good products produced by big Internet companies such as Tencent and Alibaba?
- Delphi read / write JSON format
- 【软件逆向-分析工具】反汇编和反编译工具
- 【软件逆向-基础知识】分析方法、汇编指令体系结构
- Delphi free memory
- Flex flexible layout
- Multimedia query
- How rem is used
- Leetcode92. reverse linked list II
- Pat class a 1160 forever (class B 1104 forever)
猜你喜欢
v-if VS v-show 2.0
The perfect car for successful people: BMW X7! Superior performance, excellent comfort and safety
Asemi rectifier bridge 2w10 parameters, 2w10 specifications, 2w10 characteristics
Ubantu disk expansion (VMware)
Single box check box
Flume configuration 4 - customize mysqlsource
2021 Li Hongyi machine learning (1): basic concepts
Multi person online anonymous chat room / private chat room source code / support the creation of multiple chat rooms at the same time
2.常见的请求方法
[groovy] loop control (number injection function implements loop | times function | upto function | downto function | step function | closure can be written outside as the final parameter)
随机推荐
001 chip test
The perfect car for successful people: BMW X7! Superior performance, excellent comfort and safety
Usage scenarios and solutions of ledger sharing
[deep learning] deep learning reference materials
Flume配置4——自定义MYSQLSource
Azkaban installation and deployment
The latest blind box mall, which has been repaired very popular these days, has complete open source operation source code
[Yu Yue education] National Open University autumn 2018 8109-22t (1) monetary and banking reference questions
Sqoop command
Use of kubesphere configuration set (configmap)
[groovy] string (string injection function | asBoolean | execute | minus)
Smart pointer shared_ PTR and weak_ Difference of PTR
The perfect car for successful people: BMW X7! Superior performance, excellent comfort and safety
Apache Web page security optimization
[system security] ten thousand words summary system virtualization container bottom layer principle experiment
ICSI213/IECE213 Data Structures
2. Common request methods
Azkaban安装部署
[groovy] loop control (number injection function implements loop | times function | upto function | downto function | step function | closure can be written outside as the final parameter)
[groovy] string (string splicing | multi line string)