当前位置:网站首页>When tidb and Flink are combined: efficient and easy to use real-time data warehouse
When tidb and Flink are combined: efficient and easy to use real-time data warehouse
2020-11-07 20:15:00 【InfoQ】
With the rapid development of Internet , There will be more and more kinds of business , The volume of business data will grow , When it reaches a certain scale , The traditional data storage structure can not meet the needs of enterprises , Real time data warehouse becomes a necessary basic service . In terms of dimension Join For example , Data is stored in a business data source in the form of a normal form table , A lot of Join operation , Reduce performance . If it can be completed in the process of data cleaning and importing Join, Then there is no need to analyze again Join, To improve query performance .
Using real-time data warehouse , Enterprises can achieve real-time OLAP analysis 、 Real time data Kanban 、 Real time business monitoring 、 Real time data interface service, etc . But think of real-time data warehouse , Many people's first impression is that the architecture is complex , Difficult to operate and maintain . And thanks to the new version Flink Yes SQL Support for , as well as TiDB HTAP Characteristics of , We explored an efficient 、 Easy-to-use Flink+TiDB Real time data warehouse solution .
This article will first introduce the concept of real-time data warehouse , Then introduce Flink+TiDB The architecture and advantages of real-time data warehouse , Then we give some user scenarios that are already in use , Finally, it is given in docker-compose In the environment Demo, For readers to try .
The concept of real-time data warehouse
The concept of data warehouse is in 90 Age from Bill Inmon Put forward , It refers to a topic oriented 、 Integrated 、 Relatively stable 、 A collection of historical changes , Used to support management decisions . The data warehouse at that time collected data from data sources through message queues , By calculating daily or weekly for use in reports , Also known as offline data warehouse .
Link to the original text :【https://www.infoq.cn/article/IoD228mbbr7wylDEQKkh】. Without the permission of the author , Prohibited reproduced .
版权声明
本文为[InfoQ]所创,转载请带上原文链接,感谢
边栏推荐
- 工作1-3年的程序员,应该具备怎么样的技术能力?该如何提升?
- C# 枚举权限 |和||,&和&&的区别
- 三步一坑五步一雷,高速成长下的技术团队怎么带?
- 在 Amazon SageMaker 管道模式下使用 Horovod 实现多 GPU 分布式训练
- If you want to forget the WiFi network you used to connect to your Mac, try this!
- 【C++学习笔记】C++ 标准库 std::thread 的简单使用,一文搞定还不简单?
- 留给快手的时间不多了
- HandlerMethodArgumentResolver使用和原理
- 低代码 vs 模型驱动,它们之间到底是什么关系?
- 30岁后,你还剩下什么?
猜你喜欢
随机推荐
Knowledge competition of garbage classification
C language I blog assignment 03
聊一聊数据库中的锁
【涂鸦物联网足迹】物联网主流通信方式
What kind of technical ability should a programmer who has worked for 1-3 years? How to improve?
抽絲剝繭——門面和調停者設計模式
高级并发编程系列九(Lock接口分析)
What should be considered in the promotion plan outside the station?
Kubernetes (1): introduction to kubernetes
vue踩坑:axios使用this指针
当 TiDB 与 Flink 相结合:高效、易用的实时数仓
Talk about sharing before paying
Bgfx compilation tutorial
利用线程通信、解决缓存穿透数据库雪崩
技术总监7年自述——如何选择一家好公司
DOM节点操作
HandlerMethodArgumentResolver使用和原理
How to choose a good company
浅谈HiZ-buffer
【原創】ARM平臺記憶體和cache對xenomai實時性的影響