Real time data warehouse: meituan's implementation of real-time data warehouse construction based on Flink

---

brief introduction

In recent years, , The demand of enterprises for real-time data service is increasing . This paper summarizes the performance characteristics and application scenarios of common real-time data components , This paper introduces how meituan passed Flink The engine builds a real-time data warehouse , So as to provide high efficiency 、 Robust real time data services .

One 、 The initial architecture of real-time platform

In the early days of real-time data system construction , Due to less demand for real-time data , Can't form a complete data system . We're going to use “ All the way to the end ” Development model ： By deploying on a real-time computing platform Storm Jobs process real-time data queues to extract data metrics , Push directly to real-time application services .

however , As products and business personnel