Real time data warehouse: Netease strictly selects the practice of real-time data warehouse based on Flink

---

1. background

Strictly select the real-time data warehouse project from 17 Started in the second half of the year , The background is summarized in three aspects ：
The first is long link and fast changing services , Yan Xuan is a ODM Online retailers , The whole business chain is from commodity procurement 、 production 、 Warehouse 、 At this stage of sales, you can go to the main station APP Buy on or branch factory , Then it will be delivered to consumers through merchants . Chain degree is very long , This also determines that the data domain is very wide ; Yan Xuan is a growing e-commerce , There will be many new businesses .

The second is the increasing demand for real-time data , At present, more real-time data is needed to make business decisions , We need to adjust the resource level according to the sales situation ; At the same time, some activities also need real-time data to enhance the interaction with users . If there are real-time and offline data solutions , Give priority to real-time , If you can't realize it in real time, consider offline .

The third is the increasingly high data quality requirements , Because data will directly affect business decisions , Affect the effect of online operation activities , Therefore, the requirements for data quality are getting higher and higher .

In view of this project background, three design objectives are proposed , The first is flexible and extensible , The second is high development efficiency , The third is high data quality .

2. Overall design and