Rare earth Developer Conference ｜ Apache pulsar committee Liu Dezhi shares the way of cloud native technology transformation

7 month 22 Japan —23 Two days a day ,2022 The second rare earth developers' conference is coming , This session of the general assembly uses 「 Stimulate universal gravitation , The future of nuggets Technology 」 The theme of , We will take a look at the origin of technology from the practice driven case summary .

Apache Pulsar Committer、 StreamNative Liudezhi, a solution expert, was invited to participate in this conference to share .

Liu Dezhi ： Build a real-time data warehouse based on cloud native message system

stay 《 Cloud native technology architecture changes 》 Special field , Liu Dezhi's meeting is based on Apache Pulsar Build a real-time data warehouse and give specific technical details .

Liu Dezhi  

Apache Pulsar Committer,StreamNative Solution experts , Former technical expert of Tencent billing platform . Once worked for Tencent , Responsible for the implementation of Tencent billing platform architecture and technical scheme , And dominated Tencent cloud TDMQ for Pulsar Product implementation , Rich experience in message oriented middleware development, operation and maintenance . Active in the open source project of the latest generation of cloud native message middleware Apache Pulsar Community , Have unique insight and thinking about message and flow systems 、 And rich industry practice .

Overview of the speech

Apache Pulsar Natural clouds are original , Adopt the architecture of separating storage and Computing . It can not only support a lot of MQ Key business scenarios , Don't lose the news , Provide strong consistency assurance , It can also provide high bandwidth in streaming scenarios 、 Low latency .Apache Flink Unify batch stream processing into the computing engine , Provides a consistent programming interface . It also supports batch flow , bring Pulsar and Flink Inborn is a fitting partner .Pulsar Flink Connector take Pulsar And Flink Integrate the functions of , Created a unified data architecture , It is the best solution for batch flow .

The real-time data warehouse architecture based on the data Lake realizes the flow batch unification of the storage layer , It is favored by developers and users with higher data accuracy and lower cost . As more and more users seek higher performance real-time data warehouse solutions ,Pulsar Storage computing separation architecture 、 High consistency 、 Transaction support and other features supporting the data lake have attracted the attention of developers , Yes Pulsar The demand for building real-time data warehouses is endless . In the near future , Multiple Pulsar Data Lake Connector One after another , The community is open source Pulsar-Hudi、Pulsar-Iceberg、Pulsar-Delta Lake Connector. This sharing will be combined with Pular-Flink 、Pulsar Big data ecosystem with data Lake , And show relevant cases , Share with you the latest solution of real-time data warehouse .

Poke 「 Read the original 」 Register for the conference