Apache inlong million billion level data stream processing

The world's largest Open Source Software Foundation Apache The software foundation officially announced ,Apache InLong Success comes from Apache After graduation, the incubator has become the top project in the community , This was originally donated by Tencent to Apache Community's one-stop massive data integration framework , It can provide big data developers with high-performance processing capacity for data streams of one billion level , And 100 billion level data flow highly reliable services .

Science Popularization ,Apache The software foundation is responsible for all Apache Projects should be strictly screened and incubated in incubators , You can't graduate until you meet a series of quality requirements . Top projects graduated from incubators , Indicates that the community and products of the project have been based on ASF Our elite processes and principles are well managed .

InLong Chinese name “ winged dragon ” It is the divine beast that flows into the sea in Chinese mythology , Implication InLong Value in big data community ecology ： Big data access integration .

InLong With the internal use of Tencent TDBank Based on , Rely on trillions of data acquisition and processing capacity , Collect data 、 polymerization 、 Storage 、 The whole process of sorting data processing is integrated . Not only automatic 、 Security 、 Reliable and high-performance data transmission capability , It also supports stream based data analysis 、 Modeling and Application , Help enterprises simplify the process of data processing .

such as ,InLong As a basis for SaaS Service platform of , Users can select the data publishing and subscription of topics , To report easily and quickly 、 Transfer and distribute data , The use threshold is greatly reduced . meanwhile InLong It can be for 100 Trillions of data streams provide high-performance processing capabilities , by 1000 100 million level data flow to provide highly reliable services , Ensure that users enjoy stable and reliable online services .

InLong Support multiple data access methods , Including different types of message queue service integration and real-time data extraction 、 transformation 、 Loading and rule-based sorting ; In terms of service ,InLong Provide users with unified system monitoring and alarm services . Promote data visualization through fine-grained indicators , Users can directly view the running status of queues in the unified data measurement platform , Greatly improve the initiative of the business .

actually , Inside Tencent InLong Original project name TubeMQ, Focus on high performance 、 Low cost Message Queuing service , And long-term service wechat payment 、 Tencent video 、 Guangdiantong and other products . To further release TubeMQ The surrounding ecological capacity , Tencent upgraded the project to InLong, Focus on building a one-stop massive data integration framework . Project on 2019 year 11 The moon enters Apache The incubator ,2022 year 6 Graduate in June and become a top project .

at present , Apache InLong Has been published for the first time 12 A version , Tencent big data InLong Team and community developers , Constantly update and improve its functions during incubation , such as Manager Metadata management refactoring 、 be based on Flink SQL Of Sort ETL programme 、 Tag based cross region multi cluster and other features , Provide more diversified open solutions for communities and enterprises . meanwhile ,InLong It is widely used in advertising 、 payment 、 social contact 、 game 、 Artificial intelligence and other industries , Provide efficient and convenient services for customers in various fields .

It is worth mentioning that , Before that , Another open source project of Tencent Angel Also from the LF AI Foundation graduate , Become the world's top AI Open source project . As the most comprehensive open source company in the field of big data in China ,Apache InLong Graduate into Apache Top projects , It has not only created another milestone created by Tencent big data team in the open source field , It will also further consolidate Tencent's leading position in the big data field .