当前位置:网站首页>Observation cloud and tdengine have reached in-depth cooperation to optimize the cloud experience of enterprises
Observation cloud and tdengine have reached in-depth cooperation to optimize the cloud experience of enterprises
2022-07-05 09:44:00 【Tdengine】
In recent years , Major manufacturers have embraced cloud native technology , The huge amount of data brought by this has brought a huge impact on the traditional technical architecture , Traditional monitoring bears the brunt . After the service goes to the cloud , Mostly based on Docker Container deployment 、Kubernetes Service governance , Resources are flexible and change in real time , Traditional monitoring is difficult to support the business requirements in this scenario , Observability (Observability) It has gradually developed into a hot direction in the cloud native field .
In this context , The system observable platform in the cloud era observes the cloud and the big data solution of the Internet of things. Shang Taosi data has reached in-depth strategic cooperation , Rely on domestic production Time series database TDengine Technical and performance advantages , Observing clouds will further satisfy the cloud 、 Cloud native 、 Application and business monitoring requirements , From the infrastructure to the log data to the full link application performance, we can achieve comprehensive and active observation , Let the enterprise experience the cloud more smoothly .
Multiple storage hybrid lookups , Observation cloud design DQL The engine focuses on query
In the data storage structure of observation cloud , Data acquisition software will be deployed to different user environments to collect required data , After data uploading, the system will classify the data according to the data type , After the classification is completed, all the data will pass through Worker Corresponding processing , According to the corresponding classification, it is written into various storage . After data storage , Users need to query data according to business requirements , But different data stores have different query languages , How to make up for language differences and complete query operations has become an urgent problem to be solved .
The storage used to observe the cloud can be summarized into three types , One is like TDengine General time series database (Time-Series Database), One is Elasticsearch( abbreviation ES), In addition, some data is stored in Redis、MySQL This kind of relational database , Corresponding to so many kinds of storage , Extremely unfriendly in traversing queries , Especially for the front end . The solution to observing the cloud is to redefine a query language that can query multiple data types , And from this design DQL engine , In a real sense, it realizes the vertical division of data query level .
In order to create a query language that can uniformly find multiple storage mixtures , The observation cloud has been deeply considered in its design , And rely on the concrete realization to polish constantly , Designed a query focused and simple syntax DQL engine , The code is shown as follows :
M(Metric)::nsq_nodes:(LAST(message_count) AS The number of messages ) BY
server_host
L(Log)::openway_gin:(MAX(cost_time)) {host = ‘prd-dataway’}
BY http_url
R(RUL)::view:(COUNT_DISTINCT(userid)) { app_id =
'appid_xyz' and view_path = re('.*/scene/.*') }
After solving the problem of multiple storage coexistence , To meet the needs of business development , Further improve the front-end operating experience , Observation cloud decides to upgrade the existing storage system architecture .
And TDengine cooperation , promote User experience
The time series database used to observe the cloud is InfluxDB, Applied so far , It is increasingly difficult to support privatized deployment businesses , Multiple problems and bottlenecks limit business development , Finally, the observation cloud decided to InfluxDB Replace . They turned their attention to the domestic database field , Found in years of development , Some high-quality domestic time series databases have emerged ,TDengine Is one of them . In understanding TDengine After the various characteristics of , Observation cloud tests two time series databases .
According to the above test results , Observe the cloud and find , Whether from writing 、 Query or storage ,TDengine In terms of performance, it can be said that it is comprehensively ahead of InfluxDB. Whether it's a simple query or a complex aggregate query , There are 10x~20x Performance improvement of , Storage space can be saved by about half , There is also a twofold improvement in write performance , In other words, two or three nodes may be used to meet the user's write requirements , application TDengine The latter node is about to resist , To some extent, it also saves part of the deployment cost .
at present , Observational clouds are gradually shifting data from InfluxDB Migrate to TDengine,TDengine Will be applied to multi tenant isolation 、 high frequency I/O、 In scenarios such as privatization deployment .
As a SaaS platform , Observation cloud needs to be connected to multi tenant for corresponding services , Therefore, the platform side should first consider how different tenants have different DB/Index In isolation . stay TDengine in , Multiple DB Can share one database service , Observing clouds can successfully implement this scheme .
Besides , Multi tenancy mode also generates high-frequency data writing , Each tenant may deploy multiple collection points , A collection point is equivalent to a writing client , For one tenant only , There may be ten 、 A hundred or more access clients , It is conceivable that the write volume of multi tenancy is huge .
Except for writing , The platform will also face considerable pressure at the query level , Generation of front-end indicators 、 Monitoring is always searching for data .TDengine Its logic design makes it possible to meet the requirements of high reliability , It can also meet the reading and writing requirements of big data , It can well support the system performance under the multi tenant mode .
except SaaS Out of service , Observation cloud will also be privatized and deployed according to the business needs of some customers , Privatize the deployment environment and SaaS Make a big difference , Not only to improve the deployment efficiency , You should also be able to connect with different cloud platforms , More convenient access to needed resources . Compared with InfluxDB, Domestic time series database TDengine It is obviously more friendly to privatized deployment .
Conclusion
stay TDengine With the help of , Observing the cloud greatly saves deployment and operation and maintenance costs , It also significantly improves the overall performance of the system , Let front-end users improve their experience . As TDengine Partners of , The observation cloud hopes TDengine In the future, there will be more support for mathematical functions , At the same time, strengthen some query functions to Unicode Support for , Promote the two sides to carry out more in-depth cooperation mode with richer functions .
“ Behind us SaaS And privatization deployment will use by default TDengine, Other public clouds (AWS、 Tencent, cloud, etc. ) All services on will be in the form of TDengine Mainly , For new users, we will also slowly migrate to TDengine node .”
In the future, with the continuous strengthening of bilateral cooperation , Predictably, , The technology support of the combination of strong and strong will build a more solid foundation for the intelligent monitoring of the cloud native industry “ Technology base ”, Provide technical support for enterprise digital transformation , Contribute a steady stream of creativity to the development of observability technology in China .
Introduction to observation cloud : Observation cloud , new generation SaaS Full link data observable platform , Realize unified collection 、 Uniform label 、 Unified storage and unified interface , Bring a fully functional, integrated and observable experience . Observation cloud energy full environment high base data collection , Support multi-dimensional information intelligent retrieval and analysis , And provide powerful user-defined programmability , Keep the system running under control , The root cause of the fault has nothing to hide . A smart team will observe , Observability uses observational clouds .
Introduction to Taosi data : Beijing Taosi Data Technology Co., Ltd (TAOS Data) Aim at the growing Internet of things data market , Focus on the storage of big data in time series space 、 Inquire about 、 Analysis and calculation , Do not rely on any open source or third-party software , Developed with independent intellectual property rights 、100% Autonomous and controllable high performance 、 Distributed 、 Support SQL Time series database TDengine. use AGPL license , Taosi data has TDengine The kernel of ( Storage 、 Computing engines and clusters )100% Open source , In the future, we will try our best to build a developer community , Maintain an open and open source business model .
Want to know more TDengine Database Specific details of , Welcome to GitHub View the relevant source code on .
边栏推荐
- LeetCode 556. Next bigger element III
- 一次 Keepalived 高可用的事故,让我重学了一遍它
- LeetCode 496. Next larger element I
- Gradientdrawable get a single color
- 测试老鸟浅谈unittest和pytest的区别
- Unity skframework framework (24), avatar controller third person control
- 顶会论文看图对比学习(GNN+CL)研究趋势
- [hungry dynamic table]
- 卷起来,突破35岁焦虑,动画演示CPU记录函数调用过程
- Kotlin introductory notes (VI) interface and function visibility modifiers
猜你喜欢
TDengine 已经支持工业英特尔 边缘洞见软件包
oracle 多行数据合并成一行数据
The popularity of B2B2C continues to rise. What are the benefits of enterprises doing multi-user mall system?
OpenGL - Coordinate Systems
Vs code problem: the length of long lines can be configured through "editor.maxtokenizationlinelength"
Oracle combines multiple rows of data into one row of data
Principle and performance analysis of lepton lossless compression
Why does everyone want to do e-commerce? How much do you know about the advantages of online shopping malls?
【ManageEngine】如何利用好OpManager的报表功能
Nips2021 | new SOTA for node classification beyond graphcl, gnn+ comparative learning
随机推荐
What should we pay attention to when entering the community e-commerce business?
【阅读笔记】图对比学习 GNN+CL
Wechat applet obtains household area information
MySQL does not take effect in sorting string types
[team PK competition] the task of this week has been opened | question answering challenge to consolidate the knowledge of commodity details
Oracle combines multiple rows of data into one row of data
TDengine可通过数据同步工具 DataX读写
OpenGL - Coordinate Systems
Tdengine offline upgrade process
Using request headers to develop multi terminal applications
卷起来,突破35岁焦虑,动画演示CPU记录函数调用过程
What should we pay attention to when developing B2C websites?
H.265编码原理入门
Kotlin introductory notes (VIII) collection and traversal
OpenGL - Lighting
Unity skframework framework (XXIII), minimap small map tool
高性能Spark_transformation性能
Community group buying has triggered heated discussion. How does this model work?
MySQL installation configuration and creation of databases and tables
Dry goods sorting! How about the development trend of ERP in the manufacturing industry? It's enough to read this article