Understand the window query function of tdengine in one article

As a Time series database （Time-Series Database）,TDengine  It provides the ability to automatically divide windows by time and execute queries . Do you know the purpose and usage scenario of this function ？ This article will introduce you to this function and its typical usage scenarios .

What is window query ？

Time series data often need to be queried according to the collection time , In essence, it is to divide the time window on the time axis , And aggregate and query the data in the window . For example, the simplest scenario ： Inquire about ”2018-01-01 00:00:00.000″ To ”2018-01-31 00:00:00.000″ Maximum value of raw data 、 minimum value 、 Average , be ”2018-01-01 00:00:00.000″ To ”2018-01-31 00:00:00.000″ It forms a query time window .

In practical production application , The query conditions required by business scenarios are often more complex than this , Time series database is required to be able to follow different rules 、 Divide windows along the time axis , And the time series data in each window are aggregated separately 、 Select operations such as calculation .

TDengine Detailed explanation of the three window query capabilities provided

TDengine from 2.2.x.x Since version , It supports three types of sequential window queries , Namely Equispaced window （interval）、 Status window （state_window） and Session window （session）, It greatly simplifies the processing of time series logic in application development .

Window query , The query syntax from the sub table is ：

SELECT function_list FROM tb_name [WHERE where_condition] [SESSION(ts_col, tol_val)][STATE_WINDOW(col)] [INTERVAL(interval [, offset]) [SLIDING sliding]] [FILL({NONE | VALUE | PREV | NULL | LINEAR | NEXT})]

Window query , The query syntax from the super table is ：

SELECT function_list FROM stb_name [WHERE where_condition] [INTERVAL(interval [, offset]) [SLIDING sliding]] [FILL({NONE | VALUE | PREV | NULL | LINEAR | NEXT})] [GROUP BY tags]

Let's take a look at these three types of time window query functions .

1. Equispaced window （interval）

According to the fixed time window length , Periodically window the timeline , It also allows aggregate queries on the data in each window .

  Let's take a look at the specific application scenarios of equispaced windows through a few examples .

In industrial production , There are many monitoring indicators that need to display the trend line in real time . But some indicators such as vibration , The actual acquisition frequency is very high , If you show the raw data, it will be difficult to see the trend , At the same time, because the amount of data is too dense , It will cause great pressure on the display of front-end pages .

But through interval Equispaced window , You can aggregate the original data according to the specified window length （ It is equivalent to downsampling the original data ）, Then show the trend line , The amount of data that the front-end display needs to pull from the database will be greatly reduced , So as to significantly improve the efficiency .

The following figure , The green curve is based on each 60 The raw data collected once per second directly shows the drawn curve , The orange curve is based on the original data 5 The curve of calculating the average value after dividing the window in minutes , You can see the orange curve in this case , It well reflects the overall data change trend , The amount of data required for front-end drawing is reduced 80%.

The trend line of high-frequency data acquisition in actual production , Can be considered through TDengine Of interval The downsampling capability provided by syntax , Without losing the overall trend information , Show data quickly .

2. Status window （state_window）

The status window allows users to follow an integer / String type collection amount （ Generally, it is the value representing the state , Such as equipment working mode ） To divide the window , Record the status value continuously unchanged into the same window . Then the collected values in each window are avg/max/min Wait for statistical aggregation 、 Or calculate the state duration .

For example, the machine tool operation and maintenance personnel in the processing plant , It is necessary to count the working and standby time of each machine tool , This is essentially based on the working state “status” The value of this state quantity is used to divide the window , And count the window duration .

  Above picture “status” Column , Represent the current working state of the equipment , The status window is right “status” Judge the continuous change of value , Three windows are divided , And you can aggregate each window separately , For example, calculate the start time in each window 、 End time 、 Total number of records 、 Average 、 Window duration, etc . These can be found in TDengine Pass a SQL Query statements are implemented directly , Specifically SQL The statement and query results are shown in the following figure .

 3. Session window （session）

Before discussing the session window in detail , Let's first look at a typical application scenario of the Internet of vehicles platform ： Vehicle travel is divided by vehicle driving data . A car starts from the start , The process of driving in the middle of stopping and stalling is regarded as a complete “ trip ”. When the car is moving , vehicle T-Box Generally according to 30 In seconds , Send its own status data to the Internet of vehicles platform ; When parking and stalling , No longer send . therefore , By analyzing the continuity of vehicle reported messages , The journey of the vehicle can be calculated .

The session window allows users to divide windows according to the time continuity of reported records , That is, the time interval between two adjacent records does not exceed a certain threshold , Then it belongs to the same window ; The threshold is exceeded , Then the old window ends , The new window starts ; Then make statistics such as duration for each window , Or perform various aggregation calculations on the original collected data in the window , It can be avg/max/min/count Or other user-defined functions .

  The session window is for ： Let users pass a SQL Statement to achieve sequential data “ Degree of continuity ” Auto partition window .

As shown in the figure below SQL Shown , User Settings session ( ts, 10s ), That is, the time interval between adjacent records is less than 10 The second record is divided into the same session window Inside ; And for intervals greater than 10 Two adjacent records of seconds are divided into different windows . In this way, blue is divided in the following figure 、 Red 、 Grey marks 3 A window , Each window can calculate the start of the window independently / End time 、 The length of the window 、 Statistics of interest to users such as the number of records in the window .

summary

In addition to the large amount of time series data 、 Besides the relatively simple structure , There are also a lot of timestamp processing involved in query scenarios . We often have to base on business scenarios , Group the collected data by timestamp and calculate . For this kind of calculation , If the developer reads the raw data into memory , Then the application layer program deals with the logic of time window division , You have to face the disk that reads a large number of original sequence records IO、CPU And memory overhead , At the same time, the code complexity of the business layer has also become higher .

If developers can flexibly use TDengine In this way Database The time series data window division capability provided , Combined with business scenarios , Select the appropriate window partition function to sink the relevant calculation load to the database layer , It can greatly improve the response performance of the system 、 Reduce load overhead , With half the effort .

---

Want to know more TDengine Database Specific details of , Welcome to GitHub View the relevant source code on .