Application of time series database in Hydropower Station

Hello everyone , In the last issue, Professor Wang Hongzhi introduced the frontier field of database , I wonder if the partners have a deeper understanding of the database field , In this issue, let's go back to the time series database , Let's talk about the application of time series database in the field of hydropower station . 

This article only represents personal views , If there is bias , Please ask Hai Han. ～

Traditional hydropower station monitoring system meets the challenge

In the automatic control process of hydropower station , The operation data monitoring system of the power station plays an important role . Its later stage is operation analysis 、 Trend judgment and accident handling provide a strong guarantee . The monitoring data of hydropower station has the characteristics of high acquisition frequency 、 And presents the characteristics of time series sorting , Use traditional RDB（ Such as MySQL、SQL Server、Oracle etc. ） Storage brings many problems .

First , Its storage cost is high , Low write throughput . The measuring points of large hydropower stations are very large , Even if it's just 1 Ten thousand measuring points , If you collect once per second , It takes about... A day 10 ~ 20 GB Disk storage space . The capacity of ordinary disk array is difficult to meet the storage requirements of the above massive data , Moreover, the compression of time series data in relational database is not ideal , This leads to the passage of time , Its cost will continue to rise . Besides , Under the traditional scheme , Massive data writing takes a long time , It is difficult to meet the writing requirements of tens of millions of levels .

secondly , Poor query performance . Although the relational database will be divided into databases and tables 、 Optimize index and other technical operations , But as storage data grows , Its query efficiency will still be significantly reduced , It is difficult to obtain the required data in seconds or even milliseconds . meanwhile , The table splitting strategy will also increase the complexity of query business . such as , If you divide the tables by month , Then the query of cross month data needs to pass multiple SQL Or joint query to get the desired results .

Time series database solves the bottleneck of traditional database

In view of the characteristics that most of the data stored in the hydropower plant monitoring system is time series data , Building the monitoring system of hydropower station based on time series database has become a better choice . Compared with relational database , Time series database is used in data storage and query , Have obvious performance advantages .

The efficient compression ratio of time series database can save a lot of storage space . This is mainly because it adopts the method of column storage . Each column stores an ordered set of data , Convert this data into a single line “ Array ” Data in form . Then for its data type , Each array is compressed separately using a specific compression algorithm .

In terms of inquiry , The data of time series database is usually recorded every second or every millisecond , This leads to rapid data growth . For traditional relational databases , Due to the extensive use of B+ The tree is indexed , When the amount of data reaches a certain level , Its write performance will decline significantly . The time series database can be automatically partitioned , With continuous data writing , Automatically create partitions with timestamps , Store the time series data in partitions , Ensure that the index of each partition is maintained at a small scale , So as to maintain the write performance . When querying, you can also quickly locate the required data partition , Ensure query performance .

Better database based services

The time series database of hydropower station monitoring system is mainly used for curve query 、 Report query and other services , According to the relevant query request, query the corresponding data in the time series database , Generate the corresponding result set and return . For example, report query function , Mainly for timing data sampling 、 Output , To achieve the average value of optional interval timing data 、 Maximum 、 minimum value 、 Statistical functions such as mean value . It also requires , Configure the required measuring point statistics function through the configuration interface , The front page can display the required Statistics . At present, the aggregate query function provided by the time series database can fully meet the statistical functions required by the hydropower station monitoring system , And the request time is 1 Within seconds . Another example , Curve query function , It requires that the system can provide real-time data of any historical period from the time series database , Power station personnel query real-time curve , It is required that the curves of multiple measuring points can be queried at the same time , For analysis and comparison .

All right, guys , That's all we'll talk about in this issue , See you next time .

CnosDB brief introduction

CnosDB It's a high performance 、 High usability open source distributed time series database , It has been officially released and fully open source .

Welcome to our code warehouse , One key, three links ：https://github.com/cnosdb/cnosdb