Advantages of time series database and traditional database

The problem of storing time series data in traditional relational databases
With time series data , Where should I store it ？ First, let's look at the problems that traditional relational database solutions will encounter when storing temporal data .

Many people may think that adding a timestamp column to a traditional relational database can be used as a temporal database . When the amount of data is small, it's no problem . But time series data are often generated by millions or even tens of millions of terminal devices , Write concurrency is high , It belongs to a massive data scenario .

MySQL In the scenario of massive time series data, there are the following problems ：

Storage costs are high ： For time series data compression is not good , It takes up a lot of machine resources ;
Maintenance costs are high ： Stand alone system , Need to be in the upper manual sub database sub table , Maintenance costs are high ;
Low write throughput ： Single machine write throughput is low , It is difficult to meet the write pressure of tens of millions of sequential data ;
Poor query performance ： For transaction processing , The aggregation analysis performance of massive data is poor .
in addition , Use Hadoop ecology （Hadoop、Spark etc. ） There are the following problems when storing timing data ：

High data latency ： Offline batch system , Data from generation to analyzability , It takes hours 、 Even heaven level ;
Poor query performance ： Can't make good use of index , rely on MapReduce Mission , The query time is usually at the minute level .
It can be seen that the following problems need to be solved in the temporal database ：

Writing of time series data ： How to support the writing of hundreds of millions of data points per second .
Reading of time series data ： How to support the grouping and aggregation of hundreds of millions of data at the second level .
Cost sensitive ： What is brought by massive data storage is the cost problem . How to store these data at a lower cost , It will become the most important problem to be solved in time series database .

**** The advantages of temporal databases
*** The invention of temporal database products is to solve the deficiencies and defects of traditional relational database in temporal data storage and analysis , Such products are uniformly classified as time series database .*** According to the characteristics of timing data, write 、 Storage 、 Query and other processes have been optimized , These optimizations are closely related to the characteristics of time series data ：

Storage costs ：
Use time to increase 、 Dimension repeat 、 The characteristics of smooth change of indicators , Choose a reasonable coding and compression algorithm , Improve data compression ratio ;
By pre reducing the accuracy , Aggregate historical data , Save storage space .

High concurrent write ：
Write data in bulk , Reduce network overhead ;
Data is written to memory first , Re cyclical dump Store for immutable files .

Low query latency , High query concurrency ：
Optimize common query patterns , Reduce query latency through indexing and other technologies ;
Through caching 、routing And other technologies to improve query concurrency .

Comparison of open source time series database

At present, the more popular open source time series database products in the industry are InfluxDB、OpenTSDB、Prometheus、Graphite etc. , The comparison of product characteristics is shown in the figure below ：