[tcapulusdb knowledge base] Introduction to tcapulusdb table data caching

Introduction to cache function ：

Table data caching service refers to tcaplus Incremental data can be written to the third-party system slowly （ At present, we support MySQL and Kafka）, Used for product data query and analysis . The principle is tcapsvr take （ Starting with caching enabled ） Incremental write data （ be based on tcaplus binlog） Synchronize to tcapdb,tcapdb Write again / Change to MySQL perhaps Kafka in . Therefore, this function must guarantee tcapdb The machine has access MySQL perhaps Kafka Authority , If it is MySQL, And there are also Mysql Permission to create tables . Otherwise, the function cannot be configured successfully .

Write slowly MySQL For example （Kafka The entry is similar to the configuration steps ）, The configuration method is ： Enter the table management page , Find the corresponding table , On the right " Cache " link , You can configure caching ：

The cache page is as follows , Fill in the corresponding parameters to configure caching ：

A transaction will be generated after the commit , Configure the service .

among ：

1. Cache cycle ： Represents data that users can accept from Tcapdb Delay in synchronizing to third-party systems , Minimum 1s, The bigger this is , It means that the user is less sensitive to delay .Tcapdb The side will de duplicate the requests within the cache cycle time window （ for example ,1s Inside to the same key Many times Replace operation , After weight removal, only the last time Replace The content after synchronization is sent to the third-party system , The first two requests can be omitted ）, So as to reduce the overall synchronization QPS Lifting performance .

2. key Field mapping ：tcaplus surface key Fields to MySQL Watch or Kafka Of key Mapping of fields , If there is no special requirement, the table fields of the two systems should be consistent , It is conducive to positioning . Binary type is not supported key Field （ namely tdr In the array 、 Secondary field ,pb Medium bytes 、repeated Other types ） Cache .

3. value Field mapping ：tcaplus surface value Fields to MySQL Watch or Kafka Of value Mapping of fields , If there is no special requirement, the table fields of the two systems should be consistent , It is conducive to positioning .value If the field is of binary type , Will binary Synchronize to MySQL perhaps Kafka, Users need to learn from MySQL Read deserialization resolution .

Be careful ：

1. tcaplus The cache function only synchronizes the incremental modification to the third-party system after the cache is enabled , The data before enabling is not synchronized , If you want to synchronize this part of data, you need DBA Pull out the cold standby , take txh data dump Import into the third-party system , Then enable cache .

2. The performance bottleneck of caching is tcapsvr Synchronization thread CPU,tcapsvr Side stand alone QPS Up to 8w/s.tcapdb The capability of the side can be expanded horizontally, which is not a bottleneck .

---

For more information, please pay attention to TcaplusDB WeChat official account , In the wind and rain , We've been waiting for you ！

TcaplusDB It's a distributed product of Tencent NoSQL database , The code for storage and scheduling is completely self-developed . With cache + Landing fusion architecture 、PB Levels of storage 、 Millisecond delay 、 Lossless horizontal expansion and complex data structure . At the same time, it has rich ecological environment 、 Easy migration 、 Extremely low operation and maintenance costs and five nine high availability features . Customer coverage game 、 Internet 、 government affairs 、 Finance 、 Manufacturing and the Internet of things .