2022-02-28 incluxdb high availability planning

Catalog

Abstract :

The problem is :

suffer TSM Storage engine implementation impacts :

workload :

---

Abstract :

influxdb Achieve high availability in advance , This article details how to deal with high availability

The problem is :

1. The newly added data node synchronizes data , There is data written repeatedly , Whether it will cause problems ?
2. Synchronize data to other data nodes , There is data written out of order , Whether it will cause problems ?
3. key After hashing , The service is not interrupted when the slot is migrated , How to design ? Reference resources ceph?

suffer TSM Storage engine implementation impacts :

1. In the writing tsm Before document , It's going to work on the memory cache Sort
2. wal Write the structure for order
3. cache by map structure

workload :

1. Add the concept of data node group under fragmentation , Divide the data node group under a specific partition
2. modify meta Maintain the data node network structure , Expand the data node group under the partition
3. modify meta Service Interface , The purpose is to add a data node group
4. Modify the data node network connection structure , Use data node groups to connect to each other in the cluster
5. Add and kakfa Interface of interaction