Analysis of the overall design principle of Nacos configuration center (persistence, clustering, information synchronization)

1. Nacos Configuration center function design

This paper mainly describes Nacos Configuration center function related design , First Nacos The configuration center has the following core functions ：

• Persistent storage
• Client remote access Nacos-server Configuration data of
• Dynamic refresh of client configuration , as well as Nacos-server Notification of configuration changes
• Client local cache

2. Persistence

• By default, it is stored in the built-in derby database

  We are nacos In the directory data This folder can be found under the directory , Data is stored here by default .
  

• We can also switch the storage to Mysql in

  Then how should we store data in Mysql What about China? ？

  We can go to conf Change under the directory application.properties Information in the file ：
  

  If needed Mysql, We just need to open these configuration information , Pay attention to revision mysql Connection address and other information .

  After the relevant configuration information is modified , We need to create relevant databases in the database , stay conf The folder also gives nacos-mysql.sql file .

  /* * Copyright 1999-2018 Alibaba Group Holding Ltd. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */
  
  /******************************************/
  /*  Database full name  = nacos_config */
  /*  The name of the table  = config_info */
  /******************************************/
  CREATE TABLE `config_info` (
    `id` bigint(20) NOT NULL AUTO_INCREMENT COMMENT 'id',
    `data_id` varchar(255) NOT NULL COMMENT 'data_id',
    `group_id` varchar(255) DEFAULT NULL,
    `content` longtext NOT NULL COMMENT 'content',
    `md5` varchar(32) DEFAULT NULL COMMENT 'md5',
    `gmt_create` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT ' Creation time ',
    `gmt_modified` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT ' Modification time ',
    `src_user` text COMMENT 'source user',
    `src_ip` varchar(50) DEFAULT NULL COMMENT 'source ip',
    `app_name` varchar(128) DEFAULT NULL,
    `tenant_id` varchar(128) DEFAULT '' COMMENT ' Tenant fields ',
    `c_desc` varchar(256) DEFAULT NULL,
    `c_use` varchar(64) DEFAULT NULL,
    `effect` varchar(64) DEFAULT NULL,
    `type` varchar(64) DEFAULT NULL,
    `c_schema` text,
    PRIMARY KEY (`id`),
    UNIQUE KEY `uk_configinfo_datagrouptenant` (`data_id`,`group_id`,`tenant_id`)
  ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='config_info';
  
  /******************************************/
  /*  Database full name  = nacos_config */
  /*  The name of the table  = config_info_aggr */
  /******************************************/
  CREATE TABLE `config_info_aggr` (
    `id` bigint(20) NOT NULL AUTO_INCREMENT COMMENT 'id',
    `data_id` varchar(255) NOT NULL COMMENT 'data_id',
    `group_id` varchar(255) NOT NULL COMMENT 'group_id',
    `datum_id` varchar(255) NOT NULL COMMENT 'datum_id',
    `content` longtext NOT NULL COMMENT ' Content ',
    `gmt_modified` datetime NOT NULL COMMENT ' Modification time ',
    `app_name` varchar(128) DEFAULT NULL,
    `tenant_id` varchar(128) DEFAULT '' COMMENT ' Tenant fields ',
    PRIMARY KEY (`id`),
    UNIQUE KEY `uk_configinfoaggr_datagrouptenantdatum` (`data_id`,`group_id`,`tenant_id`,`datum_id`)
  ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT=' Add tenant field ';
  
  
  /******************************************/
  /*  Database full name  = nacos_config */
  /*  The name of the table  = config_info_beta */
  /******************************************/
  CREATE TABLE `config_info_beta` (
    `id` bigint(20) NOT NULL AUTO_INCREMENT COMMENT 'id',
    `data_id` varchar(255) NOT NULL COMMENT 'data_id',
    `group_id` varchar(128) NOT NULL COMMENT 'group_id',
    `app_name` varchar(128) DEFAULT NULL COMMENT 'app_name',
    `content` longtext NOT NULL COMMENT 'content',
    `beta_ips` varchar(1024) DEFAULT NULL COMMENT 'betaIps',
    `md5` varchar(32) DEFAULT NULL COMMENT 'md5',
    `gmt_create` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT ' Creation time ',
    `gmt_modified` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT ' Modification time ',
    `src_user` text COMMENT 'source user',
    `src_ip` varchar(50) DEFAULT NULL COMMENT 'source ip',
    `tenant_id` varchar(128) DEFAULT '' COMMENT ' Tenant fields ',
    PRIMARY KEY (`id`),
    UNIQUE KEY `uk_configinfobeta_datagrouptenant` (`data_id`,`group_id`,`tenant_id`)
  ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='config_info_beta';
  
  /******************************************/
  /*  Database full name  = nacos_config */
  /*  The name of the table  = config_info_tag */
  /******************************************/
  CREATE TABLE `config_info_tag` (
    `id` bigint(20) NOT NULL AUTO_INCREMENT COMMENT 'id',
    `data_id` varchar(255) NOT NULL COMMENT 'data_id',
    `group_id` varchar(128) NOT NULL COMMENT 'group_id',
    `tenant_id` varchar(128) DEFAULT '' COMMENT 'tenant_id',
    `tag_id` varchar(128) NOT NULL COMMENT 'tag_id',
    `app_name` varchar(128) DEFAULT NULL COMMENT 'app_name',
    `content` longtext NOT NULL COMMENT 'content',
    `md5` varchar(32) DEFAULT NULL COMMENT 'md5',
    `gmt_create` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT ' Creation time ',
    `gmt_modified` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT ' Modification time ',
    `src_user` text COMMENT 'source user',
    `src_ip` varchar(50) DEFAULT NULL COMMENT 'source ip',
    PRIMARY KEY (`id`),
    UNIQUE KEY `uk_configinfotag_datagrouptenanttag` (`data_id`,`group_id`,`tenant_id`,`tag_id`)
  ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='config_info_tag';
  
  /******************************************/
  /*  Database full name  = nacos_config */
  /*  The name of the table  = config_tags_relation */
  /******************************************/
  CREATE TABLE `config_tags_relation` (
    `id` bigint(20) NOT NULL COMMENT 'id',
    `tag_name` varchar(128) NOT NULL COMMENT 'tag_name',
    `tag_type` varchar(64) DEFAULT NULL COMMENT 'tag_type',
    `data_id` varchar(255) NOT NULL COMMENT 'data_id',
    `group_id` varchar(128) NOT NULL COMMENT 'group_id',
    `tenant_id` varchar(128) DEFAULT '' COMMENT 'tenant_id',
    `nid` bigint(20) NOT NULL AUTO_INCREMENT,
    PRIMARY KEY (`nid`),
    UNIQUE KEY `uk_configtagrelation_configidtag` (`id`,`tag_name`,`tag_type`),
    KEY `idx_tenant_id` (`tenant_id`)
  ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='config_tag_relation';
  
  /******************************************/
  /*  Database full name  = nacos_config */
  /*  The name of the table  = group_capacity */
  /******************************************/
  CREATE TABLE `group_capacity` (
    `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT ' Primary key ID',
    `group_id` varchar(128) NOT NULL DEFAULT '' COMMENT 'Group ID, The empty character indicates the entire cluster ',
    `quota` int(10) unsigned NOT NULL DEFAULT '0' COMMENT ' The quota ,0 Indicates use default ',
    `usage` int(10) unsigned NOT NULL DEFAULT '0' COMMENT ' Usage quantity ',
    `max_size` int(10) unsigned NOT NULL DEFAULT '0' COMMENT ' Maximum size of a single configuration , The unit is byte ,0 Indicates use default ',
    `max_aggr_count` int(10) unsigned NOT NULL DEFAULT '0' COMMENT ' Maximum number of aggregate sub configurations ,,0 Indicates use default ',
    `max_aggr_size` int(10) unsigned NOT NULL DEFAULT '0' COMMENT ' The maximum sub configuration size of a single aggregate data , The unit is byte ,0 Indicates use default ',
    `max_history_count` int(10) unsigned NOT NULL DEFAULT '0' COMMENT ' Maximum number of change history ',
    `gmt_create` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT ' Creation time ',
    `gmt_modified` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT ' Modification time ',
    PRIMARY KEY (`id`),
    UNIQUE KEY `uk_group_id` (`group_id`)
  ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT=' colony 、 various Group Capacity information table ';
  
  /******************************************/
  /*  Database full name  = nacos_config */
  /*  The name of the table  = his_config_info */
  /******************************************/
  CREATE TABLE `his_config_info` (
    `id` bigint(64) unsigned NOT NULL,
    `nid` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
    `data_id` varchar(255) NOT NULL,
    `group_id` varchar(128) NOT NULL,
    `app_name` varchar(128) DEFAULT NULL COMMENT 'app_name',
    `content` longtext NOT NULL,
    `md5` varchar(32) DEFAULT NULL,
    `gmt_create` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
    `gmt_modified` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
    `src_user` text,
    `src_ip` varchar(50) DEFAULT NULL,
    `op_type` char(10) DEFAULT NULL,
    `tenant_id` varchar(128) DEFAULT '' COMMENT ' Tenant fields ',
    PRIMARY KEY (`nid`),
    KEY `idx_gmt_create` (`gmt_create`),
    KEY `idx_gmt_modified` (`gmt_modified`),
    KEY `idx_did` (`data_id`)
  ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT=' Multi tenant transformation ';
  
  
  /******************************************/
  /*  Database full name  = nacos_config */
  /*  The name of the table  = tenant_capacity */
  /******************************************/
  CREATE TABLE `tenant_capacity` (
    `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT ' Primary key ID',
    `tenant_id` varchar(128) NOT NULL DEFAULT '' COMMENT 'Tenant ID',
    `quota` int(10) unsigned NOT NULL DEFAULT '0' COMMENT ' The quota ,0 Indicates use default ',
    `usage` int(10) unsigned NOT NULL DEFAULT '0' COMMENT ' Usage quantity ',
    `max_size` int(10) unsigned NOT NULL DEFAULT '0' COMMENT ' Maximum size of a single configuration , The unit is byte ,0 Indicates use default ',
    `max_aggr_count` int(10) unsigned NOT NULL DEFAULT '0' COMMENT ' Maximum number of aggregate sub configurations ',
    `max_aggr_size` int(10) unsigned NOT NULL DEFAULT '0' COMMENT ' The maximum sub configuration size of a single aggregate data , The unit is byte ,0 Indicates use default ',
    `max_history_count` int(10) unsigned NOT NULL DEFAULT '0' COMMENT ' Maximum number of change history ',
    `gmt_create` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT ' Creation time ',
    `gmt_modified` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT ' Modification time ',
    PRIMARY KEY (`id`),
    UNIQUE KEY `uk_tenant_id` (`tenant_id`)
  ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT=' Tenant capacity information table ';
  
  
  CREATE TABLE `tenant_info` (
    `id` bigint(20) NOT NULL AUTO_INCREMENT COMMENT 'id',
    `kp` varchar(128) NOT NULL COMMENT 'kp',
    `tenant_id` varchar(128) default '' COMMENT 'tenant_id',
    `tenant_name` varchar(128) default '' COMMENT 'tenant_name',
    `tenant_desc` varchar(256) DEFAULT NULL COMMENT 'tenant_desc',
    `create_source` varchar(32) DEFAULT NULL COMMENT 'create_source',
    `gmt_create` bigint(20) NOT NULL COMMENT ' Creation time ',
    `gmt_modified` bigint(20) NOT NULL COMMENT ' Modification time ',
    PRIMARY KEY (`id`),
    UNIQUE KEY `uk_tenant_info_kptenantid` (`kp`,`tenant_id`),
    KEY `idx_tenant_id` (`tenant_id`)
  ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='tenant_info';
  
  CREATE TABLE `users` (
  	`username` varchar(50) NOT NULL PRIMARY KEY,
  	`password` varchar(500) NOT NULL,
  	`enabled` boolean NOT NULL
  );
  
  CREATE TABLE `roles` (
  	`username` varchar(50) NOT NULL,
  	`role` varchar(50) NOT NULL,
  	UNIQUE INDEX `idx_user_role` (`username` ASC, `role` ASC) USING BTREE
  );
  
  CREATE TABLE `permissions` (
      `role` varchar(50) NOT NULL,
      `resource` varchar(255) NOT NULL,
      `action` varchar(8) NOT NULL,
      UNIQUE INDEX `uk_role_permission` (`role`,`resource`,`action`) USING BTREE
  );
  
  INSERT INTO users (username, password, enabled) VALUES ('nacos', '$2a$10$EuWPZHzz32dJN7jexM34MOeYirDdFAZm2kuWj7VEOJhhZkDrxfvUu', TRUE);
  
  INSERT INTO roles (username, role) VALUES ('nacos', 'ROLE_ADMIN');
  

• After processing , We started nacos, Create relevant configuration information , Then check in the database . You can find that it has been used mysql As persistent .
  

3. Synchronization of configuration information

The significance of configuration information synchronization is that when our configuration information changes , You need to notify each node .

The way of synchronization was mentioned a little before , There are only two ways ：

• push （ The server takes the initiative push）

  A connection will be maintained between the client and the server , The client sends the heartbeat , Server side perception . So when the data changes , The server can directly pass socket Sending notice .

• pull （ Client polls for ）

  The scheduled task of the client pulls data every few seconds

And ours nacos It's used long pull（ Long polling ） The way .

What is long polling ？ Compared with short polling , My request will not end immediately , This conversation will remain , So when the data on the server changes, you can write back the data directly to the pipeline .

Its whole implementation mechanism is as follows ：

The client will send a request to the server every once in a while , The server will not return immediately after receiving the request , Instead, the request will be placed in a queue , This queue stores the request sessions of all clients who obtain configuration change events from the server .

When our console modifies the configuration information , The configuration update event will be triggered , Then the requests in the queue will be polled .

This long polling is not always maintained , exceed 30 It will automatically disconnect after seconds .

The above is the way of long polling to complete the dynamic update of events , Thus, the client can dynamically perceive the changes of configuration .

4. Nacos Configure the central cluster

To ensure high availability , We definitely need to build clusters . Now let's take a brief look at ：

Nacos The cluster of the configuration center is different from the cluster we built before , He has no master and slave , Because we know ,nacos All the data is stored in the database , So we don't need to ensure the consistency of his data ; If in order to ensure that data will not be lost , That's it mysql The active and standby of the layer are built ; but nacos It's not just that , At every nacos Nodes and mysql There is also a layer of disk cache , At startup, data will be obtained from the database and cached on disk , At the same time, he has a scheduled task , every other 6 Hours to update the full amount of data in the database to the disk . So when our client starts from nacos When getting data from ,nacos It is obtained directly from disk , This can improve certain performance .

What if our data changes ？ How can this decentralized cluster synchronize data ？

When the client updates the data to a node , The nacos Nodes will find other nodes in the cluster , Broadcast a data synchronization event , Received the event nacos The node will fetch data from the database and synchronize it to the cache （ Notice the increment , Not the whole amount ）. The data update order is to update the database first and then the disk .

After understanding the general principle , How can we build a cluster ？

This is mainly about the cluster construction of the configuration center , If the Registration Center cluster is built, then at least 3 Nodes , Because the registry cluster uses raft Algorithm , It's different from here .

Only the following two steps are needed to build a cluster ：

• 3 Nodes

  Just copy three copies directly , stay application Modify the port number in the file .

• modify conf In the catalog cluster.conf file

  take cluster.conf.example Copy the file and change the name , Then add the addresses and port numbers of the three nodes .

• Starting mode

  Start directly by default , There is no need to use single node mode