当前位置：网站首页>clickhouse 21.x 集群四分片一副本部署

clickhouse 21.x 集群四分片一副本部署

2022-06-22 15:41:00 【51CTO】

标签（空测试用例格分隔）：clickhouse 系列

一：系统环境部署介绍

1.1：系统初始化

系统：
   centos7.9x64
 
 主机名字:
   cat /etc/hosts 
   
   -----
   
172.16.10.11    flyfishsrvs01
172.16.10.12    flyfishsrvs02
172.16.10.13    flyfishsrvs03
172.16.10.14    flyfishsrvs04
172.16.10.15    flyfishsrvs05
172.16.10.16    flyfishsrvs06
172.16.10.17    flyfishsrvs07 

   ------
  本次部署前四台服务器 

     
      1.
      2.
      3.
      4.
      5.
      6.
      7.
      8.
      9.
      10.
      11.
      12.
      13.
      14.
      15.
      16.
      17.
      18.

1.2 系统部署jdk

在flyfishsrvs02,flyfishsrvs03,flyfishsrvs04 上面部署

tar -zxvf jdk-8u301-linux-x64.tar.gz
mv jdk1.8.0_301/ /usr/local/jdk
vim /etc/profile 
----

### jdk
export JAVA_HOME=/usr/local/jdk
export CLASSPATH=.:$JAVA_HOME/jre/lib:$JAVA_HOME/lib:$JAVA_HOME/lib/tools.jar
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin
----

source /etc/profile

java -version 

     
      1.
      2.
      3.
      4.
      5.
      6.
      7.
      8.
      9.
      10.
      11.
      12.
      13.
      14.
      15.
      16.

clickhouse 21.x 集群四分片一副本部署_clickhouse

clickhouse 21.x 集群四分片一副本部署_clickhouse_02

clickhouse 21.x 集群四分片一副本部署_clickhouse21.x 集群部署_03

1.3 配置zookeeper 集群

在flyfishsrvs02,flyfishsrvs03,flyfishsrvs04 上面部署
tar -zxvf apache-zookeeper-3.7.0-bin.tar.gz
mv apache-zookeeper-3.7.0-bin /usr/local/zookeeper

mkdir -p /usr/local/zookeeper/data
mkdir -p /usr/local/zookeeper/log

cd /usr/local/zookeeper/data/

echo 1 > myid
----
cd /usr/local/zookeeper/conf 

cp zoo_sample.cfg zoo.cfg 

vim zoo.cfg 

----
# 心跳时间
tickTime=2000
# follow连接leader的初始化连接时间，表示tickTime的倍数
initLimit=10
# syncLimit配置表示leader与follower之间发送消息，请求和应答时间长度。如果followe在设置的时间内不能与leader进行通信，那么此follower将被丢弃，tickTime的倍数
syncLimit=5
# 客户端连接端口
clientPort=2181
# 节点数据存储目录，需要提前创建，注意myid添加，用于标识服务器节点
dataDir=/usr/local/zookeeper/data
dataLogDir=/usr/local/zookeeper/log
server.1=172.16.10.12:2888:3888
server.2=172.16.10.13:2888:3888
server.3=172.16.10.14:2888:3888
---
-----

scp -r zookeeper [email protected]:/usr/local/
scp -r zookeeper [email protected]:/usr/local/
修改172.16.10.13 节点 myid 
cd /usr/local/zookeeper/data/
echo 2 > myid 
修改172.16.10.14 节点 myid
cd /usr/local/zookeeper/data/
echo 3 > myid



     
      1.
      2.
      3.
      4.
      5.
      6.
      7.
      8.
      9.
      10.
      11.
      12.
      13.
      14.
      15.
      16.
      17.
      18.
      19.
      20.
      21.
      22.
      23.
      24.
      25.
      26.
      27.
      28.
      29.
      30.
      31.
      32.
      33.
      34.
      35.
      36.
      37.
      38.
      39.
      40.
      41.
      42.
      43.
      44.
      45.

clickhouse 21.x 集群四分片一副本部署_clickhouse21.x_04

clickhouse 21.x 集群四分片一副本部署_clickhouse21.x 集群部署_06

二：部署clickhouse 21.x

2.1 安装clickhouse 21.x

clickhouse 21.x 集群四分片一副本部署_clickhouse21.x 集群部署_09

修改listen_host

vim +164 /etc/clickhouse-server/config.xml

----
<listen_host>::1</listen_host>
<listen_host>flyfishsrvs01</listen_host>  ### 其它节点修改对于主机名
-----

     
      1.
      2.
      3.
      4.
      5.
      6.
      7.
      8.

clickhouse 21.x 集群四分片一副本部署_clickhouse_10

2.2 配置集群文件

配置集群连接文件
cd /etc/clickhouse-server/config.d
vim metrika.xml 
-----
<yandex> 
    <remote_servers> 
        <clickhouse_cluster_4shards_1replicas> 
           <shard> 
               <internal_replication>true</internal_replication>
                  <replica> 
                     <host>flyfishsrvs01</host> 
                     <port>9000</port> 
                  </replica> 
           </shard> 
           <shard> 
               <internal_replication>true</internal_replication> 
                  <replica> 
                     <host>flyfishsrvs02</host> 
                     <port>9000</port> 
                  </replica> 
           </shard> 
           <shard> 
               <internal_replication>true</internal_replication> 
                  <replica> 
                     <host>flyfishsrvs03</host> 
                     <port>9000</port> 
                  </replica> 
           </shard>
           <shard> 
               <internal_replication>true</internal_replication> 
                  <replica> 
                     <host>flyfishsrvs04</host> 
                     <port>9000</port> 
                  </replica> 
           </shard>
           </clickhouse_cluster_4shards_1replicas>
    </remote_servers> 
    <zookeeper>
       <node index="1"> 
           <host>flyfishsrvs02</host> 
           <port>2181</port> 
        </node> 
       <node index="2"> 
          <host>flyfishsrvs03</host> 
          <port>2181</port> 
       </node> 
       <node index="3"> 
          <host>flyfishsrvs04</host> 
          <port>2181</port> 
       </node> 
     </zookeeper>

     <macros> 
        <shard>01</shard> 
        <replica>flyfishsrvs01</replica> 
     </macros> 
     <networks> 
         <ip>::/0</ip> 
     </networks> 
     <clickhouse_compression> 
         <case> 
            <min_part_size>10000000000</min_part_size> 
            <min_part_size_ratio>0.01</min_part_size_ratio> 
            <method>lz4</method> 
         </case> 
     </clickhouse_compression> 
</yandex>

---------------
scp metrika.xml [email protected]:/etc/clickhouse-server/config.d/
scp metrika.xml [email protected]:/etc/clickhouse-server/config.d/
scp metrika.xml [email protected]:/etc/clickhouse-server/config.d/

     
      1.
      2.
      3.
      4.
      5.
      6.
      7.
      8.
      9.
      10.
      11.
      12.
      13.
      14.
      15.
      16.
      17.
      18.
      19.
      20.
      21.
      22.
      23.
      24.
      25.
      26.
      27.
      28.
      29.
      30.
      31.
      32.
      33.
      34.
      35.
      36.
      37.
      38.
      39.
      40.
      41.
      42.
      43.
      44.
      45.
      46.
      47.
      48.
      49.
      50.
      51.
      52.
      53.
      54.
      55.
      56.
      57.
      58.
      59.
      60.
      61.
      62.
      63.
      64.
      65.
      66.
      67.
      68.
      69.
      70.
      71.
      72.

clickhouse 21.x 集群四分片一副本部署_clickhouse_14

clickhouse 21.x 集群四分片一副本部署_clickhouse21.x 集群部署_15

2.3 集群启动与验证

修改对应节点的副本主机名
vim 
#flyfishsrvs02节点修改metrika.xml中的宏变量如下： 
<macros> 
<shard>02</replica>
<replica>flyfishsrvs02</replica> 
</macros> 
#flyfishsrvs03节点修改metrika.xml中的宏变量如下: 
<macros> 
<shard>03</replica>
<replica>flyfishsrvs03</replica> 
</macros>

#flyfishsrvs04节点修改metrika.xml中的宏变量如下: 
<macros> 
<shard>04</replica>
<replica>flyfishsrvs04</replica> 
</macros>
 


     
      1.
      2.
      3.
      4.
      5.
      6.
      7.
      8.
      9.
      10.
      11.
      12.
      13.
      14.
      15.
      16.
      17.
      18.
      19.
      20.

clickhouse 21.x 集群四分片一副本部署_clickhouse_17

#每台节点启动Clickchouse服务 
service clickhouse-server start
#每台节点查看clickhouse服务状态 
service clickhouse-server status 
#每台节点重启clickhouse服务 
service clickhouse-server restart 
#每台节点关闭Clikchouse服务 
service clickhouse-server stop

     
      1.
      2.
      3.
      4.
      5.
      6.
      7.
      8.

clickhouse 21.x 集群四分片一副本部署_clickhouse_20

验证：
clickhouse-client -h flyfishsrvs01
查看集群：
select * from system.clusters;
select cluster,host_name from system.clusters;


     
      1.
      2.
      3.
      4.
      5.
      6.

clickhouse 21.x 集群四分片一副本部署_clickhouse21.x_24