Apache DolphinScheduler版本2.0.5分布式集群的安装

1. 部署模式说明

DolphinScheduler支持多种部署模式，包括单机模式(Standalone)、伪集群模式(Pseudo-Cluster)、集群模式(Cluster)等

单机模式:
单机模式(standalone)模式下，所有服务均集中于一个StandaloneServer进程中，并且其中内置了注册中心Zookeeper和数据库H2。只需配置JDK环境，就可一键启动DolphinScheduler，快速体验其功能

伪集群模式:
伪集群模式(Pseudo-Cluster)是在单台机器部署DolphinScheduler各项服务，该模式下master、worker、api server、logger server等服务都只在同一台机器上，但位于不同的进程。Zookeeper和数据库需单独安装并进行相应配置

集群模式:
集群模式(Cluster)与伪集群模式的区别就是在多台机器部署DolphinScheduler各项服务，并且Master、Worker等服务可配置多个

2. DolphinScheduler分布式集群部署

2.1 服务器部署规划

3台服务器之间，配置无密码登录。这样部署脚本就可以通过scp的方式将安装需要的资源传输到其它服务器上

服务	服务器	备注
Java	dolphin1、dolphin2、dolphin3	参考centos7同时安装java8和openJdk11、windows同时安装java8和openJdk11
Zookeeper	dolphin1、dolphin2、dolphin3	参考基于Centos7分布式安装Zookeeper3.6.3
Mysql	dolphin3	centos7安装mysql8.0.25版本
psmisc	dolphin1、dolphin2、dolphin3	Centos7安装命令：yum install psmisc
masters	dolphin1、dolphin2、dolphin3
workers	dolphin1、dolphin2、dolphin3
alertServer	dolphin1
apiServers	dolphin2
pythonGatewayServers	dolphin3

2.2 下载解压(dolphin1上)

[[email protected] ~]# wget https://downloads.apache.org/dolphinscheduler/2.0.5/apache-dolphinscheduler-2.0.5-bin.tar.gz
[[email protected] ~]#
[[email protected] ~]# tar -zxvf apache-dolphinscheduler-2.0.5-bin.tar.gz
[[email protected] ~]#
[[email protected] ~]# cd apache-dolphinscheduler-2.0.5-bin
[[email protected] apache-dolphinscheduler-2.0.5-bin]#

2.3 修改conf/config/install_config.conf(dolphin1上)

修改如下：

# ---------------------------------------------------------
# INSTALL MACHINE
# ---------------------------------------------------------
# ips为DolphinScheduler集群的所有服务器
ips="dolphin1,dolphin2,dolphin3"
masters="dolphin1,dolphin2,dolphin3"
workers="dolphin1:default,dolphin2:default,dolphin3:default"
alertServer="dolphin1"
apiServers="dolphin2"
pythonGatewayServers="dolphin3"

# 路径不存在，会自动创建
installPath="/root/dolphinscheduler-2.0.5-install"

deployUser="root"


# ---------------------------------------------------------
# DolphinScheduler ENV
# ---------------------------------------------------------
javaHome="/root/jdk1.8.0_201"

# ---------------------------------------------------------
# Database
# ---------------------------------------------------------
DATABASE_TYPE="mysql"
SPRING_DATASOURCE_URL="jdbc:mysql://dolphin3:3306/dolphinScheduler?useUnicode=true&characterEncoding=UTF-8"
SPRING_DATASOURCE_USERNAME="root"
SPRING_DATASOURCE_PASSWORD="Root_123"

# ---------------------------------------------------------
# Registry Server
# ---------------------------------------------------------
registryServers="dolphin1:2181,dolphin2:2181,dolphin3:2181"

2.4 创建元数据库和添加jar包(dolphin1上)

连接Mysql数据库，创建DolphinScheduler的数据库

mysql> create database dolphinScheduler default character set utf8 default collate utf8_general_ci;
Query OK, 1 row affected, 2 warnings (0.05 sec)

mysql>

下载Mysql的驱动包，到DolphinScheduler的lib目录下

[[email protected] ~]# wget https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.25/mysql-connector-java-8.0.25.jar -P apache-dolphinscheduler-2.0.5-bin/lib
[[email protected] ~]#

2.5 初始化元数据库(dolphin1上)

运行脚本，进行元数据库的初始化

[[email protected] ~]# sh apache-dolphinscheduler-2.0.5-bin/script/create-dolphinscheduler.sh

2.6 启动DolphinScheduler(dolphin1上)

必须在apache-dolphinscheduler-2.0.5-bin目录下，运行install.sh脚本

[[email protected] apache-dolphinscheduler-2.0.5-bin]# 
[[email protected] apache-dolphinscheduler-2.0.5-bin]# pwd
/root/apache-dolphinscheduler-2.0.5-bin
[[email protected] apache-dolphinscheduler-2.0.5-bin]# 
[[email protected] apache-dolphinscheduler-2.0.5-bin]# sh install.sh
[[email protected] apache-dolphinscheduler-2.0.5-bin]# 

会根据ips和installPath参数，自动在所有服务器上创建目录，并上传安装所需要的资源

日志文件存放在每台服务器的/root/dolphinscheduler-2.0.5-install/logs目录下

通过以下命令查询所有服务的状态

[[email protected] ~]# sh dolphinscheduler-2.0.5-install/bin/status-all.sh
[[email protected] ~]#

可能有的服务没有启动，先stop所有服务，再start所有服务

[[email protected] ~]#
[[email protected] ~]# sh dolphinscheduler-2.0.5-install/bin/stop-all.sh
[[email protected] ~]#
[[email protected] ~]# sh dolphinscheduler-2.0.5-install/bin/start-all.sh
[[email protected] ~]#

如果还有服务没启动起来，可以到对应的服务器运行命令，启动单个服务，参考如下：

# 启停Master
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh start master-server
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh stop master-server

# 启停Worker
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh start worker-server
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh stop worker-server

# 启停Api
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh start api-server
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh stop api-server

# 启停Logger
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh start logger-server
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh stop logger-server

# 启停Alert
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh start alert-server
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh stop alert-server

# 启停Python Gateway
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh start python-gateway-server
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh stop python-gateway-server

2.7 登录Web页面

登录http://dolphin2:12345/dolphinscheduler，默认的账号/密码是admin/dolphinscheduler123。登录后的界面如下所示

其中数据源中心，能定义各种数据库的连接，这样SQL任务就能进行引用了