Installation of Apache DolphinScheduler version 2.0.5 distributed cluster

1. 部署模式说明

DolphinScheduler支持多种部署模式,包括单机模式(Standalone)、伪集群模式(Pseudo-Cluster)、集群模式(Cluster)等

单机模式:
单机模式(standalone)模式下,所有服务均集中于一个StandaloneServer进程中,并且其中内置了注册中心Zookeeper和数据库H2.只需配置JDK环境,就可一键启动DolphinScheduler,快速体验其功能

伪集群模式:
伪集群模式(Pseudo-Cluster)是在单台机器部署DolphinScheduler各项服务,该模式下master、worker、api server、logger server等服务都只在同一台机器上,But in a different process.Zookeeper和数据库需单独安装并进行相应配置

集群模式:
集群模式(Cluster)与伪集群模式的区别就是在多台机器部署DolphinScheduler各项服务,并且Master、WorkerSeveral other services can be configured

2. DolphinScheduler分布式集群部署

2.1 服务器部署规划

3between servers,配置无密码登录.This way the deployment script will passscpway to transfer the resources required by the installation to other servers

服务	服务器	备注
Java	dolphin1、dolphin2、dolphin3	参考centos7同时安装java8和openJdk11、windows同时安装java8和openJdk11
Zookeeper	dolphin1、dolphin2、dolphin3	参考基于Centos7分布式安装Zookeeper3.6.3
Mysql	dolphin3	centos7安装mysql8.0.25版本
psmisc	dolphin1、dolphin2、dolphin3	Centos7安装命令：yum install psmisc
masters	dolphin1、dolphin2、dolphin3
workers	dolphin1、dolphin2、dolphin3
alertServer	dolphin1
apiServers	dolphin2
pythonGatewayServers	dolphin3

2.2 下载解压(dolphin1上)

[[email protected] ~]# wget https://downloads.apache.org/dolphinscheduler/2.0.5/apache-dolphinscheduler-2.0.5-bin.tar.gz
[[email protected] ~]#
[[email protected] ~]# tar -zxvf apache-dolphinscheduler-2.0.5-bin.tar.gz
[[email protected] ~]#
[[email protected] ~]# cd apache-dolphinscheduler-2.0.5-bin
[[email protected] apache-dolphinscheduler-2.0.5-bin]#

2.3 修改conf/config/install_config.conf(dolphin1上)

修改如下：

# ---------------------------------------------------------
# INSTALL MACHINE
# ---------------------------------------------------------
# ips为DolphinScheduler集群的所有服务器
ips="dolphin1,dolphin2,dolphin3"
masters="dolphin1,dolphin2,dolphin3"
workers="dolphin1:default,dolphin2:default,dolphin3:default"
alertServer="dolphin1"
apiServers="dolphin2"
pythonGatewayServers="dolphin3"

# 路径不存在,会自动创建
installPath="/root/dolphinscheduler-2.0.5-install"

deployUser="root"


# ---------------------------------------------------------
# DolphinScheduler ENV
# ---------------------------------------------------------
javaHome="/root/jdk1.8.0_201"

# ---------------------------------------------------------
# Database
# ---------------------------------------------------------
DATABASE_TYPE="mysql"
SPRING_DATASOURCE_URL="jdbc:mysql://dolphin3:3306/dolphinScheduler?useUnicode=true&characterEncoding=UTF-8"
SPRING_DATASOURCE_USERNAME="root"
SPRING_DATASOURCE_PASSWORD="Root_123"

# ---------------------------------------------------------
# Registry Server
# ---------------------------------------------------------
registryServers="dolphin1:2181,dolphin2:2181,dolphin3:2181"

2.4 Create metabase and addjar包(dolphin1上)

连接Mysql数据库,创建DolphinScheduler的数据库

mysql> create database dolphinScheduler default character set utf8 default collate utf8_general_ci;
Query OK, 1 row affected, 2 warnings (0.05 sec)

mysql>

下载Mysql的驱动包,到DolphinScheduler的lib目录下

[[email protected] ~]# wget https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.25/mysql-connector-java-8.0.25.jar -P apache-dolphinscheduler-2.0.5-bin/lib
[[email protected] ~]#

2.5 初始化元数据库(dolphin1上)

运行脚本,Initialize the metadata database

[[email protected] ~]# sh apache-dolphinscheduler-2.0.5-bin/script/create-dolphinscheduler.sh

2.6 启动DolphinScheduler(dolphin1上)

必须在apache-dolphinscheduler-2.0.5-bin目录下,运行install.sh脚本

[[email protected] apache-dolphinscheduler-2.0.5-bin]# 
[[email protected] apache-dolphinscheduler-2.0.5-bin]# pwd
/root/apache-dolphinscheduler-2.0.5-bin
[[email protected] apache-dolphinscheduler-2.0.5-bin]# 
[[email protected] apache-dolphinscheduler-2.0.5-bin]# sh install.sh
[[email protected] apache-dolphinscheduler-2.0.5-bin]# 

会根据ips和installPath参数,Automatically create directories on all servers,And upload the resources required for installation

Log files are stored on each server/root/dolphinscheduler-2.0.5-install/logs目录下

Query the status of all services with the following command

[[email protected] ~]# sh dolphinscheduler-2.0.5-install/bin/status-all.sh
[[email protected] ~]#

Some services may not be started,先stop所有服务,再start所有服务

[[email protected] ~]#
[[email protected] ~]# sh dolphinscheduler-2.0.5-install/bin/stop-all.sh
[[email protected] ~]#
[[email protected] ~]# sh dolphinscheduler-2.0.5-install/bin/start-all.sh
[[email protected] ~]#

If there are still services not started,You can go to the corresponding server to run the command,启动单个服务,参考如下：

# 启停Master
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh start master-server
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh stop master-server

# 启停Worker
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh start worker-server
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh stop worker-server

# 启停Api
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh start api-server
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh stop api-server

# 启停Logger
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh start logger-server
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh stop logger-server

# 启停Alert
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh start alert-server
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh stop alert-server

# 启停Python Gateway
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh start python-gateway-server
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh stop python-gateway-server

2.7 登录Web页面

登录http://dolphin2:12345/dolphinscheduler,默认的账号/密码是admin/dolphinscheduler123.登录后的界面如下所示

Among them, the data source center,Can define connections to various databases,这样SQLTasks can be referenced