dolphinscheduler2. Installation of X (valid for personal test)

0 Environmental preparation

jdk1.8
mysql5.7 above （ But the driver needs to use 8 Version of ）

install

qianfeng01:master worker
qianfeng02:worker
qianfeng03:worker

1、 Environmental Science

 ## Three virtual machines need 
 install -y psmisc

2、 Initialize metabase ( open mysql)

mysql -uroot -p'@Mmforu45'

The following is mysql Create database , New users , to grant authorization

## Building database 
CREATE DATABASE dolphinscheduler DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
## password 
 set global validate_password_policy=0;
 set global validate_password_length=4;
## Create user 
CREATE USER 'dolphinscheduler'@'%' IDENTIFIED BY '1234';
## jurisdiction 
GRANT ALL PRIVILEGES ON dolphinscheduler.* TO 'dolphinscheduler'@'%';
flush privileges;

Be careful

set global validate_password_policy=0;
 set global validate_password_length=4;
  These two are not executed before adding users , Will report a mistake 
 ERROR 1819 (HY000): Your password does not satisfy the current policy
requirements

3、 Set the configuration of the little dolphin

Installation package

link ：https://pan.baidu.com/s/1VLvMzOl-1c2e4VxNDxkkIQ?pwd=mzrh
Extraction code ：mzrh
– From Baidu network disk super member V2 The share of

Install the package and mysql The driver is thrown onto the server

tar -xvf apache-dolphinscheduler-2.0.5-bin.tar.gz -c /usr/local/
cd /usr/local/
mv apache-dolphinscheduler-2.0.5-bin dolphinscheduler

cp mysql-connector-java-8.0.16.jar /usr/local/dolphinscheduler/lib/

Be careful ：

 I started mysql What drives the selection is 5.7 edition , however mysql The initialization error message is driver, I haven't found the specific reason , You can only put 5.7 Version when 8 The version uses . Is the use of 8 Version of the driver , Initialization was successful （ I used... For the first time 5.7 The driver is also initialized successfully ）

The most important step

cd /usr/local/dolphinscheduler/conf/config/
vi install_config.conf

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

# ---------------------------------------------------------
# INSTALL MACHINE
# ---------------------------------------------------------
# A comma separated list of machine hostname or IP would be installed DolphinScheduler,
# including master, worker, api, alert. If you want to deploy in pseudo-distributed
# mode, just write a pseudo-distributed hostname
# Example for hostnames: ips="ds1,ds2,ds3,ds4,ds5", Example for IPs: ips="192.168.8.1,192.168.8.2,192.168.8.3,192.168.8.4,192.168.8.5"
ips="qianfeng01,qianfeng02,qianfeng03"
# Cluster host number 
# Port of SSH protocol, default value is 22. For now we only support same port in all `ips` machine
# modify it if you use different ssh port
sshPort="22"

# A comma separated list of machine hostname or IP would be installed Master server, it
# must be a subset of configuration `ips`.
# Example for hostnames: masters="ds1,ds2", Example for IPs: masters="192.168.8.1,192.168.8.2"
masters="qianfeng01"
# Master node 
# A comma separated list of machine <hostname>:<workerGroup> or <IP>:<workerGroup>.All hostname or IP must be a
# subset of configuration `ips`, And workerGroup have default value as `default`, but we recommend you declare behind the hosts
# Example for hostnames: workers="ds1:default,ds2:default,ds3:default", Example for IPs: workers="192.168.8.1:default,192.168.8.2:default,192.168.8.3:default"
workers="qianfeng01:default,qianfeng02:default,qianfeng03:default"
# From the node 
# A comma separated list of machine hostname or IP would be installed Alert server, it
# must be a subset of configuration `ips`.
# Example for hostname: alertServer="ds3", Example for IP: alertServer="192.168.8.3"
alertServer="qianfeng01"
# Alarm host 
# A comma separated list of machine hostname or IP would be installed API server, it
# must be a subset of configuration `ips`.
# Example for hostname: apiServers="ds1", Example for IP: apiServers="192.168.8.1"
apiServers="qianfeng01"
# Development api host 
# A comma separated list of machine hostname or IP would be installed Python gateway server, it
# must be a subset of configuration `ips`.
# Example for hostname: pythonGatewayServers="ds1", Example for IP: pythonGatewayServers="192.168.8.1"
# pythonGatewayServers="ds1"
# above python You can comment out 
# The directory to install DolphinScheduler for all machine we config above. It will automatically be created by `install.sh` script if not exists.
# Do not set this configuration same as the current path (pwd)
installPath="/usr/local/"
# The one click installation script above is to install the local dolphin on other machines , Paths must exist on other machines , Otherwise, it will report a mistake , The little dolphin of this machine will also be deleted 
# The user to deploy DolphinScheduler for all machine we config above. For now user must create by yourself before running `install.sh`
# script. The user needs to have sudo privileges and permissions to operate hdfs. If hdfs is enabled than the root directory needs
# to be created by this user
deployUser="root"
# Deployed account name 
# The directory to store local data for all machine we config above. Make sure user `deployUser` have permissions to read and write this directory.
dataBasedirPath="/tmp/dolphinscheduler"

# ---------------------------------------------------------
# DolphinScheduler ENV
# ---------------------------------------------------------
# JAVA_HOME, we recommend use same JAVA_HOME in all machine you going to install DolphinScheduler
# and this configuration only support one parameter so far.
javaHome="/usr/local/jdk"
#java The installation path 
# DolphinScheduler API service port, also this is your DolphinScheduler UI component's URL port, default value is 12345
apiServerPort="12345"
#web port 
# ---------------------------------------------------------
# Database
# NOTICE: If database value has special characters, such as `.*[]^${}\+?|()@#&`, Please add prefix `\` for escaping.
# ---------------------------------------------------------
# The type for the metadata database
# Supported values: ``postgresql``, ``mysql`, `h2``.
DATABASE_TYPE="mysql"
# Database type 
# Spring datasource url, following <HOST>:<PORT>/<database>?<parameter> format, If you using mysql, you could use jdbc
# string jdbc:mysql://127.0.0.1:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8 as example
SPRING_DATASOURCE_URL="jdbc:mysql://qianfeng03:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8"
#mysqlurl
# Spring datasource username
SPRING_DATASOURCE_USERNAME="dolphinscheduler"
#mysql Account 
# Spring datasource password
SPRING_DATASOURCE_PASSWORD="1234"
# password 
# ---------------------------------------------------------
# Registry Server
# ---------------------------------------------------------
# Registry Server plugin name, should be a substring of `registryPluginDir`, DolphinScheduler use this for verifying configuration consistency
registryPluginName="zookeeper"
#zookeeper
# Registry Server address.
registryServers="qianfeng01:2181,qianfeng02:2181,qianfeng03:2181"

# Registry Namespace
registryNamespace="dolphinscheduler"

# ---------------------------------------------------------
# Worker Task Server
# ---------------------------------------------------------
# Worker Task Server plugin dir. DolphinScheduler will find and load the worker task plugin jar package from this dir.
taskPluginDir="lib/plugin/task"

# resource storage type: HDFS, S3, NONE
resourceStorageType="HDFS"
# Resource type selection 
# resource store on HDFS/S3 path, resource file will store to this hdfs path, self configuration, please make sure the directory exists on hdfs and has read write permissions. "/dolphinscheduler" is recommended
resourceUploadPath="/dolphinscheduler"

# if resourceStorageType is HDFS,defaultFS write namenode address,HA, you need to put core-site.xml and hdfs-site.xml in the conf directory.
# if S3,write S3 address,HA,for example ：s3a://dolphinscheduler,
# Note,S3 be sure to create the root directory /dolphinscheduler
defaultFS="hdfs://qianfeng01:8020"
# Is in the web The contents in front of the end 
# if resourceStorageType is S3, the following three configuration is required, otherwise please ignore
s3Endpoint="http://192.168.xx.xx:9010"
s3AccessKey="xxxxxxxxxx"
s3SecretKey="xxxxxxxxxx"

# resourcemanager port, the default value is 8088 if not specified
resourceManagerHttpAddressPort="8088"

# if resourcemanager HA is enabled, please set the HA IPs; if resourcemanager is single node, keep this value empty
yarnHaIps=
# Nothing is empty 
# if resourcemanager HA is enabled or not use resourcemanager, please keep the default value; If resourcemanager is single node, you only need to replace 'yarnIp1' to actual resourcemanager hostname
singleYarnIp="qianfeng01"

# who has permission to create directory under HDFS/S3 root path
# Note: if kerberos is enabled, please config hdfsRootUser=
hdfsRootUser="root"
#hadoop Unclassified 
# kerberos config
# whether kerberos starts, if kerberos starts, following four items need to config, otherwise please ignore
kerberosStartUp="false"
# kdc krb5 config file path
krb5ConfPath="$installPath/conf/krb5.conf"
# keytab username,watch out the @ sign should followd by \\
keytabUserName="hdfs-mycluster\\@ESZ.COM"
# username keytab path
keytabPath="$installPath/conf/hdfs.headless.keytab"
# kerberos expire time, the unit is hour
kerberosExpireTime="2"

# use sudo or not
sudoEnable="true"

# worker tenant auto create
workerTenantAutoCreate="false"

vi /usr/local/dolphinscheduler/script/create-dolphinscheduler.sh

##Java The path must be changed 
export JAVA_HOME=/usr/local/jdk

4、 Initialize database

cd /usr/local/dolphinscheduler/script
./create-dolphinscheduler.sh

Before initialization ,mysql The driver package must be placed in lib Under the table of contents ！！！

5、 A key to install （ That is, the system automatically configures the little dolphin of this machine according to the configuration scp To the slave node ）

Three machines zookeeper Be sure to open （ If only one is turned on, no error will be reported , however web cannot access ！）

There is no need to manually scp Files to other servers , Direct operation shell！

zkServer.sh start  

cd /usr/local/dolphinscheduler
./install.sh

6、 visit

 website :  http://qianfeng01:12345/dolphinscheduler/
 account number : admin
 password : dolphinscheduler123