Distributed task scheduling XXL job

1. Centralized task scheduling

The concept of centralized task and distributed task scheduling is exactly the opposite , Centralized tasks are stand-alone tasks , A project , One server , That is what we often call monomer application . For centralized tasks , That's us Java Common scheduled tasks in development .

1.1 Timing task

A scheduled task is to perform a task at a specified time （ Business code ）;

Basic concept diagram of timed task

Timing task application scenarios

1.2 Java The way to achieve scheduled tasks

(1) while(true) + Thread.sleep

(2) java.util.Timer + java.util.TimerTask
Timer It's a timer tool , It is used to schedule the execution of specified tasks in a background thread , It can plan to perform a task once or repeatedly .

(3) ScheduledExecutorService
ScheduledExecutorService It's from jdk1.5 Started to be introduced for concurrency tool classes , It is the most ideal way to realize timed tasks .

(4) Quartz
Quartz Scheduling is an open source scheduling framework , from Java Written , be used for Java Timed task scheduling in ecology , It is a flexible and convenient 、 Use a simple timed task scheduling framework , You can talk to Spring Integrated use .

(5) Spring Task
Spring The lightweight scheduled task calling tool provided by the framework , Easy to use ;

(6) SpringBoot annotation @EnableScheduling + @Scheduled ; The bottom layer depends on Spring Task;

1.3 Defects of centralized task scheduling

• Fragmentation tasks are not supported . When processing ordered data , Multiple machines perform tasks in pieces to process different data .
• Unified lifecycle management is not supported . Shut down without restarting the service , Start the task .
• No clustering support , There is a problem of repeated task execution .
• Failed retries are not supported . After an exception, the task ends , The task cannot be re executed according to the execution status control .
• Dynamic adjustment is not supported . Modify task parameters without restarting the service .
• No alarm mechanism . There is no alarm mechanism after the task fails .
• Task data is difficult to count . When the amount of task data is large , The execution of tasks cannot be counted efficiently .

2. Distributed task scheduling

2.1 Solution

Because the centralized scheduled task scheduling needs to solve a series of problems , So there are some solutions in the process of evolution

• Database unique constraints ;
• Use profile 、redis、mysql As a switch for dispatching ;
• Use distributed lock to realize scheduling control ;
• Using distributed task scheduling platform TBSchedule、Elastic-Job、Saturn、XXL-JOB etc. ;
• Self research framework .

2.2 The benefits of distributed task scheduling

Solve the congenital defect of centralized task scheduling .

• High availability . Under the cluster architecture , Some nodes have exceptions , It does not affect the execution of the task .
• Dynamic configuration . Execute the task cycle , And other task related attributes are modified constantly .
• Life cycle management . The startup task can be executed for a single task without downtime , Stop tasks for management .
• Failure mechanism . Execution failure occurs during task execution , It can support alarm , Retry the task and quickly check the execution log .
• Data statistics . When there are many scheduled tasks , How many missions , Which tasks have failed , Which tasks were successfully executed .
• Shard to perform . For batch processed data , Let the machine perform the task , Slice the processed data .

2.3 Open source framework

TBSchedule
The open source distributed task calling system launched by Taobao , It is widely used in many Internet companies , But now there is no update maintenance ;
GitHub Address : https://github.com/taobao/TBSchedule

Elastic-Job
The distributed task scheduling framework launched by Dangdang , Many companies' products are using this distributed task scheduling framework ;
Official website http://elasticjob.io

Saturn
The open source distributed task scheduling platform launched by vipshop , It is based on Elastic-Job Developed , Added some features , Vipshop and some Internet companies are using .
GitHub Address ： https://github.com/vipshop/Saturn

PowerJob
Next generation distributed task scheduling and computing framework , Support CRON、API、 Fixed frequency 、 Fixed delay and other scheduling strategies , Provide workflow to orchestrate tasks, resolve dependencies , Easy to use , Powerful , The documentation is complete .
Official website : http://www.powerjob.tech/
GitHub Open source address : https://github.com/KFCFans/PowerJob
Gitee Open source address : https://gitee.com/KFCFans/PowerJob

XXL-JOB
Meituan open source lightweight distributed task scheduling platform , Its core design goal is lightweight 、 Easy to expand 、 Rapid development 、 Open the box , Many companies have used online products . Project on 2015 year 11 Release the first version in 1.0, The latest version 2.xxx
Official website ： https://www.xuxueli.com/xxl-job/
GitHub Address ： https://github.com/xuxueli/xxl-job/
Gitee Address : https://gitee.com/xuxueli0323/xxl-job/

3. xxl-job frame

xxl-job Is a distributed task scheduling platform . Learning blog ： https://www.cnblogs.com/newAndHui/p/13862285.html

3.1 Architecture diagram

xxl-job Architecture diagram

Task trigger architecture
Task scheduling — Trigger task — actuator

3.2 xxl-job Installation instructions

3.2.1 Community support and source code download

Xuxueli community https://www.xuxueli.com/page/projects.html
xxl-job Of GitHub Address https://github.com/xuxueli/xxl-job/

3.2.2 initialization admin database

Initialize the task call center admin database , sql The script is in the source directory xxl-job/doc/db/ Next , tables_xxl_job.sql file .

3.2.3 modify admin The configuration file

Major changes db To configure ;

port , The context name can also be modified .

3.2.4 visit admin home page

start-up admin application , visit admin home page http://127.0.0.1:8081/xxl-job-admin, Check the actuator list 、 Task list .
user name admin
password 123456

3.2.5 Add actuator

Add an actuator on the actuator management page .

Add actuator successfully ：

3.2.6 New task

Add a new task on the task management page .

New task succeeded ：

3.2.7 The actuator depends on the core package

xxl-job-executor-sample Actuator Application , Add dependency , Configuration properties , Write tasks, etc ;

1） add to maven rely on

<dependency>
    <groupId>com.xuxueli</groupId>
    <artifactId>xxl-job-core</artifactId>
    <version>${project.parent.version}</version>
</dependency>

2） Profile information
db Related configuration information , The log directory configuration can be customized ;

### Address of task scheduling center 
### xxl-job admin address list, such as "http://address" or "http://address01,http://address02"
xxl.job.admin.addresses=http://127.0.0.1:8081/xxl-job-admin

### xxl-job, access token
xxl.job.accessToken=

### xxl-job executor appname
xxl.job.executor.appname=xxl-job-executor-sample
### xxl-job executor registry-address: default use address to registry , otherwise use ip:port if address is null
xxl.job.executor.address=
### xxl-job executor server-info
xxl.job.executor.ip=
xxl.job.executor.port=9998
### xxl-job executor log-path
xxl.job.executor.logpath=/data/applogs/xxl-job/jobhandler
### xxl-job executor log-retention-days
xxl.job.executor.logretentiondays=30

3） Configuration class XxlJobConfig
Instantiate the actuator XxlJobSpringExecutor

package com.xxl.job.executor.core.config;

import com.xxl.job.core.executor.impl.XxlJobSpringExecutor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/** * xxl-job config * * @author xuxueli 2017-04-28 */
@Configuration
public class XxlJobConfig {
    
    private Logger logger = LoggerFactory.getLogger(XxlJobConfig.class);

    @Value("${xxl.job.admin.addresses}")
    private String adminAddresses;

    @Value("${xxl.job.accessToken}")
    private String accessToken;

    @Value("${xxl.job.executor.appname}")
    private String appname;

    @Value("${xxl.job.executor.address}")
    private String address;

    @Value("${xxl.job.executor.ip}")
    private String ip;

    @Value("${xxl.job.executor.port}")
    private int port;

    @Value("${xxl.job.executor.logpath}")
    private String logPath;

    @Value("${xxl.job.executor.logretentiondays}")
    private int logRetentionDays;


    @Bean
    public XxlJobSpringExecutor xxlJobExecutor() {
    
        logger.info(">>>>>>>>>>> xxl-job config init.");
        XxlJobSpringExecutor xxlJobSpringExecutor = new XxlJobSpringExecutor();
        xxlJobSpringExecutor.setAdminAddresses(adminAddresses);
        xxlJobSpringExecutor.setAppname(appname);
        xxlJobSpringExecutor.setAddress(address);
        xxlJobSpringExecutor.setIp(ip);
        xxlJobSpringExecutor.setPort(port);
        xxlJobSpringExecutor.setAccessToken(accessToken);
        xxlJobSpringExecutor.setLogPath(logPath);
        xxlJobSpringExecutor.setLogRetentionDays(logRetentionDays);

        return xxlJobSpringExecutor;
    }

    /** *  For multiple network cards 、 Container internal deployment, etc , With the help of  "spring-cloud-commons"  Provided  "InetUtils"  Component flexible custom registration IP; * * 1、 Introduce dependencies ： * <dependency> * <groupId>org.springframework.cloud</groupId> * <artifactId>spring-cloud-commons</artifactId> * <version>${version}</version> * </dependency> * * 2、 The configuration file , Or container startup variables  * spring.cloud.inetutils.preferred-networks: 'xxx.xxx.xxx.' * * 3、 obtain IP * String ip_ = inetUtils.findFirstNonLoopbackHostInfo().getIpAddress(); */
}

4） establish handler Interface
The old version was created handler class , Inherited IJobHandler class , Rewrote execute Method , And is scheduled to execute execute Method .

public class GlueJobHandler extends IJobHandler {
    
	private long glueUpdatetime;
	private IJobHandler jobHandler;
	public GlueJobHandler(IJobHandler jobHandler, long glueUpdatetime) {
    
		this.jobHandler = jobHandler;
		this.glueUpdatetime = glueUpdatetime;
	}
	public long getGlueUpdatetime() {
    
		return glueUpdatetime;
	}

	@Override
	public void execute() throws Exception {
    
		XxlJobHelper.log("----------- glue.version:"+ glueUpdatetime +" -----------");
		jobHandler.execute();
	}
}

The new version is no longer inheritance IJobHandler, Instead, write a class with annotations @Component Instantiate into container , Add annotations to the methods in the class @XxlJob("jobName") Annotation indicates that this method is to schedule the tasks to be executed .

/** * XxlJob Development example （Bean Pattern ） * *  Development steps ： * 1、 Task development ： stay Spring Bean In the example , Development Job Method ; * 2、 Annotation configuration ： by Job Method add annotation  "@XxlJob(value=" Customize jobhandler name ", init = "JobHandler Initialization method ", destroy = "JobHandler Destruction method ")", annotation value The value corresponds to the new task created by the dispatch center JobHandler The value of the property . * 3、 Execution log ： Need to pass through  "XxlJobHelper.log"  Print execution log ; * 4、 The result of the mission ： The default task result is  " success "  state , There is no need to actively set ; If there is a demand , For example, set the task result as failure , Can pass  "XxlJobHelper.handleFail/handleSuccess"  Set the task results independently ; * * @author xuxueli 2019-12-11 21:52:51 */
@Component
public class SampleXxlJob {
    
    private static Logger logger = LoggerFactory.getLogger(SampleXxlJob.class);
    /** * 1、 Simple task example （Bean Pattern ） */
    @XxlJob("demoJobHandler")
    public void demoJobHandler() throws Exception {
    
        XxlJobHelper.log("XXL-JOB, Hello World.");
        System.out.println(" Yes  demoJobHandler --");

        for (int i = 0; i < 5; i++) {
    
            XxlJobHelper.log("beat at:" + i);
            TimeUnit.SECONDS.sleep(2);
        }
        // default success
    }
}    

4. xxl-job Cluster deployment

4.1 xxl-job-admin Cluster deployment

First, deploy several machines in the cluster xxl-job-admin,8080 ,8081 Port two dispatching centers ;
operation ：xxl-job-admin.jar Package preparation
Start two task scheduling centers ：
java -jar xxl-job-admin.jar --server.port=8080
java -jar xxl-job-admin.jar --server.port=8081
Visit the task scheduling platform ：
http://localhost:8080/xxl-job-admin/

http://localhost:8081/xxl-job-admin/

4.2 xxl-job-executor-example Cluster deployment

4.2.1 Cluster deployment configuration

If xxl-job-admin The task scheduling center is a cluster deployment ,xxl-job-executor-sample The actuator should be registered through the load balancing address , Can be xxl.job.admin.addresses Configure as load balancing address or domain name ; Then start multiple machines respectively actuator Apply to register . Before registration, you need to prepare the configuration of load balancing , Let's introduce ：
Configuration of actuator registration address , The registered address of the actuator is domain name ->nginx agent ( Load balancing )-> Task scheduling platform

##  stand-alone 
# xxl.job.admin.addresses=http://127.0.0.1:8081/xxl-job-admin
##  Cluster deployment , Register the actuator through the load balanced domain name address   Domain name in hosts Configuration in file 
xxl.job.admin.addresses=http://www.jobs.com/xxl-job-admin

windows System hosts File configuration domain name , And refresh dns To configure ; When accessing www.jobs.com It is equivalent to visiting 127.0.0.1 The host address of . Of course, we can also do without domain name resolution , Directly configure the host address .hosts file location C:\Windows\System32\drivers\etc

Refresh dns To configure ： ipconfig /flushdns

Nginx Load balancing interception configuration , When the actuator is applied to xxl-job-admin Registration time , Load balancing forwarding .

Nginx Load balancing configuration , Randomly forward to two task scheduling centers ：admin-8080, admin-8088;
If you visit http://localhost:80/xxl-job-admin Will be randomly forwarded to the following address ：
http://localhost:8080/xxl-job-admin
http://localhost:8081/xxl-job-admin
Be careful , above 80 yes nginx Listening port .

After revising nginx.conf After profile , Check that the configuration file is configured correctly ：

nginx -t -c /softwareInstalled/nginx-1.20.1/conf/nginx.conf

If nginx Has been launched , Reload configuration , And restart the process gracefully

nginx -s reload

If nginx Didn't start , So start directly Nginx

start nginx

4.2.2 Register the actuator

According to the principle of the above figure , Register two registrars to the task scheduling center
The first actuator modifies the configuration and starts ：

server.port=8082
xxl.job.admin.addresses=http://www.jobs.com/xxl-job-admin
xxl.job.executor.port=9998

The second actuator modifies the configuration and starts

server.port=8088
xxl.job.admin.addresses=http://www.jobs.com/xxl-job-admin
xxl.job.executor.port=9999

Check the task scheduling center “ Actuator Management ”, Two actuators have been registered successfully .
http://localhost:8080/xxl-job-admin/jobgroup

http://localhost:8081/xxl-job-admin/jobgroup

From the actuator registration flow chart, we can analyze ,8082,8088 The actuator is registered to 8080 and 8081 Dispatching center ; When the task scheduling center triggers a task, it passes 9998、9999 The port scheduling actuator performs tasks .（8082、8088 The port is external to the actuator web port ）

4.2.3 Task scheduling test

Perform a task manually , Check the execution log . According to the weight, it is forwarded to register to 9998 On the actuator of .

Will be registered to 9998 The actuator of stops . Manually trigger a task again , Sign up to 9999 Our actuators can still successfully perform tasks .

From the following task scheduling diagram, we can analyze ,8080,8081 The dispatch center is connected with the actuator 9998,9999 Port communication . The routing strategy can also be adjusted by yourself （ Random 、 polling 、 Specify actuator services, etc ）.

5. xxl-job design idea

• Abstract scheduling behavior into “ Dispatching center ” Public platform , The platform itself does not assume the business logic ,“ Dispatching center ” Responsible for initiating scheduling requests .
• Scatter the task abstraction JobHandler, Leave it to “ actuator ” Unified management ,“ actuator ” Responsible for receiving scheduling requests and executing corresponding JobHandler Business logic in . therefore ,“ Dispatch ” and “ Mission ” The two parts can be decoupled from each other , Improve the overall stability and scalability of the system ;
• Scheduling module （ Dispatching center ）： Manage scheduling information , Send the scheduling request according to the scheduling configuration , They do not undertake business code . The scheduling system is decoupled from the task , Improved system availability and stability , At the same time, the performance of scheduling system is no longer limited by task module ;
• Support Visualization 、 Simple and dynamic management and scheduling information , Including task creation 、 to update 、 Delete 、GLUE Development and task alerts, etc , All the above operations will take effect in real time , At the same time, it supports monitoring scheduling results and execution logs , Support actuators Failover.
• Perform module （ actuator ）： Responsible for receiving scheduling requests and executing task logic , Task module focuses on task execution and other operations , Development and maintenance are simpler and more efficient ; receive “ Dispatching center ” Execution request for 、 Terminate requests, log requests, etc .

6. xxl-job-admin Core code

6.1 admin Initialization work

1）JobRegistryHelper#getInstance.start() Ensure that the list of actuators obtained when the task is executed is running , Start a daemon thread .

 Every time 30s Query the automatically registered executors in the database ;
 Inquire about 90s Actuators that are not automatically registered again ;
delete  Delete 90s Actuators that are not registered again register surface ;
upate  to update group Tabular addressList;

2）JobRegistryHelper#getInstance.start() Start a thread , Scan the failure log to determine whether to retry , If you need to retry, execute the trigger ;

3）initRpcProvider(); initialization rpc service ,AdminBiz（ Task execution result callback 、 Register actuator 、 Remove the actuator ）

4）JobScheduleHelper#getInstance.start();

scheduleThread  structure 5s Tasks to be performed in ;
ringThread  perform 5s Trigger of internal task ;

6.2 The trigger works

1） More failures than 10 The second task uses the slow thread pool to execute ;
2） Query the details of the task to be performed ;
3） The address notified from the actuator address list according to the routing policy ;

7. Interview questions

Is your project distributed ？ How is the project timing task done ？
Distributed task scheduling, if you can implement it , What's the idea , How to achieve ？
Have you learned about some open source solutions to solve this problem ？

B Station video recommendation ：
https://www.bilibili.com/video/BV1w541187Wa

Personal blog

Welcome to personal blog : https://www.crystalblog.xyz/

Alternate address : https://wang-qz.gitee.io/crystal-blog/