[Elastic-Job] Overview of Distributed Scheduling Tasks

Foreword

  Our company uses task scheduling, evolving from spring's own stand-alone task framework to using distributed scheduling ElasticJob, mainly because our company's business and data volume have gradually increased, and the stand-alone model has seriously affectedperformance.However, there are still more mature distributed task frameworks in the market. The more well-known ones are Elastic-Job and xxl-Job. Why did our company choose Elastic-Job?

Reference documents:

Content

Understand some basic concepts before starting
What is a scheduled task
   Task scheduling is the system for automatic completionA specific task, the process of performing a task at an agreed specific time.With task scheduling, more manpower can be liberated by the system to automatically perform tasks
What is a distributed scheduling task
  Split a task intoMultiple independent task items, one or several fragmented items are executed by distributed servers respectively

Stand-alone mode of task scheduling
1.timer: It is a timer class, through which you can configure the specified timed task.The timerTask class is a timed task class, which implements the Runnable interface. The disadvantage is that the thread will be terminated if the exception is not touched.
2.scheduleExecutorService: The relative delay or cycle is scheduled as a timed task, and the disadvantage is that there is no determined date or time
br> 3. Spring timing framework: There are many functions to configure. If the system uses a single machine, you can give priority to the spring timer

Distributed task scheduling needs products on the market

• Elastic-job: Dangdang's elastic distributed task scheduling system based on quartz secondary development, with rich and powerful functions, using zookeeper to achieve distributed coordination, high task availability and sharding
font>
• Saturn: A distributed task scheduling platform open sourced by Vipshop. It can be configured globally, monitored in a unified manner, with high task availability and sharding concurrency. It is based on elastic-job.Improved
• xxl-job: A distributed task scheduling platform on Dianping. It is a lightweight distributed task scheduling platform. Its core design goals are rapid development, easy learning, and lightweight, easy to expand, the source code has been developed and connected to the online product lines of many companies out of the box
• TBSchedule: Taobao's excellent high-performance distributed scheduling framework is currently used in the process scheduling systems of many Internet companies such as Ali, Jingdong, Alipay, and Gome
• Elastic-job is a distributed scheduling solution, open sourced by Dangdang.com. It consists of two independent sub-projects elastic-job-lite and elastic=cloud, using elastic-job can quickly realize distributed task scheduling

Main functions of Elastic-job
1. Distributed scheduling coordination
In a distributed environment, tasks can be executed according to the specified scheduling policy, and can avoid repeated execution of multiple instances of the same task
2. Rich scheduling strategies
Execute timed tasks based on the mature timed task job framework quartz corn expression
3. Elastic expansion and contraction
When an instance is added to the cluster, it should be able to be elected and perform tasks. When an instance is reduced in the cluster, the tasks it performs can be transferred to otherinstance to execute
4. Failover
After an instance fails to execute the task, it will be transferred to another instance for execution
5.Error execution job re-trigger
If the job misses the execution time for some reason, it will automatically record the missed execution job and trigger it automatically after the last job is completed
6. Support parallelismScheduling
Supports task sharding. Task sharding refers to the simultaneous execution of multiple small task items in a task environment in multiple instances
7. Job sharding consistency
After the task is sharded, ensure that the same shard has only one execution instance in the distributed environment
8. Support job life cycle operations
The task can be started and stopped dynamicallyOperation
9. Rich job types
Support three job types: simple, dataFlow, script

[Summary]

No matter what kind of scheduling task, its internal mechanism is similar. As long as one internal implementation mechanism can be understood clearly, it will be handy to use another scheduling task., I will continue to learn about this part of the content in the future