当前位置:网站首页>August 14, 2020: what are the execution engines for data tasks?
August 14, 2020: what are the execution engines for data tasks?
2020-11-06 21:55:00 【Fuda Dajia architect's daily question】
Fogo's answer 2020-08-14:
The most basic thing is crontab Timing task .
Considering the distributed partition timing task and high availability , It can be used elastic-job Such a framework . But such a framework cannot handle complex task dependencies .
Big data processing frameworks generally use DAG( Directed acyclic graph ) Workflow scheduling system , adopt DAG The graph describes the complex dependencies between tasks . stay Spark In job scheduling system , Is based on DAG describe rdd Dependencies between .
版权声明
本文为[Fuda Dajia architect's daily question]所创,转载请带上原文链接,感谢
边栏推荐
- 统计项目代码行数
- 2020-08-24:什么是小文件?很多小文件会有什么问题?很多小文件怎么解决?(大数据)
- Ora-02292: complete constraint violation (midbjdev2.sys_ C0020757) - subrecord found
- ORA-02292: 违反完整约束条件 (MIDBJDEV2.SYS_C0020757) - 已找到子记录
- Summary of front-end interview questions (C, s, s) that front-end engineers need to understand (2)
- Introduction to the development of small game cloud
- What the hell is fastthreadlocal? The existence of ThreadLocal!!
- 2020-08-18:介绍下MR过程?
- 2020-08-29:进程线程的区别,除了包含关系之外的一些区别,底层详细信息?
- C calls SendMessage to refresh the taskbar icon (the icon does not disappear at the end of forcing)
猜你喜欢
Summary of common SQL statements
The native API of the future trend of the front end: web components
2020-08-30:裸写算法:二叉树两个节点的最近公共祖先。
How to make characters move
The method of local search port number occupation in Windows system
vue3 新特性
Novice guidance and event management system in game development
Epu360: all the H5 templates you want are here, e-book, big turntable, red envelope rain, questionnaire survey
[self taught unity2d legendary game development] map editor
#JVM 类加载机制
随机推荐
1万辆!理想汽车召回全部缺陷车:已发生事故97起,亏损将扩大
细数软件工程----各阶段必不可少的那些图
C and C / C + + mixed programming series 5 - GC collaboration of memory management
DC-1 target
The Interpreter pattern of behavior pattern
NAND FLASH的接口控制设计
[elastic search engine]
What are the highlights of Huawei mate 40 series with HMS?
Metersphere developer's Manual
Why is quicksort so fast?
The memorandum model of behavior model
How much disk space does a new empty file take?
Qt音视频开发46-视频传输UDP版
2020 database technology conference helps technology upgrade
Can you do it with only six characters?
Zero basis to build a web search engine of its own
To solve the problem that the data interface is not updated after WPF binding set
Unexpected element.. required element
超高频RFID医疗血液管理系统应用
Epu360: all the H5 templates you want are here, e-book, big turntable, red envelope rain, questionnaire survey