当前位置:网站首页>Azkaban actual combat
Azkaban actual combat
2022-07-05 02:46:00 【A vegetable chicken that is working hard】
A single job Case study
1. establish job Description file
- vim job1.job
#first.job
type=command
command=echo 'this is my first job'
2. take job Resource files are packaged into zip file
- install zip,unzip command
yum install -y unzip zip
- Compress
zip job1.zip job1.job
- Azkaban Uploaded workflow file Only support xxx.zip file ,zip Should include xxx.job Files and any files needed to run the job ( file name The suffix must be .job, Otherwise, I can't recognize ), The job name must be unique in the project
3. adopt azkaban Of web Management platform creation project And upload job Of zip package
- establish project
- Upload zip package
- Start execution of the job
- Job Successful implementation
- Click to see job journal
many job Workflow case
1. establish job Description file
- first job:start.job
vim start.job
#start.job
type=command
command=touch /usr/local/azkaban/jobs/t2test.txt
- the second job:step1.job rely on start.job
vim step1.job
#step1.job
type=command
dependencies=start
command=echo "this is step1 job"
- Third job:step2.job rely on start.job
vim step2.job
#step2.job
type=command
dependencies=start
command=echo "this is step2 job"
- The fourth one job:finish.job rely on step1.job and step2.job
vim finish.job
#finish.job
type=command
dependencies=step1,step2
command=echo "this is finish job"
2. take job Resource files are packaged into zip file
- zip jobs.zip start.job step1.job step2.job finish.job
3. adopt azkaban Of web Management platform creation project And upload job Of zip package
Java Operational tasks
1. To write java Program
import java.io.FileOutputStream;
import java.io.IOException;
public class AzkabanTest {
public void run() throws IOException {
FileOutputStream fos = new FileOutputStream("/usr/local/azkaban/t3test.txt");
fos.write("this is a java progress".getBytes());
fos.close();
}
public static void main(String[] args) throws IOException {
AzkabanTest azkabanTest = new AzkabanTest();
azkabanTest.run();
}
}
2.java The program is written as jar
- Generate class file
- jar -cvf AzkabanTest.jar AzkabanTest.class
- Move to linux Inside
- A test run
java -jar AzkabanTest.jar
3. To write job
- vim java.job
type=javaprocess
java.class=AzkabanTest
classpath=/usr/local/azkaban/jobs/t3/jar/*
4. take job become involved zip
- zip java.zip java.job
5.web operation
HDFS Operational tasks
1. establish job Description file
- vim fs.job
type=command
command=/usr/local/hadoop-2.7.1/bin/hadoop fs -mkdir /azkaban
2. take job Resource files are packaged into zip file
zip fs.zip fs.job
3.web operation
MapReduce Mission
1. establish job Description file
- vim mapreduce.job
#mapreduce job
type=command
command=/usr/local/hadoop-2.7.1/bin/hadoop jar /usr/local/hadoop-2.7.1/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar wordcount /wordcountinput /wordcountinputazkaban
2. take job Resource files are packaged into zip file
zip mapreduce.zip mapreduce.job
3.web operation
边栏推荐
- The perfect car for successful people: BMW X7! Superior performance, excellent comfort and safety
- 问题解决:AttributeError: ‘NoneType‘ object has no attribute ‘append‘
- Sqoop命令
- Elfk deployment
- Character painting, I use characters to draw a Bing Dwen Dwen
- Linux安装Redis
- Apache Web page security optimization
- 【LeetCode】222. The number of nodes of a complete binary tree (2 mistakes)
- 使用druid連接MySQL數據庫報類型錯誤
- Medusa installation and simple use
猜你喜欢
openresty ngx_ Lua execution phase
[uc/os-iii] chapter 1.2.3.4 understanding RTOS
ELFK部署
问题解决:AttributeError: ‘NoneType‘ object has no attribute ‘append‘
Qrcode: generate QR code from text
Scientific research: are women better than men?
Character painting, I use characters to draw a Bing Dwen Dwen
Write a thread pool by hand, and take you to learn the implementation principle of ThreadPoolExecutor thread pool
Yuan universe also "real estate"? Multiple second-hand trading websites block metauniverse keywords
Exploration of short text analysis in the field of medical and health (II)
随机推荐
Simple use of devtools
ASP. Net core 6 framework unveiling example demonstration [01]: initial programming experience
Application and Optimization Practice of redis in vivo push platform
Eight days of learning C language - while loop (embedded) (single chip microcomputer)
Bert fine tuning skills experiment
【LeetCode】106. Construct binary tree from middle order and post order traversal sequence (wrong question 2)
The perfect car for successful people: BMW X7! Superior performance, excellent comfort and safety
tuple and point
Scientific research: are women better than men?
Devtools的简单使用
El select, El option drop-down selection box
Design and practice of kubernetes cluster and application monitoring scheme
腾讯云,实现图片上传
The phenomenology of crypto world: Pioneer entropy
Watch the online press conference of tdengine community heroes and listen to TD hero talk about the legend of developers
2021 Li Hongyi machine learning (1): basic concepts
Cut! 39 year old Ali P9, saved 150million
Leetcode takes out the least number of magic beans
Azkaban实战
Write a thread pool by hand, and take you to learn the implementation principle of ThreadPoolExecutor thread pool