Azkaban actual combat

A single job Case study

1. establish job Description file

• vim job1.job

#first.job
type=command
command=echo 'this is my first job'

2. take job Resource files are packaged into zip file

• install zip,unzip command

yum install -y unzip zip

• Compress

zip job1.zip job1.job

• Azkaban Uploaded workflow file Only support xxx.zip file ,zip Should include xxx.job Files and any files needed to run the job （ file name The suffix must be .job, Otherwise, I can't recognize ）, The job name must be unique in the project

3. adopt azkaban Of web Management platform creation project And upload job Of zip package

• establish project
  
• Upload zip package
  
• Start execution of the job
  
  
  
• Job Successful implementation
  
• Click to see job journal
  

many job Workflow case

1. establish job Description file

• first job：start.job

vim start.job
	#start.job
	type=command
	command=touch /usr/local/azkaban/jobs/t2test.txt

• the second job：step1.job rely on start.job

vim step1.job
	#step1.job
	type=command
	dependencies=start
	command=echo "this is step1 job"

• Third job：step2.job rely on start.job

vim step2.job
	#step2.job
	type=command
	dependencies=start
	command=echo "this is step2 job"

• The fourth one job：finish.job rely on step1.job and step2.job

vim finish.job
	#finish.job
	type=command
	dependencies=step1,step2
	command=echo "this is finish job"

2. take job Resource files are packaged into zip file

• zip jobs.zip start.job step1.job step2.job finish.job

3. adopt azkaban Of web Management platform creation project And upload job Of zip package

Java Operational tasks

1. To write java Program

import java.io.FileOutputStream;
import java.io.IOException;

public class AzkabanTest {
    
	public void run() throws IOException {
    
		FileOutputStream fos = new FileOutputStream("/usr/local/azkaban/t3test.txt");
		fos.write("this is a java progress".getBytes());
		fos.close();
    }

	public static void main(String[] args) throws IOException {
    
		AzkabanTest azkabanTest = new AzkabanTest();
		azkabanTest.run();
	}
}

2.java The program is written as jar

• Generate class file
  
• jar -cvf AzkabanTest.jar AzkabanTest.class
  
• Move to linux Inside
  
• A test run

java -jar AzkabanTest.jar

3. To write job

• vim java.job

type=javaprocess
java.class=AzkabanTest
classpath=/usr/local/azkaban/jobs/t3/jar/* 

4. take job become involved zip

• zip java.zip java.job

5.web operation

HDFS Operational tasks

1. establish job Description file

• vim fs.job

type=command
command=/usr/local/hadoop-2.7.1/bin/hadoop fs -mkdir /azkaban

2. take job Resource files are packaged into zip file

zip fs.zip fs.job

3.web operation

MapReduce Mission

1. establish job Description file

• vim mapreduce.job

#mapreduce job
type=command
command=/usr/local/hadoop-2.7.1/bin/hadoop jar /usr/local/hadoop-2.7.1/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar wordcount /wordcountinput /wordcountinputazkaban

2. take job Resource files are packaged into zip file

zip mapreduce.zip mapreduce.job

3.web operation