Awk getting started to proficient series - awk quick start

brief introduction

AWK Is an excellent text processing tool ,Linux And Unix One of the most powerful data processing engines available in the environment . This programming and data manipulation language （ Its name comes from its founder Alfred · AI Hou 、 Peter · Weinberg and Brian · The first letter of colingham's surname ） The greatest function of depends on one's knowledge .awk A new version of the improved version nawk,gawk, Now default linux The daily use of the system is gawk, You can use the command to view the awk The source of the （ls -l /bin/awk ）

A simple example

# Create a file 
vim awk.txt
Beth 4.00 0
Dan  3.75 0
Kathy 4.00 10
Mark 5.00 20
Mary 5.50 22
Susie 4.25 18
 Field explanation :  full name    Hourly wage    Working hours 

# Print each employee's name and compensation 
awk '$3>0 {print $1,$2*$3}' awk.txt
$3>0  Is the mode 
print $1,$2*$3 It's action 
# Want to know which employees are lazy 
awk '$3==0 {print $1}' awk.txt

Execution result diagram :

AWK Program structure  

• AWK Program execution flow

  awk The basic operation of is in a sequence of input lines ,  Scan each line in succession ,  Search can be " Pattern " matching (match)  The line of . If it matches, execute " action ", Continue until all inputs are read 

•   Pattern - Action analysis

(1) Pattern - Actions exist  awk '$3==0 {print $1}' awk.txt
(2) Mode of existence , Action doesn't exist  awk '$3==0' awk.txt
(3) Mode does not exist , Action exists  awk '{print $1}' awk.txt
(4) Neither can exist ( Can't run )

AWK The running format of the command

# Followed by documents 
awk '$3==0 {print $1}' awk.txt   It's followed by a file 
awk '$3==0 {print $1}' awk.txt awk02.txt   Followed by two files 
# Waiting for input 
awk '$3==0 {print $1}'  There is no file after it , Wait for input before judging 
# take awk Put the program into the file 
 cat program
 $3==0 {print $1}
  Carry out orders : awk -f program awk.txt

 AWK The output format of

• data type ​

• ​ Numbers and strings 

• Rows and fields

awk Read one line at a time from its input , Break the rows into fields ( By default, the field is regarded as a sequence of non blank characters ).
 The first field of the current input line is called $1, The second is $2, By analogy , A whole line is written as $0, The number of fields in each row may be different .

•   Case study

# Print every line 
awk '{print}' awk.txt  or  awk '{print $0}' awk.txt
# Print some fields 
awk '{print $1,$3}' awk.txt
# Print the number of fields per line ( Built in variables NF)
awk '{print NF}' awk.txt
# Print the first and last fields 
awk '{print $1,$NF}' awk.txt
# Calculate and print 
awk '{print $1,$2 * $3}' awk.txt
# Print line number (NR)
awk '{print NR,$0}' awk.txt
# Splice strings and fields 
awk '{print $1," Today's income is ",$2 * $3}' awk.txt
# Format output 
awk '{ printf("%s  Today's income is  $%.2f\n",$1,$2*$3) }' awk.txt
 Fixed width output 
awk '{ printf("%-8s  Today's income is  $%6.2f\n",$1,$2*$3) }' awk.txt
 Output sort 
awk '{ printf("%6.2f,%-8s  Today's income is  $%6.2f\n",$2*$3,$1,$2*$3) }' awk.txt |sort -nk3 -t,

 AWK Pattern matching

•   Single mode

# The hourly wage is greater than 5 The record of 
awk '$2>5 {print $0}' awk.txt
# Pay more than 50 The employees' 
awk '$2*$3>50 {print $1,$2*$3}' awk.txt
# The query name is Mark The record of 
awk '$1=="Mark" {print $0}' awk.txt
# Regular expression matching names with Mar The record of 
awk '/Mar/ {print $0}' awk.txt

•   Pattern combination  

# Print those $2 At least for 4, perhaps $3 At least for 20 The line of 
awk '$2>=4||$3>=20 {print $0}' awk.txt
awk '!($2<4&&$3<20) {print $0}' awk.txt
# Print those $2 At least for 4, also $3 At least for 20 The line of 
awk '$2>=4 && $3>=20 {print $0}' awk.txt

•  BEGIN And END

 Special mode  BEGIN  Matched before the first line of the first input file , END  Match after the last line of the last input file is processed .
awk 'BEGIN {*********} END{***********}'
awk 'BEGIN {print "NAME  RATE   HOURS"} {print} END{print "END"}' awk.txt
awk 'BEGIN {print "NAME  RATE   HOURS";print "------"} {print} END{print "------";print "END"}' awk.txt

  use AWK Calculation

• Calculate the sum

 # The working hours exceed 15 Total number of employees per hour
awk '$3>15 {emp=emp+1} END{print emp," The working hours exceed 15 Number of employees per hour "}' awk.txt

•   Calculate the average

 # Calculate the average salary of employees
awk '{pay=pay+$2*$3} END{print NR," The total number of employees ";print " Total wage ",pay; print " Average wage ",pay/NR}' awk.txt

•   Find the maximum

 # Find the employee with the highest hourly wage
awk '$2 >maxrate {maxrate=$2;maxemp=$1} END{print " The employee with the highest hourly wage is :",maxemp," Wages are :",maxrate}' awk.txt

•   Print the last line

 awk '{last=$0} END{print last}' awk.txt

  String splicing

#name Add a space between
awk '{names=names $1 " "} END{print names}' awk.txt

 

  Built-in functions

 #length Find the string length
Calculate the length of the name
awk '{print $1,length($1)}' awk.txt

# Count the lines of text , Total number of fields , Total bytes
awk '{nc=nc+length($0)+1;nw=nw+NF} END{print NR,"lines,",nw,"words,",nc,"characters"}' awk.txt

  Flow control statement

• if-else sentence

 # Find hourly wages more than $6.00 Total and average remuneration of employees
awk '$2>6 {n=n+1;pay=pay+$2*$3} END{if(n>0) print n,"employees,total pay is",pay,"average pay is",pay/n;else print "not exit"}' awk.txt

•  while sentence

  Calculation 1 To 100 The sum of the
awk 'BEGIN{ test=100; total=0; while(i<=test) { total+=i; i++; } print total; }' 5050

#shell Script
#!/bin/bash
total=0
i=0
while [ $i -le 100 ]
do
let total+=$i
let i++
done
echo $total

•  for sentence

 # Calculation 1 To 100 The sum of
awk 'BEGIN{ total=0; for(i=0;i<=100;i++) { total+=i; } print total; }'

•   Array

 # Print each line upside down
awk '{line[NR] = $0} END {i=NR; while (i>0){ print line[i];i=i-1}}' awk.txt
awk '{line[NR] = $0} END{for(i=NR;i>0;i--){print line[i]}}' awk.txt

 AWK Production cases

 # The total number of rows entered
awk 'END{print NR}' awk.txt
# Print page 2 That's ok
awk 'NR==2 {print $0}' awk.txt
# Print the last field of each line
awk '{print $NF}' awk.txt
# Print the last field of the last line
awk '{field=$NF} END{print field}' awk.txt
# The number of print fields is more than 2 Input lines of
awk 'NF>2 {print $0}' awk.txt
# Print the last field with a value greater than 4 Input line for
awk '$NF>4{print $0}' awk.txt