当前位置：网站首页>Awk getting started to proficient series - awk quick start

Awk getting started to proficient series - awk quick start

2022-07-04 12:03:00 【Lian Zhibo】

brief introduction

AWK Is an excellent text processing tool ,Linux And Unix One of the most powerful data processing engines available in the environment . This programming and data manipulation language （ Its name comes from its founder Alfred · AI Hou 、 Peter · Weinberg and Brian · The first letter of colingham's surname ） The greatest function of depends on one's knowledge .awk A new version of the improved version nawk,gawk, Now default linux The daily use of the system is gawk, You can use the command to view the awk The source of the （ls -l /bin/awk ）

A simple example

      
      
       
       # Create a file 
       
       
vim awk.txt
       
       
Beth 4.00 0
       
       
Dan  3.75 0
       
       
Kathy 4.00 10
       
       
Mark 5.00 20
       
       
Mary 5.50 22
       
       
Susie 4.25 18
       
       
 Field explanation :  full name    Hourly wage    Working hours 
       
       

       
       
# Print each employee's name and compensation 
       
       
awk '$3>0 {print $1,$2*$3}' awk.txt
       
       
$3>0  Is the mode 
       
       
print $1,$2*$3 It's action 
       
       
# Want to know which employees are lazy 
       
       
awk '$3==0 {print $1}' awk.txt
      
      
      
      
       
       1.
       
       2.
       
       3.
       
       4.
       
       5.
       
       6.
       
       7.
       
       8.
       
       9.
       
       10.
       
       11.
       
       12.
       
       13.
       
       14.
       
       15.
       
       16.

Execution result diagram :

AWK Entry to Mastery Series ——awk Quick start _shell

AWK Entry to Mastery Series ——awk Quick start _ Regular expressions _02

AWK Entry to Mastery Series ——awk Quick start _ Regular expressions _03

AWK Program structure

AWK Program execution flow

awk The basic operation of is in a sequence of input lines , Scan each line in succession , Search can be " Pattern " matching (match) The line of . If it matches, execute " action ", Continue until all inputs are read

Pattern - Action analysis

      
      
       
       (1) Pattern - Actions exist  awk '$3==0 {print $1}' awk.txt
       
       
(2) Mode of existence , Action doesn't exist  awk '$3==0' awk.txt
       
       
(3) Mode does not exist , Action exists  awk '{print $1}' awk.txt
       
       
(4) Neither can exist ( Can't run )
      
      
      
      
       
       1.
       
       2.
       
       3.
       
       4.

AWK The running format of the command

      
      
       
       # Followed by documents 
       
       
awk '$3==0 {print $1}' awk.txt   It's followed by a file 
       
       
awk '$3==0 {print $1}' awk.txt awk02.txt   Followed by two files 
       
       
# Waiting for input 
       
       
awk '$3==0 {print $1}'  There is no file after it , Wait for input before judging 
       
       
# take awk Put the program into the file 
       
       
 cat program
       
       
 $3==0 {print $1}
       
       
  Carry out orders : awk -f program awk.txt
      
      
      
      
       
       1.
       
       2.
       
       3.
       
       4.
       
       5.
       
       6.
       
       7.
       
       8.
       
       9.

AWK The output format of

data type

Numbers and strings

Rows and fields

      
      
       
       awk Read one line at a time from its input , Break the rows into fields ( By default, the field is regarded as a sequence of non blank characters ).
       
       
 The first field of the current input line is called $1, The second is $2, By analogy , A whole line is written as $0, The number of fields in each row may be different .
      
      
      
      
       
       1.
       
       2.

Case study

      
      
       
       # Print every line 
       
       
awk '{print}' awk.txt  or  awk '{print $0}' awk.txt
       
       
# Print some fields 
       
       
awk '{print $1,$3}' awk.txt
       
       
# Print the number of fields per line ( Built in variables NF)
       
       
awk '{print NF}' awk.txt
       
       
# Print the first and last fields 
       
       
awk '{print $1,$NF}' awk.txt
       
       
# Calculate and print 
       
       
awk '{print $1,$2 * $3}' awk.txt
       
       
# Print line number (NR)
       
       
awk '{print NR,$0}' awk.txt
       
       
# Splice strings and fields 
       
       
awk '{print $1," Today's income is ",$2 * $3}' awk.txt
       
       
# Format output 
       
       
awk '{ printf("%s  Today's income is  $%.2f\n",$1,$2*$3) }' awk.txt
       
       
 Fixed width output 
       
       
awk '{ printf("%-8s  Today's income is  $%6.2f\n",$1,$2*$3) }' awk.txt
       
       
 Output sort 
       
       
awk '{ printf("%6.2f,%-8s  Today's income is  $%6.2f\n",$2*$3,$1,$2*$3) }' awk.txt |sort -nk3 -t,
      
      
      
      
       
       1.
       
       2.
       
       3.
       
       4.
       
       5.
       
       6.
       
       7.
       
       8.
       
       9.
       
       10.
       
       11.
       
       12.
       
       13.
       
       14.
       
       15.
       
       16.
       
       17.
       
       18.
       
       19.
       
       20.

AWK Pattern matching

Single mode

      
      
       
       # The hourly wage is greater than 5 The record of 
       
       
awk '$2>5 {print $0}' awk.txt
       
       
# Pay more than 50 The employees' 
       
       
awk '$2*$3>50 {print $1,$2*$3}' awk.txt
       
       
# The query name is Mark The record of 
       
       
awk '$1=="Mark" {print $0}' awk.txt
       
       
# Regular expression matching names with Mar The record of 
       
       
awk '/Mar/ {print $0}' awk.txt
      
      
      
      
       
       1.
       
       2.
       
       3.
       
       4.
       
       5.
       
       6.
       
       7.
       
       8.

Pattern combination

      
      
       
       # Print those $2 At least for 4, perhaps $3 At least for 20 The line of 
       
       
awk '$2>=4||$3>=20 {print $0}' awk.txt
       
       
awk '!($2<4&&$3<20) {print $0}' awk.txt
       
       
# Print those $2 At least for 4, also $3 At least for 20 The line of 
       
       
awk '$2>=4 && $3>=20 {print $0}' awk.txt
      
      
      
      
       
       1.
       
       2.
       
       3.
       
       4.
       
       5.

BEGIN And END

      
      
       
        Special mode  BEGIN  Matched before the first line of the first input file , END  Match after the last line of the last input file is processed .
       
       
awk 'BEGIN {*********} END{***********}'
       
       
awk 'BEGIN {print "NAME  RATE   HOURS"} {print} END{print "END"}' awk.txt
       
       
awk 'BEGIN {print "NAME  RATE   HOURS";print "------"} {print} END{print "------";print "END"}' awk.txt
      
      
      
      
       
       1.
       
       2.
       
       3.
       
       4.

use AWK Calculation

Calculate the sum

      
      
       
       # The working hours exceed 15 Total number of employees per hour 
       
       
awk '$3>15 {emp=emp+1} END{print emp," The working hours exceed 15 Number of employees per hour "}' awk.txt
      
      
      
      
       
       1.
       
       2.

Calculate the average

      
      
       
       # Calculate the average salary of employees 
       
       
awk '{pay=pay+$2*$3} END{print NR," The total number of employees ";print " Total wage ",pay; print " Average wage ",pay/NR}' awk.txt
      
      
      
      
       
       1.
       
       2.

Find the maximum

      
      
       
       # Find the employee with the highest hourly wage 
       
       
awk '$2 >maxrate {maxrate=$2;maxemp=$1} END{print " The employee with the highest hourly wage is :",maxemp," Wages are :",maxrate}' awk.txt
      
      
      
      
       
       1.
       
       2.

Print the last line

      
      
       
       awk '{last=$0} END{print last}' awk.txt
      
      
      
      
       
       1.

String splicing

      
      
       
       #name Add a space between 
       
       
awk '{names=names $1 " "} END{print names}' awk.txt
      
      
      
      
       
       1.
       
       2.

AWK Entry to Mastery Series ——awk Quick start _ Regular expressions _04

Built-in functions

      
      
       
       #length  Find the string length 
       
       
 Calculate the length of the name 
       
       
awk '{print $1,length($1)}' awk.txt
       
       

       
       
# Count the lines of text , Total number of fields , Total bytes 
       
       
awk '{nc=nc+length($0)+1;nw=nw+NF} END{print NR,"lines,",nw,"words,",nc,"characters"}' awk.txt
      
      
      
      
       
       1.
       
       2.
       
       3.
       
       4.
       
       5.
       
       6.

Flow control statement

if-else sentence

      
      
       
       # Find hourly wages more than  $6.00  Total and average remuneration of employees 
       
       
awk '$2>6 {n=n+1;pay=pay+$2*$3} END{if(n>0) print n,"employees,total pay is",pay,"average pay is",pay/n;else print "not exit"}' awk.txt
      
      
      
      
       
       1.
       
       2.

while sentence

      
      
       
        Calculation 1 To 100 The sum of the 
       
       
awk 'BEGIN{ test=100; total=0; while(i<=test) {     total+=i;     i++; } print total; }' 5050
       
       

       
       
#shell Script 
       
       
#!/bin/bash
       
       
total=0
       
       
i=0
       
       
while [ $i -le 100 ]
       
       
do
       
       
let total+=$i
       
       
let i++
       
       
done
       
       
echo $total
      
      
      
      
       
       1.
       
       2.
       
       3.
       
       4.
       
       5.
       
       6.
       
       7.
       
       8.
       
       9.
       
       10.
       
       11.
       
       12.
       
       13.

for sentence

      
      
       
       # Calculation 1 To 100 The sum of 
       
       
awk 'BEGIN{ total=0; for(i=0;i<=100;i++) {     total+=i; } print total; }'
      
      
      
      
       
       1.
       
       2.

Array

      
      
       
       # Print each line upside down 
       
       
awk '{line[NR] = $0} END {i=NR; while (i>0){ print line[i];i=i-1}}' awk.txt
       
       
awk '{line[NR] = $0} END{for(i=NR;i>0;i--){print line[i]}}' awk.txt
      
      
      
      
       
       1.
       
       2.
       
       3.

AWK Production cases

      
      
       
       # The total number of rows entered 
       
       
awk 'END{print NR}' awk.txt
       
       
# Print page 2 That's ok 
       
       
awk 'NR==2 {print $0}' awk.txt
       
       
# Print the last field of each line 
       
       
awk '{print $NF}' awk.txt
       
       
# Print the last field of the last line 
       
       
awk '{field=$NF} END{print field}' awk.txt
       
       
# The number of print fields is more than  2  Input lines of 
       
       
awk 'NF>2 {print $0}' awk.txt
       
       
# Print the last field with a value greater than  4  Input line for 
       
       
awk '$NF>4{print $0}' awk.txt
      
      
      
      
       
       1.
       
       2.
       
       3.
       
       4.
       
       5.
       
       6.
       
       7.
       
       8.
       
       9.
       
       10.
       
       11.
       
       12.