Text processing tool in shell, cut [option parameter] filename Description: the default separator is the built-in variable of tab, awk [option parameter] '/pattern1/{action1}filename and awk

1. Text processing tools

1.1cut

cut The job of “ cut ”, Specifically, it is used to cut data in the file .cut Command to cut bytes from each line of a file 、 Characters and fields and put these bytes 、 Character and field output .

1.1.1 Basic usage

cut [ Option parameters ] filename
explain ： The default separator is tab

1.1.2 Option parameter description

Option parameters	function
-f	Column number , Extract which column
-d	Separator , Splits columns according to the specified separator , The default is tabs “\t”
-c	Cut by character Add after n It means which column to take such as -c 1

1.1.3 Case study

1.1.3.1 Data preparation

[[email protected] scripts]# vim cut_test.txt

1.1.3.2 cutting cut_test.txt First column

[[email protected] scripts]# cut -d " " -f 1 cut_test.txt

1.1.3.3 cutting cut_test.txt second 、 The three column

[[email protected] scripts]# cut -d " " -f 2,3 cut_test.txt

1.1.3.4 stay cut_test.txt Cut out in file nan

[[email protected] scripts]# cat cut_test.txt | grep nan | cut -d " " -f 1

1.1.3.5 want passwd China and Israel bash First at the end , 6、 ... and , Seven columns

[[email protected] scripts]# cat /etc/passwd | grep bash$ | cut -d “:” -f 1,6,7

If there are too many columns, you can't 1,2,3,4,5,6,7,8,9,10… Wait , It can be used “-” Come on , Let's introduce

1.1.3.6 want passwd China and Israel bash At the end of the paragraph 1-4 Column

[[email protected] scripts]# cat /etc/passwd | grep bash$ | cut -d “:” -f 1-4

1.1.3.7 want passwd China and Israel bash At the end of the paragraph 4 All columns after column

[[email protected] scripts]# cat /etc/passwd | grep bash$ | cut -d “:” -f 4-

1.1.3.8 want passwd China and Israel bash At the end of the paragraph 4 All columns before column

[[email protected] scripts]# cat /etc/passwd | grep bash$ | cut -d “:” -f -4

1.1.3.9 Selection system PATH A variable's value , The first 2 individual “：” All paths after start ：

[[email protected] scripts]# echo $PATH | cut -d “:” -f 3-

1.1.3.10 cutting ifconfig Post print IP Address

[[email protected] scripts]# ifconfig ens33 | grep netmask | cut -d " " -f 10

Why 10 Well , because inet There is 8 A space

1.2awk

awk and gawk It's the same ,awk yes gawk A soft connection of
A powerful text analysis tool , Read the document line by line , Slice each line with a space as the default separator , The cut part is analyzed again .

1.2.1 Basic usage

awk [ Option parameters ] ‘/pattern1/{action1} /pattern2/{action2}…’ filename
pattern： Express awk What to look for in the data , It's a matching pattern
action： A series of commands executed when a match is found

1.2.2 Option parameter description

Option parameters	function
-F	Specify the input file separator
-v	Assign a user-defined variable

1.2.3 Case study

1.2.3.1 Search for passwd Document to root All lines at the beginning of the keyword , And output the 7 Column

1.2.3.2 Search for passwd Document to root All lines at the beginning of the keyword , And output the 1 Column and the first 7 Column , In the middle to “,” Division of no.

use cut You cannot change what is used to separate the output , and awk Sure , remember , Need to use “” The parcel

1.2.3.3 Display only /etc/passwd The first and seventh columns of , Comma separated , And add column names before all rows user,shell Add on last line "end of file"

[[email protected] scripts]# cat /etc/passwd | awk -F “:” ‘BEGIN{print “user,shell”}{print $1","$7} END{print “end of file”}’

1.2.3.4 take passwd Users in files id Increase in numerical value 1 And the output

because {} There are code blocks , You can change it directly inside

If {} There is too much code in the code block , It will be troublesome to change , Therefore use -v, In this way, when you change the number outside, the code inside the code block will be changed

1.2.4awk Built in variables for

Variable	explain
FILENAME	file name
NR	Number of records read （ Line number ）
NF	The number of fields in the browsing record （ After cutting , Number of columns ）

1.2.4.1 Statistics passwd file name , Line number of each line , Columns per row

[[email protected] scripts]# awk -v i=1 -F “:” ‘{print “ file name ：” FILENAME “ Line number ：” NR “ Number of columns ：” NF}’ /etc/passwd

1.2.4.2 Inquire about ifconfig The line number of the empty line in the command output result

[[email protected] scripts]# ifconfig | grep -n ^$

Output with the previous method will have ":", But the result is not desired , use awk solve

[[email protected] scripts]# ifconfig | awk ‘/^$/ {print NR}’

You can also add things at will

[[email protected] scripts]# ifconfig | awk ‘/^$/ {print " Blank line ："NR}’

1.2.4.3 cutting IP

Use cut when -f After that, you need to count the pile of spaces in front

Use awk after , There is no need to count the first pile of spaces