Shell learning tutorial (super detailed and complete)

In order to facilitate future work and review , Spit out blood, sort out records, learn shell Script notes , Reading this article requires linux Familiar with the system , I hope that's helpful ！

List of articles

• • One 、 What is? shell？
  • • Why learn and use shell？
  • Two 、shell The classification of
  • • Bash Common shortcut key
    • I / O redirection
    • • linux Standard input and output
      • Input redirection
      • Output redirection
      • • /dev/null file
      • Multiple commands are executed sequentially
    • shell Script execution
  • 3、 ... and 、shell Variable
  • • 1. Naming rules for variables ：
    • • shell Special symbols in
    • 2. Classification of variables ：
    • • 2.1 User defined variables ：
      • • 2.1.1 Variable definitions
        • 2.1.2 Variable call
        • 2.1.3 Variable view
        • 2.1.4 Variable deletion
      • 2.2 environment variable ：
      • • 2.2.1 Environment variable Settings
        • 2.2.2 Query and delete environment variables
        • 2.2.3 System default environment variable
      • 2.3 Position parameter variable ：
      • 2.4 Predefined variables ：
      • 3. A read-only variable ：
      • 4. Accept keyboard input ：
  • Four 、shell Operator
  • • 1. Arithmetic operator
    • 2. Relational operator
    • 3. Logical operators
    • 4. String operators
    • 5. File test operators （ a key ）
  • 5、 ... and 、 Process control
  • • 1. if conditional
    • • 1.1 Single branch if Conditions
      • 1.2 Double branch if Conditional statements
      • 1.3 Multiple branches if Conditional statements
    • 2. Multiple branches case Conditional statements
    • 3. for loop
    • 4. while loop
    • 5. until loop
    • 6. function
    • 7. Special process control statements
    • • 7.1 exit sentence
      • 7.2 break sentence
      • 7.3 break sentence
  • 6、 ... and 、 Character interception 、 Replace and process commands
  • • Regular expressions
    • 6.1 Character interception 、 Replace the command
    • • 6.1.1 cut Column extraction command
      • 6.1.2 awk Programming
      • • 6.1.2.1 awk summary
        • 6.1.2.2 printf Format output
        • 6.1.2.3 awk Basic use
        • 6.1.2.4 awk Conditions
        • • BEGIN
          • END
          • Relational operator
          • Regular expressions
        • 6.1.2.5 awk Built-in variables
        • • awk Common statistical examples
        • 6.1.2.6 awk Process control
        • 6.1.2.7 awk function
        • 6.1.2.8 awk Invoke script in
      • 6.1.3 sed Text selection 、 Replace 、 Delete 、 New orders
      • • • 6.1.3.1 Extract row data
          • 6.1.3.2 Delete row data
          • 6.1.3.3 Append insert row data
          • 6.1.3.4 Replace row data
          • 6.1.3.5 String substitution
    • 6.2 Character processing commands
    • • 6.2.1 sort Sort order
      • 6.2.2 uniq Cancel duplicate lines
      • 6.2.3 wc Statistical orders

One 、 What is? shell？

shell It's a use. C A program written in a language , It is used by users Linux The bridge .Shell It's a command language , Another programming language .

Shell It's an application , This application provides an interface , Users access the services of the operating system kernel through this interface .

Why learn and use shell？

Shell It's a built-in script , Program development is very efficient , Depending on powerful commands, you can quickly complete the development task （ The batch ） Grammar is simple , The code is easy to write , Easy to learn

---

Two 、shell The classification of

cat /etc/shells
stay linux There are many types of shell, Different shell With different functions ,shell It also determines the syntax of the functions in the script ,Linux In the default shell yes / b a s h / b a s h （ heavy spot ） color{#FF3030}{/bash/bash（ a key ）} /bash/bash（ a key ）, The popular shell Yes ash、bash、ksh、csh、zsh etc. , Different shell They all have their own characteristics and uses .

Writing specifications ：

#!/bin/bash				[ Specifies to inform the system of the shell Interpreter ]
Shell Related instructions 

File naming conventions ：

file name .sh .sh yes linux Next bash shell Default suffix for

Bash Common shortcut key

Shortcut key

effect

ctrl+A

Move the cursor to the beginning of the command line . If the command we enter is too long , Use... When you want to move the cursor to the beginning of the command line .

ctrl+E

Move the cursor to the end of the command line .

ctrl+C

Forcibly terminate the current command .

ctrl+L

Clear the screen , amount to clear command .

ctrl+U

Delete or cut the command before the cursor . I entered a very long command , Do not use the backspace key to delete character by character , It's more convenient to use this shortcut

ctrl+K

Delete or cut the content after the cursor .

ctrl+Y

Paste ctrl+U or ctul+K Cut content .

ctrl+R

Search in history command , Press down ctrl+R after , The search interface will appear , Just enter the search content , It's going to search through history commands .

ctrl+D

Exit the current terminal .

ctrl+Z

Pause , And put it backstage . This shortcut key involves the content of work management , We will introduce in detail in the chapter of system management .

ctrl+S

Pause screen output .

ctrl+Q

Restore screen output .

I / O redirection

linux Standard input and output

equipment

Device name

File descriptor

type

keyboard

/dev/stdin

0

The standard input

Monitor

/dev/stdout

1

standard output

Monitor

/dev/stderr

2

Standard error output

Input redirection

Input redirection ： Refers to not using the standard input port provided by the system , And make a new designation . In other words , Input redirection is to input files without using the standard input port , Instead, the specified file is used as the standard input device .（ The simple understanding of redirection is to use “<” To modify the standard input device ）

type

Symbol （ grammar ）

function

The standard input

command < file 1

Order the file to be 1 As a standard input device

The identifier qualifies the input

command << identifier

The command reads content into standard input , Until I met “ identifier ” Up to the delimiter

I / O redirection （ Use at the same time ）

command < file 1 > file 2

Order the file to be 1 As standard input , The file 2 As standard output .

Output redirection

Output redirection ：（ Generally speaking , Redirecting output is to write the file information to be output into a file , Instead of outputting the file information to be output to the console （ display ）, stay linux in , The default standard output device is the console （ Or display ）, The information output by the user will be displayed to the console by default

& Represents all files , Whether the file is right or wrong ,1 Represents the standard output file ,2 Represents standard error output .

type

Symbol

effect

Mark output redirection

command > file

In the form of coverage , Output the correct output of the command to the specified file or device

Mark output redirection

command >> file

In addition , Output the correct output of the command to the specified file or device

Standard error output redirection

Wrong command 2 > file

In the form of coverage , Output the command error output to the specified file or device

Standard error output redirection

Wrong command 2 >> file

In addition , Output the command error output to the specified file or device

Correct output and error output are saved at the same time

command > file 2>&1

In the form of coverage , Save both the correct output and the wrong output in the same file .

Correct output and error output are saved at the same time

command >> file 2>&1

In an additional way , Save both the correct output and the wrong output in the same file .

Correct output and error output are saved at the same time

command &> file

In the form of coverage , Save both the correct output and the wrong output in the same file .

Correct output and error output are saved at the same time

command &>> file

In an additional way , Save both the correct output and the wrong output in the same file .

Correct output and error output are saved at the same time

command >> file 1 2>> file 2

Append the correct output to the file 1 in , Append the wrong output to the file 2 in .

/dev/null file

If you want to execute a command , But I don't want to display the output on the screen , So you can redirect the output to **/dev/null** in

[[email protected] ~]$  command > dev/null

Multiple commands are executed sequentially

Multiple command executors

effect

Format

;

command 1 ; command 2

Multiple commands are executed in sequence , There is no logical connection between commands

&&

command 1 && command 2

When ordered 1 The right to perform （ = 0 ） , be life Make 2 only Meeting Of board That's ok When life Make 1 Of board That's ok No just indeed （ =0）, Order 2 Will execute When ordered 1 Incorrect execution （ =0）, Order 2 Will execute the command 1 Incorrect execution （≠0）, Order 2 Not execute

ll

command 1 || command 2

When ordered 1 Incorrect execution （ ≠ 0 ） , be life Make 2 only Meeting Of board That's ok When life Make 1 just indeed Of board That's ok （ ≠0）, Order 2 Will execute the command 1 The right to perform （ =0）, Order 2 Will execute the command 1 The right to perform （=0）, Order 2 Not execute

shell Script execution

[[email protected] ~]$ vim test.sh
#!/bin/bash
echo “hello world”

Two ways to execute shell Script

The first one is ： Add execution rights to the file

[[email protected] ~]$ chmod u+x test.sh
[[email protected] ~]$ ./test.sh  # Absolute path or relative path execution 

The second kind （ understand ）： adopt Bash Call the execution script

[[email protected] ~]$ bash test.sh

---

3、 ... and 、shell Variable

What is a variable ？
In a script cycle , The quantity whose value can be changed is a variable .

1. Naming rules for variables ：

When you define variables , There are some rules to follow :

1. Only English letters can be used for naming , Numbers and underscores , The first character cannot begin with a number .

2. There can be no spaces on the left and right sides of the equal sign , You can use underscores “_”, Value of variable if there is a space , Single or double quotes are required to include . Such as :“test=“hello world!””. The contents enclosed in double quotation marks “$”,“(” And backquotes have special meanings , The contents enclosed in single quotation marks are ordinary characters .

3. You can't use punctuation , Out of commission bash Keywords in （ You can use help Command to view reserved keywords ）.

4. Environment variables are recommended to be capitalized , Easy to distinguish

5. If you need to increase the value of a variable , Then you can stack the variable values . But variables need to be enclosed in double quotes "$changeThe\; amountname"oruse${ Variable name } Contains the variable name .

   [[email protected] ~]$ test=123
   [[email protected] ~]$ test="$test"456[root@localhost]$ echo KaTeX parse error: Expected 'EOF', got '#' at position 13: test 123456 #̲ Superimposed variable test, The variable value becomes … test=$test789[root@localhost]$ echo $test
   123456789
   # Superimposed variable test, Variable values are programmed 123456789

On the question of single and double quotation marks ：
Double quotation marks identify variables , Double quotes can escape （ Be similar to “*”）
Single quotation marks are unrecognized variables , Only output as is , Single quotation marks cannot be escaped

shell Special symbols in

Symbol

effect

’ ’

Single quotation marks . All special symbols in single quotation marks , Such as “$” and ”( The quotation marks ) There's no special meaning . Single quotation marks are all ordinary characters , It will output as is

“ ”

Double quotes . In double quotation marks, special symbols have no special meaning , however “$”,"`"（esc Under key ） and “” It's an exception , Have “ Call the value of the variable ”、“ Quote orders ” and “ Escape character ” The special meaning of .

· ·

The quotation marks . The contents enclosed in back quotation marks are system commands , stay Bash It will be executed first in . and ( ) do use One sample , No too PUSH commend send use () It works the same , However, it is recommended to use () It works the same , However, it is recommended to use (), Because backquotes are very easy to read wrong .

$()

The same as backquotes , Used to reference system commands .( Recommended )

()

Used when a series of commands are executed ,() The command in will be in the sub Shell Run in

{}

Used when a series of commands are executed ,{ } The command in will be in the current Shell In the implementation of . It can also be used for variable deformation and replacement .

[ ]

For testing variables .

stay Shell Script ,# The first line represents the comment .

$

The value used to call the variable , If you need to call variables name The value of , Need to use $name To get the value of a variable in the same way .

Escape character , Special symbols that follow will lose their special meaning , Change to normal characters . Such as $taketransportOut“$” Symbol , Improper doing is variable reference .

---

Single and double quotes

[[email protected] ~]$ name=sc
# Defining variables name  The value of is sc（ Is the most honest person , Super brother, i !）
[[email protected] ~]$ echo '$name'
$name
# If you use single quotation marks for output , be $name Intact output 
[[email protected] ~]$ echo "$name"
sc
# If double quotation marks are used for output , The variable... Will be output name Value  sc

[[email protected] ~]$ echo `date`
2018 year 10 month 21 Monday, Sunday 18:16:33 CST
# The command enclosed in back quotation marks will execute normally 
[[email protected] ~]$ echo '`date`'
`date`
# But if the backquote command is enclosed in single quotes , Then this command will not execute ,―date` Will be output as normal characters 
[[email protected] ~]$ echo "`date'"
2018 year 10 month 21 Monday, Sunday 18:14:21 CST
# If it is enclosed in double quotation marks , Then this command will be executed normally 

The quotation marks

[[email protected] ~]$ echo ls
ls
# If the command does not contain in backquotes , The order will not be executed , It's direct output 
[[email protected] ~]$ echo `ls`
anaconda-ks.cfginstall.loginstall.log.syslog sh test testfile
# Only include commands in backquotes , This command will be executed 
[[email protected] ~]$ echo $(date)
2018 year 10 month 21 Monday, Sunday 18:25:09 CST
# Use $( command ) It's also possible 

2. Classification of variables ：

1. User defined variables : This variable is the most common variable , The variable name and value are freely defined by the user .

2. environment variable : This kind of variable mainly stores the data related to the operating environment of the system , For example, the current login user , User's home directory , Command prompt, etc . It's not easy to understand , So do you still remember in Windows in , Multiple users can log on to the same computer , And each user can define their own desktop style and resolution , These are actually Windows Operating environment of , You can think of it as Windows Environment variables to understand . The variable name of the environment variable can be defined freely , But generally, the variable name of the environment variable that works on the system is preset by the system .

3. Position parameter variable : This variable is mainly used to pass parameters or data to the script , Variable name cannot be customized , Variable action is fixed .

4. Predefined variables : yes Bash Variables already defined in , Variable name cannot be customized , Variable action is also fixed .

Variable classification

name

effect

Content

User defined variables

Customize

Customize

Customize

User defined environment variables

Customize

Customize

Customize

The system has its own environment variables (/etc/profile)

determine

determine

Customize

Position parameter variable

determine

Customize

Customize

Predefined variables

determine

Customize

Customize

2.1 User defined variables ：

2.1.1 Variable definitions

[[email protected] ~]$ 2name="shen chao"
-bash: 2name=shen chao: command not found
# Variable names cannot start with numbers 
[[email protected] ~]$ name = "shenchao"
-bash: name: command not found
# There can be no spaces on the left and right sides of the equal sign 
[[email protected] ~]$ name=shen chao
-bash: chao: command not found
# Value of variable if there is a space , Must contain in quotation marks 

2.1.2 Variable call

[[email protected] ~]$ name="shen chao"
# Defining variables name
[[email protected] ~]$ echo $name # Call variables using   $ Variable name 
shen chao
# Output variables name Value 

2.1.3 Variable view

[[email protected] ~]$ set [ Options ]
 Options :
-u: If this option is set , An error will be reported when calling undeclared variables （ By default, there is no prompt ）
-x: If this option is set , Before the command is executed , Will output the command first 
+< Parameters > : Cancel a certain set Parameters that have been started .

[[email protected] ~]$ set
BASH=/bin/bash
… Omit part of the output …
name='shen chao'
# Use it directly set  command , Will query all variables in the system , Contains user-defined variables and environment variables 
[[email protected] ~]$ set -u
[[email protected] ~]$ echo $file
-bash: file: unbound variable
# When you set -u After the options , If you call a variable that is not set, an error will be reported . The default is no output .
[[email protected] ~]$ set -x
[[email protected] ~]$ ls
+ls --color=auto
anaconda-ks.cfginstall.loginstall.log.syslog sh tdir testtestfile
# If you set -x Options , Before each command is executed , First output the command once 

[[email protected] ~]$ set +x
# Cancel the startup x Parameters 

2.1.4 Variable deletion

[[email protected] ~]$ unset  Variable name 

2.2 environment variable ：

2.2.1 Environment variable Settings

[[email protected] ~]$  export age="18"
# Use export Declared variables are environment variables 

2.2.2 Query and delete environment variables

env Command and set Distinction of command ：
set Command to see all variables , and env The command can only view environment variables .

[[email protected] ~]$ unset gender   # Delete environment variables gender
[[email protected] ~]$ env | grep gender

2.2.3 System default environment variable

[[email protected] ~]$ env
HOSTNAME=localhost.localdomain      # Host name 
SHELL=/bin/bash                     # Current shell
TERM=linux                          # Terminal environment 
HISTSIZE=1000                       # Number of historical commands 
SSH_CLIENT=192.168.4.1594824 22     # The current operating environment is using ssh Connected , The client is recorded here ip
SSH_TTY=/dev/pts/1                  #ssh When connecting to the terminal pts/1
USER=root                           # Currently logged in users 
.......... More parameters can be used set and env Command view .............

2.3 Position parameter variable ：

Position parameter variable

effect

$n

n Is the number ,$0 At present Shell The name of the script ,$1- 9 generation surface The first One To The first Nine individual ginseng Count , Ten With On Of ginseng Count Need to be want use Big enclosed Number package contain , Such as 9 Represents the first to ninth parameters , More than ten parameters need to be enclosed in braces , Such as 9 Represents the first to ninth parameters , More than ten parameters need to be enclosed in braces , Such as {10}

$*

This variable represents all the parameters on the command line ,$ Treat all parameters as a whole

[email protected]

This variable also represents all the parameters on the command line , however [email protected] Treat each parameter differently

$#

This variable represents the number of all parameters on the command line

$1 You wrote it for you shell The first parameter passed by the script ,$2 You wrote it for you shell The second parameter passed by the script …

[[email protected] sh]$ vim test.sh
#!/bin/sh
echo "shell The name of the script itself : $0"
echo " Pass to shell The first parameter of : $1"
echo " Pass to shell Second parameter of : $2"

Save after exit , you are here Test.sh Enter... In the directory where you are bash Test.sh 1 2

Results output ：

shell The name of the script itself : Test.sh
 Pass to shell The first parameter of : 1
 Pass to shell Second parameter of : 2

KaTeX parse error: Undefined control sequence: \* at position 1: \̲*̲ All parameters received will be taken as a whole …@ Then all parameters to be received will be distinguished . for instance :

[[email protected] sh]$ vi parameter2.sh
#!/bin/bash
for i in"$*"
# Definition for loop ,in There are several values behind ,for How many times will it cycle , Be careful “$*” Use double quotation marks 
# Each cycle will turn in The following values are assigned to the variable i
#Shell hold $* All parameters in are considered as a whole , So this for Loop only once 
	do
		echo "The parameters is: $i"
		# Print variables $i Value 
	done
x=1
# Defining variables x The value of is 1
for y in"[email protected]"
# Again in The latter has several values ,for Cycle a few times , Assign values to variables each time y
# But Shel1 Zhongba “[email protected]” Each parameter in is considered independent , therefore “[email protected]” There are several parameters in , It's going to cycle a couple of times 
	do
		echo "The parameter$x is: $y"
		# Output variables y Value 
		x=$(( $x +1 ))
		# Natural variable x Add... For each cycle 1, In order to see the output more clearly 
	done

2.4 Predefined variables ：

Predefined variables

effect

$

Return status of last executed command . If the value of this variable is 0, Prove that the last command was executed correctly ; If the value of this variable is not О( Which number is it , It's up to the order itself ）, The last command was executed incorrectly .

$$

The process number of the current process （PID)

$!

The process number of the last process running in the background (PID)

First look at it. ”$” This variable , Let me give you an example

[[email protected] sh]$ ls
count.sh hello.sh parameter2.sh parameter.sh
#ls The command is executed correctly 
[[email protected] sh]$ echo $?
# Predefined variables “$?” The value of is 0, Prove that the last command was executed correctly 
[[email protected] sh]$ ls install.log
ls: cannot access install.log: There is no file or directory 
# There is no... In the current directory install.log file , therefore ls The order is wrong 
[[email protected] sh]$ echo $?
2
# Variable “$?” Return to a non О Value , Prove that the last command was not executed correctly 
# As for the return value of the error , Is writing ls Command defined , If the file does not exist, the value is returned 2

Let me explain again ”$$”and”$!” These two predefined variables

[[email protected] sh]$ vi variable.sh
#!/bin/bash
echo "The current process is $$"
# Output the... Of the current process PID.
# This PID Namely variable.sh When this script executes , Of the generated process PID
find /root -name hello.sh &
# Use find Command in root Directory lookup hello.sh file 
# Symbol & Put the command in the background to execute , Work management will be introduced in detail in the chapter of system management 
echo "The last one Daemon process is $!"
# Output the of the process executing the command in the background PID, That is output find Ordered PID Number 

3. A read-only variable ：

[[email protected] sh]$ vi readonly.sh
#!/bin/bash
a=10
# grammar ：readonly  Variable name 
readonly a
a=20   # Will report a mistake readonly variable
echo $a

4. Accept keyboard input ：

[[email protected] ~]$ read [ Options ][ Variable name ]
 Options :
	-a  Followed by a variable , This variable will be considered an array , Then assign it a value , By default, spaces are used as separators .
	-p： “ Prompt information ”： Waiting for the read When the input , Output prompt message 
	-t：  Number of seconds ：read The command will wait for the user to enter , Use this option to specify the waiting time 
	-n：  Numbers ：read The command accepts only the specified number of characters , Will execute 
	-s：  Hide input data , It is applicable to the input of confidential information 
    -d：  Followed by a sign , In fact, only the first character after it is useful , As a sign of the end .
    -e：  Command completion function can be used when inputting .
 Variable name :
 Variable names can be customized , If you don't specify a variable name , Will save the input into the default variable REPLY.
 If only one variable name is provided , Then the entire input line is assigned to the variable .
 If more than one variable name is provided , The input line is divided into several words , One by one, each variable is assigned , And the last variable on the command line takes all the remaining words 

Write an example to explain read command ：

[[email protected] sh]$ vi read.sh
#!/bin/bash

read -t 30 -p "Please input your name: " name
# Tips “ Please enter a name ” And wait for 30  second , Save user input into variables name  in 
echo "Name is $name"
# Look at the variables “$name” Whether your input is saved in 

read -s -t 30 -p "Please enter your age: " age
# Tips “ Please enter age ” And wait for 30 second , Save user input into variables age in 
# Age is privacy , So we use “-s” Option hides the input 
echo -e "
"
# Adjust the output format , If you do not output line breaks , For a while, the age output will not wrap 
echo "Age is $age"

read -n 1 -t 30 -p "Please select your gender[M/F]:" gender
# Tips “ Please choose gender ” And wait for 30 second , Save user input into variables gender
# Use “-n1” Option will execute if it receives only one input character （ You don't have to enter enter ）
echo -e "
"
echo "Sex is $gender"

---

Four 、shell Operator

stay shell in , Operators are the same as other programming scripting languages , Common operators are arithmetic operators 、 Relational operator 、 Logical operators 、 String operators 、 File test operators, etc

1. Arithmetic operator

Native bash Simple mathematical operations are not supported , But you can do it with other commands , for example awk and expr,expr The most commonly used .
expr Is an expression calculation tool , It can be used to evaluate expressions .
for example , Add two numbers ( Note the use of back quotes ` Not single quotes ')：

[[email protected] ~]$ vi computer.sh
#!/bin/bash
val=`expr 2 + 2`
echo " The sum of the two is  : $val"
# Be careful 
# Space between expression and operator , for example  2+2  It is not right , Must be written as  2 + 2, This is different from most programming languages we are familiar with .
# The complete expression is to be  ` `  contain , Note that this character is not a common single quotation mark , stay  Esc  Key bottom .

The following table lists the common arithmetic operators , Assumed variable a by 10, Variable b by 20

Operator

explain

give an example

Add

expr $a + $b The result is 30.

-

Subtraction

expr $a - $b The result is -10.

*

Multiplication

expr $a * $b The result is 200.

/

division

expr $b / $a The result is 2.

%

Remainder

expr $b % $a The result is 0.

=

assignment

a=$b Will put the variable b The value is assigned to a.

==

equal . Used to compare two numbers , Same returns true（ really ）.

[ $a == $b ] return false（ false ）.

!=

It's not equal . Used to compare two numbers , If not, return to true.

[ $a != $b ] return true.

Be careful ： Conditional expressions should be placed between square brackets , And there should be spaces , Must be written as [ $a == $b ].

[[email protected] ~]$ vi computers.sh
#!/bin/bash
a=10
b=20
echo ' '
echo 'a+b= ' `expr $a + $b`
echo 'a-b= ' `expr $a - $b`
echo 'a*b= ' `expr $a * $b`
echo 'a/b= ' `expr $a / $b`
echo 'a%b= ' `expr $a % $b`

# Judge whether it is equal 
if [ $a == $b ]
then
	echo 'a be equal to b'
else
	echo 'a It's not equal to b'
fi

2. Relational operator

Relational operators only support numbers , String not supported , Unless the value of the string is a number .
The following table lists the common relational operators , Assumed variable a by 10, Variable b by 20：

Operator

word

explain

give an example

-eq

equal

Check whether two numbers are equal , Equal return true.

[ $a -eq $b ] return false.

-ne

not equal

Check whether two numbers are equal , Unequal return true.

[ $a -ne $b ] return true.

-gt

great than

Check whether the number on the left is greater than that on the right , If it is , Then return to true.

[ $a -gt $b ] return false.

-lt

less than

Check if the number on the left is less than the number on the right , If it is , Then return to true.

[ $a -lt $b ] return true.

-ge

great than or equal

Check whether the number on the left is equal to or greater than the number on the right , If it is , Then return to true.

[ $a -ge $b ] return false.

-le

less than or equal

Check whether the number on the left is less than or equal to the number on the right , If it is , Then return to true.

[ $a -le $b ] return true.

[[email protected] ~]$ [ 10 -gt 10 ] 
[[email protected] ~]$ echo $? 
 1
[[email protected] ~]$ [ 10 -eq 10 ] 
[[email protected] ~]$ echo $? 
 0

Case study ： Judge whether the current input user exists . If it exists, it will prompt “ Users exist ” Otherwise prompt “ The user doesn't exist ”.

If you want to in shell Script use linux command , have access to $() Parcel order
for example ：disk_size=$(df -h | awk ‘NR==2 {print $5}’)

[[email protected] ~]$ vim demo.sh 
#!/bin/bash
# Accept user input 
read -p ' Please enter the user name to query :' username

# Get the specified user name in passwd Number of occurrences in the file 
count=$(cat /etc/passwd | grep $username | wc -l)
#count=`cat /etc/passwd | grep $username | wc -l`

# Judge the number of occurrences , If the number of times =0 Then the user does not exist , On the contrary 
if [  $count == 0 ]
then 
		echo ' The user doesn't exist '
	else 
		echo ' Users exist '
fi

3. Logical operators

The following table lists the common Boolean operators , Assumed variable a by 10, Variable b by 20：

Operator

explain

give an example

!

Non operation , Expression for true Then return to false, Otherwise return to true.

[ ! false ] return true.

-o

or （ perhaps ） operation , There is an expression for true Then return to true.

[ $a -lt 20 -o $b -gt 100 ] return true.

-a

And （ also ） operation , Both expressions are true To return to true.

[ $a -lt 20 -a $b -gt 100 ] return false.

Or operations ： One is true is true , All false is false
And operation ： If one is false, it is false , All true is true

4. String operators

The following table lists the common string operators , Assumed variable a by “abc”, Variable b by “efg”：

Operator

explain

give an example

=

Checks if two strings are equal , Equal return true.

[ $a = $b ] return false.

!=

Checks if two strings are equal , Unequal return true.

[ $a != $b ] return true.

-z

Check if the string length is 0, by 0 return true.

[ -z $a ] return false.

-n

Check if the string length is 0, Not for 0 return true.

[ -n $a ] return true.

str

Check if the string is empty , Not empty return true.

[ $a ] return true.

5. File test operators （ a key ）

The file test operator is used to detect Unix/Linux Various attributes of the file .

The operator

explain

give an example

-b file

Check if the file is a block device file , If it is , Then return to true.

[ -b $file ] return false.

-c file

Check whether the file is a character device file , If it is , Then return to true.

[ -c $file ] return false.

-d file

Check if the file is a directory , If it is , Then return to true.

[ -d $file ] return false.

-f file

Check if the file is a normal file （ It's not a catalog , It's not a device file ）, If it is , Then return to true.

[ -f $file ] return true.

-g file

Check if the file is set SGID position , If it is , Then return to true.

[ -g $file ] return false.

-k file

Check whether the file is set with the adhesive bit (Sticky Bit), If it is , Then return to true.

[ -k $file ] return false.

-p file

Check if the file is a named pipeline , If it is , Then return to true.

[ -p $file ] return false.

-u file

Check if the file is set SUID position , If it is , Then return to true.

[ -u $file ] return false.

-r file

Check whether the file is readable , If it is , Then return to true.

[ -r $file ] return true.

-w file

Check whether the file is writable , If it is , Then return to true.

[ -w $file ] return true.

-x file

Check if the file is executable , If it is , Then return to true.

[ -x $file ] return true.

-s file

Check if the file is empty （ Is the file size greater than 0）, Not empty return true.

[ -s $file ] return true.

-e file

Test files （ Including directory ） Whether there is , If it is , Then return to true.

[ -e $file ] return true.

Be careful ： There are several judgments , If only one part meets , They think they have authority .

---

5、 ... and 、 Process control

1. if conditional

1.1 Single branch if Conditions

grammar ：

if [  Conditional judgment  ]
	then
		 Program 
fi

Case study ： Statistics root partition usage

[[email protected] ~]$ vi sh/if1.sh
#!/bin/bash

# Statistics root partition usage 
rate=$(df -h | grep "/dev/sda2" | awk '{print $5}’| cut -d "%"-f1)
# The root partition usage is assigned to the variable as a variable value rate
if [ $rate -ge 80 ]
# Judge rate If the value of is greater than or equal to 80, execute then Program 
	then
		echo "Warning!/dev/sda3 is fu11!!"
	# Print warning messages . In practice , You can also send mail to the administrator .
fi

Case study ： Create directory

[[email protected] ~]$ vi sh/add_dir.sh
#!/bin/bash
# Create directory , Judge whether it exists , Existence ends , Instead, create 
echo " The current script name is $0"
DIR="/media/cdrom"
if [ ! -e $DIR ]
then
	mkdir -p $DIR
fi
echo "$DIR  Create success "

1.2 Double branch if Conditional statements

grammar ：

if [  Conditional judgment  ]
	then
		 When conditions are met , The procedure to be carried out 
	else
		 When conditions do not hold , Another program executed 
fi

Case study 1： Backup mysql database

[[email protected] ~]$ vi sh/bakmysql.sh
#!/bin/bash
# Backup mysql database .

ntpdate asia.pool.ntp.org &>/dev/null
# Synchronization system time 
date=$(date +%y%m%d)
# According to the current system time “ Specific date ” Format assignment sub variable date
size=$(du -sh/var/lib/mysql)
# Statistics mysql The size of the database , And give the size to size Variable 

if [ -d /tmp/dbbak ]
# Determine whether the backup directory exists , Is it a directory 
	then
	# If the judgment is true , Execute the following script 
	echo "Date : $date!" > /tmp/dbbak/dbinfo.txt
	# Write the current date to a temporary file 
	echo "Data size : $size" >> /tmp/dbbak/dbinfo.txt
	# Write the database size to the temporary file 
	cd/tmp/dbbak
	
	# Go to the backup directory 
	tar -zcf mysql-lib-$date.tar.gz /var/lib/mysql dbinfo.txt &> /dev/null
	# Pack and compress the database and temporary files , Throw all the output into the dustbin （ Don't want to see any output ）
	rm -rf /tmp/dbbak/dbinfo.txt
	# Delete temporary files 
else
	mkdir /tmp/dbbak
	# If the judgment is false , Create a backup directory 
	echo "Date : $date!" > /tmp/dbbak/dbinfo.txt
	echo "Data size : $size" >> /tmp/dbbak/dbinfo.txt
	# Save dates and database sizes as temporary files 
	cd /tmp/dbbak
	tar -zcf mysql-lib-$date.tar. gz dbinfo.txt /var/lib/mysql &> /dev/null
	# Compress backup database and temporary files 
	rm -rf/tmp/dbbak/dbinfo.txt
	# Delete temporary files 
fi

Case study 2： Judge apache Whether to start , If not, it will start automatically

[[email protected] ~]$ vi sh/autostart.sh
#!/bin/bash
# Judge apache Whether to start , If not, it will start automatically 

port=$(nmap -sT 192.168.4.210 | grep tcp | grep http | awk '{print $2}’)
# Use nmap Command scan server , And intercept  apache Status of the service , Assign variable port
# As long as the state is open, It proves normal startup 
if [ "$port" == "open"]
# If the variable port The value of is “open”
	then
	echo "$(date) httpd is ok!” >> /tmp/autostart-acc.log
	# Then prove apache  Normal start , Write a sentence in the normal log 
else
	/etc/rc.d/init.d/httpd start &>/dev/null
	# Otherwise, prove apache Has not started , Auto start apache
	echo "$(date) restart httpd !!" >> /tmp/autostart-err.log
	# And record the automatic startup in the error log apche  Time for 
fi

nmap Port scan command , The format is as follows ：

[[email protected] ~]$ nmap -sT  Domain name or IP
 Options :
-s       scanning 
-T       Scan all open TCP port 


# got it nmap command reference , The command we use in the script is to intercept http The state of , As long as the state is “or.

# To prove it apache Start up normal , Otherwise, prove apache Startup error . Take a look at the results of the commands in the script :
[[email protected] ~]$ nmap -sT 192.168.4.210 | grep tcp | grep http | awk ' fprint $2}’
# Scan the specified computer , Extraction contains tcp  The line of , The extraction contains httpd  The line of , Take the second column open
# Assign the intercepted value to the variable port

1.3 Multiple branches if Conditional statements

grammar ：

if [  Conditional judgment 1 ]
	then
		 When the conditional determinant 1 When established , Execution procedure 1
elif [  Conditional judgment 2 ]
	then
		 When the conditional determinant 2 When established , Execution procedure 2
… Omit more conditions …
else
	 When all the conditions are not true , Finally, execute this program 
fi

Case study ： Determine what file the user entered

[[email protected] ~]$ vi sh/if-elif.sh
#!/bin/bash
# Determine what file the user entered 

read -p "Please input a filename: " file
# Receive input from keyboard , And give variables file
if [ -z "$file” ]
# Judge file Is the variable empty 
	then
		echo "Error, please input a filename"
		# If it is empty , Execution procedure 1, That is, output error message 
		exit 1
		# Exit procedure , And the return value is Ⅰ( Assign the return value to the variable $P）
elif [ ! -e "$file” ]
		# Judge file Is there a value for 
		then
		echo "Your input is not a file!"
		# Such as 1 The fruit does not exist , Then execute the procedure 2
		exit 2
		# Exit procedure , And define the return value as 2
elif [ -f "$file” ]
		# Judge file Whether the value of is a normal file 
		then
		echo "$file is a regulare file!”
		# If it's a regular document , Then execute the procedure 3
elif [ -d "$file” ]
		# To break file Whether the value of is a directory file 
		then
		echo "$file is a directory!"
		# If it's a catalog file , Net execution program 4
else
	echo "$file is an other file!”
	# If none of the above judgments is , Then execute the procedure 5
fi

2. Multiple branches case Conditional statements

case Statement and if…elif…else Statements are also multi branch conditional statements , But and if The difference between multi branch conditional statements is that ,case Statement can only judge one kind of conditional relation , and if Statement can judge a variety of conditional relations .

case The sentence syntax is as follows :

case $ Variable name  in
	" value 1")
	 If the value of the variable is equal to the value 1, Then execute the procedure 1
	;;
	" value 2")
	 If the value of the variable is equal to the value 2, Then execute the procedure 2
	::
	… Omit other branches …
	*)
	 If none of the values of the variables are above , Then execute this procedure 
	;;
esac

This sentence needs to pay attention to the following :

• case sentence , It takes out the value in the variable , Then compare with the values in the body of the statement one by one . If the values match , Then execute the corresponding program , If the values don't match , Then compare the next value in turn . If all the values don't match , execute “*)” (* Represents all other values ） Program in .
• case Statement to “case” start , With “esac” ending .

Each branch program is followed by “;;” Double semicolon end , Represents the end of the segment ( Never forget , Every time case sentence , Don't forget the double semicolon ）.

Case study ：

[[email protected] ~]$ vi sh/if-case.sh
#!/bin/bash
read -p " Please enter a character , And press Enter confirm ：" KEY
case "$KEY" in
	[a-z]|[A-Z])
	echo " You entered the letters "
	;;
	
	[0-9])
	echo " You have entered a number "
	;;
	
	*)
	echo " You entered other characters "
	;;
esac

3. for loop

for A cycle is a fixed cycle , In other words, you know how many times you need to do the loop , Sometimes I put for A loop is called a counting loop .for There are two kinds of grammars of :

Grammar 1 :

for  Variable  in  value 1  value 2  value 3 …( It can be a file, etc )
	do
		 Program 
	done
	
	 In this grammar for Number of cycles , Depending on in The number of the following values （ The blank space to separate ）, If there are a few values, loop them a few times , And every loop assigns values to variables .
	 in other words , hypothesis in Then there are three values ,for It's going to cycle three times , The first loop will put the value 1 Assign variable , The second loop will put the value 2 Assign variable , And so on .

Grammar II :

for ((  Initial value ; Cycle control conditions ; Variable change  ))
	do
		 Program 
	done
	
 Attention should be paid to Grammar 2 :
 Initial value : At the beginning of the loop , You need to give a variable an initial value , Such as i=1;

 Cycle control conditions : Used to specify the number of times the variable loops , Such as i<=100, Then as long as i Is less than or equal to 100, The cycle will continue ;

 Variable change : After each cycle , How the variable should change , Such as i=i+1. After each cycle , Variable i All the values are added 1.

An example of grammar ： Print time

[[email protected] ~]$ vi sh/for.sh
#!/bin/bash
# Print time 

for time in morning noon afternoon evening
	do
		echo "This time is $time!"
	done

An example of grammar ： Batch decompress scripts

[[email protected] ~]$ vi sh/auto-tar. sh
#!/bin/bash
# Batch decompress scripts 

cd/lamp
# Enter the compressed package directory 
ls *.tar.gz > ls.log
# Put all the .tar.gz The end of the file overwrites the file to ls.log  Temporary file 
for i in $(cat ls.log) `
# Or write like this for i in `cat ls.log`

# Read ls.log The content of the document , How many values are there in the file , How many times will it cycle , Each loop assigns the file name to the variable i
	do
		tar -zxf $i &>/dev/null
		# Add compression , And discard all output 
	done
rm -rf /lamp/ls.log
# Delete temporary files ls.log

Grammar 2 example ： from 1 Add to 100

[[email protected] ~]$ vi sh/add. sh
#!/bin/bash
# from 1 Add to 100

s=0
for (( i=1;i<=100;i=i+1 ))
# Define the cycle 100  Time 
do
	s=$(( $s+$i ))
	# Each loop gives a variable s assignment 
	done
echo "The sum of 1+2+...+100 is : $s"
# Output 1 Add to 100 And 

Grammar 2 example ： Batch add a specified number of users

[[email protected] ~]$ vi useradd.sh
#!/bin/bash
# Batch add a specified number of users 

read -p "Please input user name: " -t 30 name
# Let the user enter the user name , Save the input into a variable name

read -p "Please input the number of users: " -t 30 num
# Let the user enter the number of users to add , Save the input into a variable num

read -p "Please input the password of users: " -t 30 pass
# Let the user enter the initial password , Save the input as a variable pass

if [ ! -z "$name" -a ! -z "$num"-a ! -z "$pass"]
# Judge that the three variables are not empty 
then
y=$(echo $num | sed 's/[0-9]//g')
# Define the value of the variable as the result of subsequent commands 
# The function of the follow-up command is , Put variables num  Replace the value of with empty . If you can replace it with blank , prove num  The value of is a number 
# If it can't be replaced with empty , prove num The value of is not a number . We use this method to judge variables num  The value of is a number 
	if [ -z "$y"]
	# If the variable y The value of is empty , prove num Variables are numbers 
		then
		for (( i=1 ; i<=$num; i=i+1 ))
		# loop num The number of times specified by the variable 
			do
			/usr/sbin/useradd $name$i &>/dev/null
			# Add users , The user name is a variable name  The value of plus variable i The number of 
			echo $pass | /usr/bin/passwd --stdin $name$i &>/dev/null
			# Set the initial password for the user as a variable pass  Value 
			done
	fi
fi

Grammar 2 example ： Bulk delete users

[[email protected] ~]$ vi sh/userdel.sh
#!/bin/bash
# Bulk delete users 

user=$(cat /etc/passwd | grep " /bin/bash"|grep -v "root"Icut -d ":" -f 1)
# Read user information file , Extract users who can log in , Cancel root user , Intercept the user name in the first column 

for i in $user
# loop , How many ordinary users , How many times 
	do
		userdel -r $i
		# Each cycle , Delete the specified ordinary user 
	done

4. while loop

Yes while Circularly speaking , As long as the conditional judgment holds , The cycle goes on and on , Until the conditional judgment doesn't hold , The cycle stops .

grammar ：

while [  Conditional judgment  ]
	do
		 Program 
	done

Case study ：1 Add to 100

[[email protected] ~]$ vi sh/addnum.sh
#!/bin/bash
# from 1 Add to 100

i=1
s=0
# To the variable i And variables s assignment 

while [ $i -le 100 ]
# If the variable i Is less than or equal to 100, Then execute the loop 
	do
		s=$(( $s+$i ))
		i=$(( $i+1 ))
	done
echo "The sum is: $s"

Case study ： Compare and judge the entered values

[[email protected] ~]$ vi sh/addnum.sh
#!/bin/bash
PRICE=$(expr $RANDOM % 1000)
TIMES=0

echo " The price of the goods is 0-999 Between , Guess what it is ？"
while true
do
  read -p " Please enter the price you guessed ：" INT
let TIMES++

	if [ $INT -eq $PRICE ] ; then
	  echo " Congratulations on your guesses , The actual price is  $PRICE"
	  echo " You guessed all together  $TIMES  Time "
	exit 0
	elif [ $INT -gt $PRICE ] ; then
	  echo " It's too high "
	else
	  echo " It's too low "
	fi
done

5. until loop

and while The cycle is reversed ,until As long as the conditional judgment does not hold, the loop will be carried out , And execute the loop program . Once the cyclic condition is established , Then stop the cycle .

grammar :

until [  Conditional judgment  ]
	do
		 Program 
	done

Case a ：1 Add to 100

[[email protected] ~]$ vi sh/until.sh
#!/bin/bash
# from 1 Add to 100

i=1
s=0
#t To the variable i And variables s assignment 

until [ $i -gt 100 ]
# Loop until variable i The value is greater than 100, Stop the cycle 
	do
		s=$(( $s+$i ))
		i=$(( $i+1 ))
	done
echo "The sum is: $s"

6. function

grammar ：

function  Function name  () {
	 Program 
}

Case study ： Receive the number entered by the user , And then from 1 Add to this number

[[email protected] ~]$ vi sh/function.sh
#!/bin/bash
# Receive the number entered by the user , And then from 1 Add to this number 

function sum () {
	# Defined function sum
	s=0
	for (( i=0; i<=$num;i=i+1 ))
		# Cycle until i Greater than $1 until .$1 Is the function sum  The first parameter of 
		# Position parameter variables can also be used in functions , But the $1 It refers to the first parameter of the function 
		do
			s=$(( $i+$s ))
		done
	echo "The sum of 1+2+3...+$1 is :$s"
	# Output 1 Add to $1 And 
}

read -p "Please input a number: " -t 30 num
# Receive the number entered by the user , And assign the value to the variable num
y=$(echo $num | sed 's/[0-9]//g')
# Put variables num Replace the value of with empty , And give variables y

if [ -z "$y"]
# Judgment variable y Is it empty , To determine the variable num Whether the is a number 
	then
		sum $num
		# call sum function , And put the variables num The value of is passed as the first parameter to sum function 
else
		echo "Error!! Please input a number!"
		# If the variable num  The value of is not a number , Then the error message will be output 
fi

7. Special process control statements

7.1 exit sentence

The system has exit Ordered , Used to exit the login status of the current user . But in the Shell Script ,exit Statement is used to exit the current script . in other words , stay Shell Script , As long as I meet exit sentence , Subsequent procedures are no longer executed , And just exit the script .

exit The grammar is as follows :

exit [ Return value ]

If exit The return value is defined after the command , The return value after the execution of this script is the return value defined by us . By querying $ This variable , To see the return value . If exit There is no return value defined after , The return value after script execution is execution exit The statement before , The return value of the last command executed . Write a exit Example :

[[email protected] ~]$ vi sh/exit.sh
#!/bin/bash
# demonstration exit The role of 

read -p "Please input a number: " -t 30 num
# Receive user input , And assign the input to the variable num
y=$ (echo $num | sed 's/[0-9]//g')
# If the variable num  The value of is a number , Then put num Replace the value of with empty , Otherwise don't replace 
# Assign the replaced value to the variable y
[ -n "$y" ] && echo "Error! Please input a number!" && exit 18
# Judgment variable y If the value of is not empty , Output error message , Exit script , The return value of exit is 18
echo "The number is: $num"
# If you don't quit working overtime , Then print the variable num Number in 

7.2 break sentence

When the program is executed to break When the sentence is , Will end the entire current cycle . and continue Statement is also a statement that ends a loop , however continue Statement a single current loop , And the next cycle will continue .

Case study ：

[[email protected] ~]$ vi sh/break.sh
#!/bin/bash
# demonstration break  Out of the loop 

for (( i=1;i<=10; i=i+1 ))
# Cycle ten times 
	do
		if ["$i" -eq 4 ]
		# If the variable i The value is equal to the 4
			then
			break
			# Exit the whole cycle 
		fi
	echo $i
	# Output variables i Value 
	done

Execute this script , Because once the variable i The value is equal to the 4, The whole cycle will jump out , So it should only cycle three times :

[[email protected] ~]$ chmod 755 sh/break.sh
[[email protected] ~]#sh/break.sh
1
2
3

7.3 break sentence

continue It is also a statement to end process control . If in a loop ,continue Statement will only end a single current loop .

Case study ：

[[email protected] ~]$ vi sh/break.sh
#!/bin/bash
# demonstration continue

for (( i=1;i<=10;i=i+1 ))
# Cycle ten times 
	do
		if ["$i" -eq 4 ]
		# If the variable i The value is equal to the 4
			then
			continue
			# Exit for continue
		fi
	echo $i
	# Output variables i Value 
	done

Execute this script :

[[email protected] ~]$ chmod 755 sh/continue.sh
[[email protected] ~]#sh/break.sh
1
2
3
5
6
7
8
9
10
# Less 4 This output 

---

6、 ... and 、 Character interception 、 Replace and process commands

Regular expressions

Metacharacters

describe

Example

Escape character , Escape special characters , Ignore its special significance

a.b matching a.b, But can't match ajb,. Be escaped as a special meaning

^

Match the beginning of the line ,awk in ,^ Is the beginning of the matching string

^tux Match with tux Beginning line

$

Match the end of the line ,awk in ,$ Is the end of the matching string

tux$ Match with tux The line at the end

.

Match break Any single character other than

ab. matching abc or bad, Unmatched abcd or abde, Only single characters can be matched

[ ]

Match included in [ character ] Any one of the characters

coo[kl] Can match cook or cool

[^]

matching [^ character ] Any character other than

123[^45] Can't match 1234 or 1235,1236、1237 Fine

[-]

matching [] Any character within the specified range in , Write it in increments

[0-9] Can match 1、2 or 3 Wait for any number

Match previous items 1 Time or 0 Time

colour Can match color perhaps colour, Can't match colouur

Match previous items 1 Times or more

sa-6+ matching sa-6、sa-666, Can't match sa-

*

Match previous items 0 Times or more

co*l matching cl、col、cool、coool etc.

()

Match expression , Create a substring for matching

ma(tri) matching max or maxtrix

{n}

Match previous items n Time ,n Can be for 0 The positive integer

[0-9]{3} Match any three digits , It can be extended to [0-9][0-9][0-9]

{n,}

Previous items need to match at least n Time

[0-9]{2,} Matching any two or more digits is not supported {n,}{n,}{n,}

{n,m}

Specify that the previous item matches at least n Time , Match at most m Time ,n<=m

[0-9]{2,5} Match any number from two to five digits

|

Alternate matching | Either of the two sides

ab(c|d) matching abc or abd

6.1 Character interception 、 Replace the command

6.1.1 cut Column extraction command

[[email protected] ~]$ cut [ Options ]  file name 
 Options :
-f  Column number :  Extract which column 
-d  Separator :  Splits columns according to the specified separator 
-n	 De segment multibyte characters 
-c  character in range :  Do not rely on separators to distinguish , But through the character range （ The first line is 0） To extract fields .“n-” Says from the first n Characters to the end of the line ;“n-m” From n Character to character m Characters ;“ One m” Says from the first 1 Character to character m Characters .
--complement	 Complement the selected bytes 、 Character or field 
--out-delimiter	 Specifies that the output is a field separator 

cut The default separator for commands is the tab character , That is to say “tab” key , However, the support for space characters is not very good . Let's create a test file first , And then look at it cut The role of command :

[[email protected] ~]$ vi student.txt
id	name	gender	mark
1	liming	m		86
2	sc		m		67
3	tg		n		90


[[email protected] ~]$ cut -f 2 student.txt
# Extract the second column 

If you want to extract multiple columns, just use the column number directly “,” Separate , The order is as follows :

[[email protected] ~]$ cut -f 2,3 student.txt

cut It can be extracted according to characters , We need to pay attention to “8-” It means to extract the tenth character of all lines from the beginning to the end of the line , and “10-20” Represents extracting the 10th to 20th characters of all lines , and “-8” Stands for extracting all lines from the beginning of the line to the eighth character :

[[email protected] ~]$ cut -c 8- student.txt
# Extract the eighth character from the beginning to the end of the line , It seems very chaotic , That's because the number of characters in each line is not equal 


[[email protected] ~]$ cut -d ":" -f 1,3 /etc/passwd
# With “:” As a separator , extract /etc/passwd_ The first and third columns of the document 

If I want to use cut Command intercept df The first and third columns of the command , That's what happens :

[[email protected]~]$ df -h | cut -d " " -f 1,3
Filesystem 
/dev/sda2 
tmpfs 
/dev/sda1

6.1.2 awk Programming

6.1.2.1 awk summary

AWK Is a language for processing text files , Is a powerful text analysis tool .

6.1.2.2 printf Format output

[[email protected] ~]$ printf ‘ Output type output format ’  Output content 

 The output type :
%c:     ASCII character . Displays the first character of the corresponding parameter 
%-ns:    Output string , minus sign “-” Indicates left alignment ( Default right alignment ),n Number refers to the number of characters output , Just write a few parameters %-ns
%-ni:    Output integer ,n The number refers to the output of several numbers 
%f：     Output the number of digits to the right of the decimal point 
%m.nf:   Output floating point number ,m and n It's the number. , Refers to the number of integers and decimals output . Such as %8.2f Represents the total output 8 digit , among 2 Place is a decimal ,6 Bits are integers. .

 Output format :
a:  Output warning sound 
:  Output backspace key , That is to say Backspace key 
:  Clear the screen 

:  Line break 

:  enter , That is to say Enter key 
	:  Horizontal output backspace key , That is to say Tab  key 
:  Vertical output backspace key , That is to say Tab  key 

To demonstrate printf command , We need to revise what just happened cut Command to use student.txt file , The contents of the document are as follows :

[[email protected] ~]$ vi student.txt
ID      Name    php  	 Linux  	MySQL 	  Average
1       AAA      66         66       66           66
2       BBB      77         77       77           77
3       CCC      88         88       88           88

#printf Format output file 
[[email protected] ~]$ printf '%s	 %s	 %s	 %s	 %s	 %s	 
’ $(cat student.txt)
#%s Corresponding to the following parameters respectively ,6 Just write 6 individual 
ID      Name    php   Linux  MySQL   Average
1       AAA      66         66       66           66
2       BBB      77         77       77           77
3       CCC      88         88       88           88

If you don't want to output your grades as strings , Instead, the output is integer and floating-point , So be it :

[[email protected] ~]$ printf '%i	 %s	 %i	 %i	 %i	 %8.2f	 
’  $(cat student.txt | grep -v Name)

6.1.2.3 awk Basic use

[[email protected] ~]$ awk‘ Conditions 1{ action 1}  Conditions 2{ action 2}…’  file name 
 Conditions （Pattern）:
	 Generally, relational expressions are used as conditions . There are a lot of these relational expressions , for example :
	x > 10   Judgment variable x Is it greater than 10
	x == y   Judgment variable x Is it equal to a variable y
	A ~ B    Judgment string A Whether it can match B The substring of the expression 
	A !~ B   Judgment string A Whether it does not contain matching B The substring of the expression 
	
 action （Action） :
	 Format output 
	 Flow control statement 

 Common parameters ：

   -F	 Specify the field separator to use when entering 
   -v	 Custom variable 
   -f	 Read from script awk command 
   -m	 Yes val Value set intrinsic limit 

Let's study here first awk Basic usage , That is, just look at what the format output action does .

[[email protected] ~]$ awk '{printf $2 "	" $6 "
"}’ student.txt
# Output the second and sixth Columns 

For example, just intercepted df The result of the command ,cut The command is out of strength , Let's see awk command :

[[email protected] ~]$ df -h | awk '{print $1 "	" $3}'
# Intercept df The first and third columns of the command 

6.1.2.4 awk Conditions

Conditions of type

Conditions

explain

awk Reserved words

BEGIN

stay awk At the beginning of the program , Execute before reading any data .BEGIN The post action is executed only once at the beginning of the program

awk Reserved words

END

stay awk The program has processed all the data , Execute at the end of .END The post action is executed only once at the end of the program

Relational operator

>

Greater than

Relational operator

<

Less than

Relational operator

>=

Greater than or equal to

Relational operator

<=

Less than or equal to

Relational operator

==

be equal to . Used to judge whether two values are equal , If you assign a value to a variable , Please use “” Number

Relational operator

!=

It's not equal to

Relational operator

A~B

Judgment string A Whether it can match B The substring of the expression

Relational operator

A!~B

Judgment string A Whether it does not contain matching B The substring of the expression

Regular expressions

/ Regular /

If in "//" Characters can be written in , Regular expressions can also be supported

BEGIN

BEGIN yes awk Reserved words , Is a special type of condition .BEGIN The time to execute is “ stay awk At the beginning of the program , Execute before reading any data ”. once BEGIN After the action is executed once , When awk Start reading data from the file ,BEGIN The conditions are no longer valid , therefore BEGIN A defined action can only be performed once .
for example :

[[email protected] ~]$ awk 'BEGIN{printf "This is a transcript 
" } {printf $2 "	" $6 "
"}’ student.txt
#awk The command will not be executed as long as the complete single quotation mark is not detected , So the newline of this command does not need to be added “|”, It's a line of command 
# Two actions are defined here 
# The first action uses BEGIN Conditions , So it will print before reading the file data “ This is a report card ”( It's only going to be executed once )
# The second action prints the second and sixth fields of the file 

END

END It's also awk Reserved words , But it happens to be with BEGIN contrary .END Is in awk The program has processed all the data , Execute at the end of .END The post action is executed only once at the end of the program . for example :

[[email protected] ~]$ awk 'END{printf "The End 
"} {printf $2 "	" $6 "
"}’ student.txt
# Enter... At the end of the output “The End”, This is not the content of the document itself , And it's only going to be executed once 

Relational operator

Take a few examples of relational operators . Suppose I want to see if the average score is greater than or equal to 87 Who is the student of the minute , You can enter the command like this :
Example 1:

[[email protected] ~]$ cat student.txt | grep -v Name | awk '$6 >= 87 {printf $2 "
"}'
# Use cat Output file content , use grep Invert contains “Name” The line of 
# Determine the sixth field （ Average score ） Greater than or equal to 87 Split line , If the judgment is true , Then the sixth column is marked （ Student name $2）

After adding the condition , Only when the condition is satisfied will the action be executed , If the conditions are not met , The action does not run . Through this experiment , You can see , although awk Is the column extraction command , But also read in by line . This Command execution process That's true :

1） If there is BEGIN Conditions , Then execute first. BEGIN Defined actions .
2） without BEGIN Conditions , Read in the first line , Assign the data in the first row to $0、$1、$2 Equivariant . among $0 Represents the overall data of this row ,$1 Represents the first field ,$2 Represents the second field .
3） Judge whether the action is executed according to the condition type . If the conditions are met , Then perform the action , Otherwise, read the next row of data . If there is no condition , Then each line performs an action .
4） Read in the next line of data , Repeat the above steps .

Another example , If I want to see Sc The average score of users :

Example 2:

[[email protected] ~]$ awk '$2 ~ /AAA/ {printf $6 "
"}' student.txt
# If the input in the second field contains “Sc” character , Print the sixth field data 
85.66

Pay attention to awk in , Use “//” String contained ,awk Command will find . That is to say, the string must be “//” contain ,awk Commands can be correctly recognized .

Regular expressions

If you want to make awk Identify strings , You have to use “//” contain , for example :
Example 1:

[[email protected] ~]$ awk '/Liming/ {print}’student.txt
# Print Liming The achievement of 

When using df The command to view partition usage is , If I just want to see the usage of the real system partition , I don't want to check the usage of discs and temporary partitions , Then you can :

Example 2:

[[email protected] ~]$ df -h | awk '/sda[O-9]/ {printf $1 "	" $5 "
"}’
# The query contains sda Lines of numbers , And print the first and fifth fields 

6.1.2.5 awk Built-in variables

awk Built-in variables

effect

$0

Represents the present awk Read the entire row of data . We know awk Read data line by line ,$0 It represents the entire row of data currently read into the row .

$n

It stands for the No. of the current reading line n A field . such as ,$1 It means the first one 1 A field ( Column ),$2 It means the first one 2 A field ( Column ), And so on

NF

Fields owned by the current row （ Column ） total .

NR

At present awk Rows processed , Is the row number of the total data .

FS

User defined separator .awk The default delimiter for is any space , If you want to use other delimiters （ Such as “:”）, Need FS Variable definitions .

ARGC

Number of command line parameters .

ARGV

Array of command line arguments .

FNR

The current number of records in the current file （ The input file starts with 1）.

OFMT

The output format of the value （ The default is %.6g）.

OFS

Separator for output field （ Default is space ）.

ORS

Output record separator （ The default is line break ）.

RS

Enter record separator （ The default is line break ）.

awk Common statistical examples

1、 Print the first column of the file ( Domain ) ：
 awk '{print $1}' filename
 
2、 Print the first two columns of the file ( Domain ) ：
 awk '{print $1,$2}' filename
 
3、 Print out the first column , Then print the second column  ： 
awk '{print $1 $2}' filename

4、 Print the total number of lines in the text file  ： 
awk 'END{print NR}' filename

5、 Print the first line of text  ：
awk 'NR==1{print}' filename

6、 Print the second line and the first column of the text  ：
sed -n "2, 1p" filename | awk 'print $1'



1.  Get the first column 
ps -aux | grep watchdog | awk '{print $1}'

2.  Get the first column , Second column , The third column 
ps -aux | grep watchdog | awk '{print $1, $2, $3}'

3.  Get the first column of the first row , Second column , The third column 
ps -aux | grep watchdog | awk 'NR==1{print $1, $2, $3}'

4.  Get the number of lines NR
df -h | awk 'END{print NR}'

5.  Get the number of columns NF（ Here is to get the number of columns in the last row , Note that the number of columns per row may be different ）
ps -aux | grep watchdog | awk 'END{print NF}'

6.  Get the last column 
ps -aux | grep watchdog | awk '{print $NF}'

7.  Operate on files 
awk '{print $1}' fileName

8.  Specify the separator （ Here we use : Division ）
ps -aux | grep watchdog |awk  -F':' '{print $1}'

9.  No error will be reported if the range is exceeded 
ps -aux | grep watchdog | awk '{print $100}'




[[email protected] ~]$ cat /etc/passwd | grep "/bin/bash" | awk '{FS=":"} {printf $1 "	" $3 "
"}’
# Query the user name and... Of the user who can log in UID

here “:” The delimiter takes effect , But the first line doesn't work , So we forgot “BEGIN” Conditions , Then try again ;

[[email protected] ~]$ cat /etc/passwd | grep "/bin/bash" | awk 'BEGIN {FS=":"} {printf $1 "	" $3 "
"}’


[[email protected] ~]$ cat /etc/passwd | grep "/bin/bash" | awk 'BEGIN {FS=":"} {printf $1 "	" $3 "	  Line number :” NR "	  Number of fields :" NF "
"}’
# Explain awk command 
# Start execution { The separator is “:”}{ Output the first field and the third field output line number (NR value ） Number of fields (NF value ）}
root     0       Line number :1        Number of fields :7
user1   501      Line number :2        Number of fields :7

If I just want to see sshd Information about this pseudo user , You can use it in this way :

[[email protected] ~]$ cat /etc/passwd | awk 'BEGIN {FS=":"} $1=="sshd" {printf $1 "	" $3 "	  Line number :" NR "	  Number of fields :" NF "
"}’
# You can see sshd  Pseudo user UID yes 74, yes /etc/passwd_ File first 28 That's ok , This trip has 7 A field 

6.1.2.6 awk Process control

Let's use it again student.txt File do an exercise , The later use is more complicated , Let's take a look at the contents of this file :

[[email protected] ~]$ cat student.txt
ID      Name    php   Linux  MySQL   Average
1       AAA      66         66       66           66
2       BBB      77         77       77           77
3       CCC      88         88       88           88

Let's take a look at how to awk Define the values of variables and call variables in . Suppose I want to make statistics PHP The total score , Then it should be like this :

[[email protected] ~]$ awk 'NR==2 {php1=$3}
NR==3 {php2=$3}
NR==4 {php3=$3;totle=phpl+php2+php3;print "totle php is " totle}’ student.txt
# Statistics PHIP The total score 

Let's explain the order .“NR2 {iphp1=$3}” ( On the condition that NR2, The action is php1=$3） If the input data is the second line （ The first line is the title line ）, Just assign the value of the third field in the second row to the variable “php1”.
“NR3 {php2=$3}" If the input data is the third line , Just assign the value of the third field in the third row to the variable “php2”.“NR4 {php3=$3;totle=phpl+php2+php3;print “totle php is " totle}”（“NR==4” Is the condition , Back ( All the actions in are actions ) If the input data is the fourth line ﹐ Just assign the value of the third field in the fourth row to the variable "php3”; And then define variables totle The value of is “php1+php2+php3”; Then the output “totle php is” keyword , Add variables after totle Value .

stay awk Programming , Because the command statement is very long , You should pay attention to the following when entering the format :

Multiple conditions { action } You can split it with spaces , You can also use carriage return to split .

In an action , If you need to execute multiple commands , Need to use “;” Division , Or use carriage return to split .

stay awk in , The assignment and call of variables do not need to add “$” operator .

Condition to determine whether the two values are the same , Please use “==”, In order to distinguish from variable assignment .

After looking at how to implement process control , Suppose that if Linux Results are greater than 90, Is a good man ( learn PHP It means a lot of pressure !) :

[[email protected] ~]$ awk '{if (NR>=2) {if ($4>60) printf $2 "is a good man!
"}}’ student.txt
# There are two in the program if Judge , The first judgment line number is greater than 2, The second judgment Linux Results are greater than 90 branch 
Liming is a good man !
Sc is a good man !

Actually in awk in if Judgment statement , It can be used directly awk To replace with its own conditions , The script can be rewritten like this :

[[email protected] ~]$  awk ’NR>=2 {test=$4}
test>90 {printf $2 "is a good man! 
"}’ student.txt
# First judge if the line number is greater than 2, Just assign the fourth field to the variable test
# In judging if test The value is greater than 90 branch , Just print a good man 
Liming is a good man!
Sc is a good man!

6.1.2.7 awk function

awk Programming also allows functions to be used in programming , Let's talk about it awk The custom function of .awk Functions are defined as follows :

function  Function name （ parameter list ）{
	 The body of the function 
}

We define a simple function , Use the function to print student.txt Name and average grade of the students , You should write functions like this ：

[[email protected] ~]$ awk 'function test(a,b) { printf a "	" b "
"}
# Defined function test, Contains two parameters , The content of the function body is to output the values of these two parameters 
{ test($2,$6) } ' student.txt
# Call function test, And pass values to the two parameters .
Name    Average
AAA      87.66
BBB      85.66
CCC      91.66

6.1.2.8 awk Invoke script in

For small single line programs , Pass the script as a command line argument to awk It's very simple , It is more difficult to deal with multiline programs . When the program is multiline , Using external scripts is a good fit . First, write the script in the external file , And then you can use awk Of -f Options , Make it read the script and execute .
for example , We can write a awk Script :

[[email protected] ~]$ vi pass.awk
BEGIN {FS=":"}
{ print $1 "	"  $3}

And then you can use “ One f” Option to call this script :

[[email protected] ~]$ awk -f pass.awk /etc/passwd
rooto
bin1
daemon2
… Omit part of the output …

6.1.3 sed Text selection 、 Replace 、 Delete 、 New orders

sed It is mainly used to select data 、 Replace 、 Delete 、 New orders .

grammar :

[[email protected] ~]$ sed [ Options ] ‘[ action ]’  file name 

 Options :
-n:   commonly sed The command will output all the data to the screen , If you add this option , It will only pass through sed Command processing lines output to the screen .
-e:   Allows you to apply more than one... To input data sed Command to edit .
-f  Script name :  from sed Read in the script sed operation . and awk Ordered -f Very similar .
-r:   stay sed Extended regular expressions are supported in .
-i:   use sed The result of the modification directly modifies the file that reads the data , Not from the screen 

 action :
num a :  Additional , Add one or more rows after the current row . When adding multiple lines , Except for the last line , You need to use... At the end of each line “” It means the data is not finished .num What line is it 
c :  Row substitution , use c The following string replaces the original data line , When replacing multiple lines , Except for the last line , At the end of each line “” It means the data is not finished .
num i :  Insert , Insert one or more lines before the current line . When inserting multiple lines , Except for the last line , You need to use... At the end of each line “” It means the data is not finished .num What line is it 
d ;  Delete , Delete the specified row .
p :  Print , Output the specified line .
s :  String replacement , Replace one string with another . The format is “ Line scope s/" Old string / New string /g”（ and vim The substitution format in is similar to ）.

Yes sed Command everyone to pay attention to ,sed The changes do not directly change the contents of the file （ If it's the output of a command received with a pipeline character , In this case, there are no documents ）, Instead, the modification results are only displayed on the screen , Unless used “-i” Option will directly modify the file .

6.1.3.1 Extract row data

Let's give a few examples sed What are the orders for . Suppose I want to check student.txt The second line of , Then we can use “p” Action :

[[email protected] ~]$ sed  '2p' student.txt
ID      Name    php   Linux  MySQL   Average
1       AAA      66         66       66           66
2       BBB      77         77       77           77
3       CCC      88         88       88           88

Specify a row to output , Use -n Options

[[email protected] ~]$ sed -n  '2p' student.txt
1       AAA      66         66       66           66

6.1.3.2 Delete row data

[[email protected] ~]$ sed  '2,4d' student.txt
# Delete the second to fourth rows of data 

6.1.3.3 Append insert row data

[[email protected] ~]$ sed '2a hello' student.txt
# Add... After the second line  hello

“a” Data will be added after the specified row , If you want to insert data before a specified row , You need to use “i” action :

[[email protected] ~]$ sed '2i hello world' student.txt
# Insert two rows of data before the second row 

If you want to append or insert multiple rows of data , Except for the last line , Add... At the end of each line “” It means the data is not finished . Look again. “-n” The role of options :

[[email protected] ~]$ sed -n '2i hello world' student.txt
# Only view sed Command operated data 

6.1.3.4 Replace row data

“-n” Only view sed Command operated data , Instead of looking at all the data .
Let's take a look at how to implement row data replacement , hypothesis AAA My grades are so good , I really don't want to see his achievements stimulate me , Then you can use it "c" action :

[[email protected] ~]$ cat student.txt | sed '2c No such person'

sed The command does not modify the contents of the file by default , If I really need to let sed The command directly handles the contents of the file , have access to “-i” Options . But be careful , This is very easy to misoperate , Be careful when operating system files . have access to
Such an order :

[[email protected] ~]$ sed -i '2c No such person' student.txt

6.1.3.5 String substitution

“c” The action is to replace the whole line , If you just want to replace some data in the line , Then use “s” Action .g bring sed All matching strings in the file are replaced , The modified content will be sent to standard output , The original file will not be modified .

[[email protected] ~]$ sed 's/ Old string / New string /g'  file name 
[[email protected] ~]$ sed ' Line scope s/ Old string / New string /g'  file name 

Alternate formats and vim Very similar , Suppose I feel my own PHP The grades are too low , Want to cheat and give him a higher score , You can do this :

[[email protected] ~]$ sed '3s/74/99/g' student.txt
# In the third line , hold 74 Switch to 99

So it looks better . If I want to AAA The teacher's grades were commented out , Let him no longer take effect . You can do this :

[[email protected] ~]$ sed '2s/^/#/g' student.txt
# Here we use regular expressions ,“^” At the head of the line 

stay sed Only row ranges can be specified in , So it's a pity that I'm between them , You can't just comment them out , So we can do this :

[[email protected] ~]$ sed -e 's/AAA//g ; s/BBB//g' student.txt
# At the same time “Liming” and “Tg” Replace empty 

“-e” Option can execute multiple at the same time sed action , Of course, if you just perform an action, you can also use “-e” Options , But it doesn't make any sense at this time . Also pay attention to , Use... Between multiple actions “;” No. or enter split , For example, the previous command can also be written like this :

[[email protected] ~]$ sed -e 's/Liming//g
>s/Tg//g'’ student.txt

6.2 Character processing commands

6.2.1 sort Sort order

[[email protected]~]$ sort [ Options ]  file name 
 Options :

-f:  Ignore case 
-b:  Ignore the blanks in front of each line 
-n:  Sort by numeric , By default, string sort is used 
-r:  Reverse sorting 
-u:  Delete duplicate lines . Namely uniq command 
-t:  Specify the separator , The default is the separator and tab 
-k n[,m]: ― Sort by specified field range . From n Field start ,m End of field （ Default to end of line ）

Case study ：

sort Commands are sorted by the first character at the beginning of each line by default , such as :

[[email protected]~]$ sort /etc/passwd
# Sort user information files 

If you want to reverse sort , Please use “-r” Options :

[[email protected]~]$ sort -r/etc/passwd
# Reverse sorting 

If you want to specify the fields to sort , Need to use “-t” Option to specify the separator , And use “-k” Option specifies the field number . Join me and I want to follow UID Field sorting /etc/passwd file :

[[email protected]~]$ sort -t ":" -k 3,3 /etc/passwd
# Specifies that the separator is “:”, Start with the third field , Sort at the end of the third field , Just sort in the third field 

because sort The default is to sort by character , Of the previous users UID The first character of is 1, So this sort . To sort by numbers , Please use “-n” Options :

[[email protected]~]$ sort -n -t ":" -k 3,3 /etc/passwd

Of course “-k” Options can be used directly “-k 3”, Represents sorting from the third field to the end of the line （ The first character is sorted first , If the same , The second character is reordered , Know the end of the line ）.

6.2.2 uniq Cancel duplicate lines

[[email protected]~]$ uniq [ Options ]  file name 
 Options ：
	-i： Ignore case 

6.2.3 wc Statistical orders

[[email protected]~]$ wc [ Options ]  file name 
 Options ：
	-l： Just count the lines 
	-w： Just count the number of words 
	-m： Just count the number of characters