Use of sed in shell script

---

Preface

sed

sed summary

sed Is a powerful and simple text analysis conversion tool , Can read text , And edit the text content according to the specified conditions （ Delete 、 Replace 、 add to 、 Mobile, etc ), Finally, output all lines or only some lines of processing .

sed It can also realize quite complex text processing operation without interaction , Widely used in shell Script , It is used to complete various automatic processing tasks

sed Workflow

sed The workflow of mainly includes reading 、 Perform and display three processes

Read :
sed From the input stream （ file 、 The Conduit 、 The standard input ） Read a line in and store it in a temporary buffer （ Also called pattern space ,patternspace) .

perform :
By default , be-all sed Commands are executed sequentially in pattern space , Unless the address of the line is specified , otherwise sed The command will be executed on all lines in turn

Show :
Send the modified content to the output stream . After sending the data , Mode space will be cleared .
Before all the contents of the file are processed , The above process will be repeated , Until all content is processed

notes : By default, all sed Commands are executed in pattern space , So the input file doesn't change , Unless you're using redirection to store output

sed working principle

The process ：

Read a new line into the cache space ;

Extract the first instruction from the specified operation instruction , Judge whether it matches pattern;

If it doesn't match , Then ignore the post read edit command , Back to the first 2 Step 2: continue to take out the next instruction ;

If the match , Then the subsequent edit command is executed for the cached row ; After completion , Back to the first 2 Step 2: continue to take out the next instruction ;

When all instructions are applied , Output the contents of the cache line ; Back to the first 1 Next, read on to the next line ;

When all the lines have been processed , end ;

sed Command common usage

Usually called sed There are two formats for commands , among " Parameters " Refers to the target file of the operation , Use when there are multiple operands , Comma between files " ," Separate , and scriptfile Represents a script file , Need to use " -f " Option assignment , When the script file appears before the target file , Indicates that the input target file is processed through the specified script file .

sed[ Options ]' operation ' Parameters 

sed[ Options ]-f scriptfile Parameters 

sed Common options

-e  or  --expression= : Represents a text file that uses a specified command or script to process input .

-f  or  --file= : Indicates that the input text file is processed with the specified script file .

-h  or  --help : Display help .

-n、--quiet  or  silent : Show only processed results .

-i.bak : Edit text file directly .

-r,-E : Using extended regular expressions .

-s : Treat multiple files as separate files , Instead of a single continuous long file stream 

sed Common operations of commands

" operation " Used to specify the action behavior of file operation , That is to say sed The order of . It is usually used " [n1 [,n2] ] " Format of operation parameters .n1、n2 It's optional , Represents the number of rows selected for operation ,

a∶  increase , Add a line below the current line to specify .

c∶  Replace , Replace the selected row with the specified content .

d∶ Delete , Delete selected rows .

i∶  Insert , Insert a row above the selected row to specify .

p∶  Print , If you also specify a row , Indicates to print the specified line ; If no line is specified , It means printing everything ;  If there are non printing characters , with  AscII Code output . It is usually associated with _n" Use options together .

s∶ Replace , Replace specified characters .

y∶  Character conversion , The character length before and after conversion must be the same .

sed Use of commands

Output eligible text

Example ：

1. Output everything , Equate to cat test.txt

2. Output No 3 That's ok

3. Output 3-5 That's ok

4. Output odd lines ,n Read in the next line

5. Output all even lines ,n Read in the next line

6. Output No 1-5 Odd rows between rows ( The first 1、3、5 That's ok )

7. Output No 1 To the first 10 Even rows between rows

8. Output No 10 Even lines from line to end of file

In execution “sed -n ‘10,${n;p}’ test.txt” On command , Read the second 1 Line is the... Of the file 10 That's ok , Read the second 2 Line is the... Of the file 11 That's ok , And so on , So the even line of the output is the... Of the file 11 That's ok 、13 Line until the end of the file , It includes the empty line

9. Directly display the required second line

sed When a command is combined with a regular expression , Slightly different format , Regular expression to "/" Surround

Example ：

1. The output contains the The line of

2. Output from 4 Line to the first containing the The line of

3. The output contains the The line number of the line where , Equal sign (=) Used to output line numbers

4. Output to PI Beginning line

5. Output lines ending in numbers

6. Output contains words wood The line of ,<、> For word boundaries

In addition to the qualified text （d）

Several common deletion usages of the command

Example ：

In the following order nl The command is used to count the number of lines in a file , Combined with this command, you can see the result of command execution more intuitively

1. Delete the first 3 That's ok

2. Delete 3-5 That's ok

3. Delete include cross The line of .（ Original paragraph 8 The line is deleted ; If you want to delete, do not include cross The line of , use ! The symbol indicates the reverse operation , Such as ’/cross/!d’）

4. Delete lines that start with lowercase letters
![ Insert picture description here ](https://img-blog.csdnimg.cn/29c5b9fc72e24c7da67a222f413a6c8c.png?x-oss-process=image/watermark,type_d3F5LXplbmhlaQ,shadow_50,text_Q1NETiBA5LiA6Imy6IKh5aSn6LeM,size_18,color_FFFFFF,t_70,g_se,x_16

5. Delete with "." The line at the end

6. Delete all blank lines

Replace eligible text

In the use of sed Command to replace the operation needs to use s( String substitution ）、c（ The whole line / Replace the whole piece )、y( Character conversion ） Command options

The common usage is as follows ：

Example ：

1. First in each line “the” Replace with “THE”

2. Place the 2 individual “l” Replace with “L”

3. Put all... In the file “the” Replace with “THE”

4. Put all... In the file “o” Delete ( Replace with empty string )

5. Insert at the beginning of each line “#” Number

6. Include in “the” Insert at the beginning of each line of “#” Number

7. Insert a string at the end of each line " EOF"

8. Will be the first 3-5 All in row “the” Replace with “THE”

9. Will include “the” Of all rows “o” Replace with “0”

10. use sed modify /etc/sysconfig/selinux Enhancements in files

sed -i.bak 's/SELINUX=disabled/SELINUX=enable/' /etc/sysconfig/selinux

Migrate eligible text

In the use of sed When the command migrates qualified text , The following parameters are commonly used ：

H： Copy to clipboard 

g、G： Overlay the data in the clipboard / Append to specified line 

w： Save as a file 

r： Read the specified file 

a： Add the specified content 

I,i： Ignore case 

Example ：

1. Will include “the” Move to the end of the file ,"{ ; }" For multiple operations

2. Will be the first 1-5 Line content transferred to 16 After line

3. Will include “the” Save as file “kk.txt”

4. Will file /etc/hostname Add the content of to include “the” After each line of

5. In the 3 Insert a new row after row , The content is “New”

6. Include in “the” Insert a new row after each row of 、 The content is “New”

7. In the 3 Insert multiline content after row , In the middle of the “\n” Means line break

Editing files with scripts

Use …sed The script stores multiple editing instructions in a file （ One editing instruction per line ）, adopt " -f " Option to call .

for example :
Execute the following command to convert the 1-5 Line content transferred to 16 After line

sed '1,5{H;d};16G' test.txt

You can use script file mode instead :

[[email protected] ~]# vim ttt.list

1,5H
1,5d
16G


[[email protected] ~]# sed -f ttt.list  test.txt 
The year ahead will test our political establishment to the limit.
PI=3.141592653589793238462643383249901429
a wood cross!
Actions speak louder than words


#woood #
#woooooood
# AxyzxyzxyzxyzC
I bet this place is really spooky late at night!
Misfortunes never come alone/single.

he was short and fat.
He was wearing a blue polo shirt with black pants.
The home of Football on BBC Sport online.
the tongue is boneless but it breaks bones.12!
google is the best tools for search keyword.

sed Direct operation of documents

Example ：

Write a script , Used to adjust vsftpd Service configuration , Require anonymous users to be banned , But allow local users to （ It is also allowed to write ）

[[email protected] ~]#vim local_only_ftp.sh
 
#!/bin/bash
 
# Specify sample file path 、 Profile path 
SAMPLE="/usr/share/doc/vsftpd-3.0.2/EXAMPLE/INTERNET_STTE/vsftpd.conf"
CONFIG="/etc/vsftpd/vsftpd.conf"
 
# Back up the original profile , The test file name is /etc/vsftpd/vsftpd.conf.bak Whether the backup file exists , If not, use cp Command for file backup 
[ ! -e "SCONFIG.bak" ] && cp $CONFIG $CONFIG.bak 
 
# Adjust based on sample configuration , Overlay existing files 
sed -e '/^anonymous_enable/s/YES/NO/g' $SAMPLE >$CONFIG
sed -i -e '/^local_enable/s/NO/YES/g' -e '/^write_enable/s/NO/YES/g' $CONFIG
grep "listen" $CONFIG || sed -i '$alisten=YES' $CONFIG
 
# start-up vsftpd  service , And it is set to run automatically after power on 
systemctl restart vsftpd 
systemctl enable vsftpd
 
[[email protected] ~] #chmod +x local_only_ftp.sh

summary

One of the three Internet swordsmen sed Use