Common text processing tools

1. File content view command

---

1.1 View the contents of the text file

1.1.1 cat

cat You can view the text content

Format ：

cat [OPTION]... [FILE]...

Common options ：

• -E： Show line end symbol $
• -A： Show all controls
• -n： Number each line shown
• -b： Non empty line number
• -s： Compress consecutive empty lines into one line

1.1.2 nl

According to the line Numbers , amount to cat -b

1.1.3 tac

Reverse display text content

1.1.4 rev

Reverse the contents of the same line

1.2 View non text file content

1.2.1 hexdump

1.2.2 od

1.2.3 xxd

2. View the contents of the file in pages

---

2.1 more

Can achieve paging view file , It can be used to paginate the output information with the pipeline

Format

more [OPTIONS...] FILE...

Options ：

• -d： Display page turning and exit display

2.2 less

less You can also view files or STDIN Output ,less The order is man The pager used by the command

Useful commands for viewing include ：

/ Text   Search for   Text 
n/N  Skip to the next   or   Last match 

3. Show text before or after line

---

3.1 head

You can display the front line of a file or standard input

Format ：

head [OPTION]... [FILE]...

Options ：

• -c # Specify before getting # byte
• -n # Specify before getting # That's ok ,# If it's negative , It means to get the penultimate from the file header # front
• -# ditto

3.2 tail

tail and head contrary , The standard input file or the inverted line view

Format ：

tail [OPTION]... [FILE]...

Common options ：

• -c # After specifying acquisition # byte
• -n # After specifying acquisition # That's ok , If # It's a negative number , Says from the first # From the beginning of the line to the end of the file
• -# ditto
• -f Trace display file fd New additions , Common log monitoring , amount to --follow=descriptor, When a file is deleted, a new file with the same name is created , Will not be able to continue tracking files
• -F Trace filename , amount to --follow=name --retry, When the file is deleted, create a file with the same name , Will be able to continue tracking files
• tailf similar tail -f, Don't access files when they don't grow , Saving resource ,centos8 No such tool

4. Extract text by column cut

---

cut Commands can extract text files or STDIN The specified column of data

Format

cut [OPTION]... [FILE]...

Common options ：

• -d DELIMITER: Indicates the separator , Default tab
• -f FILEDS:

            #: The first # A field , for example :3

            #,#[,#]： Discrete multiple fields , for example :1,3,6

            #-#： Continuous multiple fields , for example :1-6

             A mixture of ：1-3,7

• -c Cut by character
• --output-delimiter=STRING Specify output separator

5. Merge multiple files paste

---

paste Merge the rows of multiple documents with the same line number

Format

paste [OPTION]... [FILE]...

Common options ：

• -d # Separator ： Specify the separator , The default with TAB
• -s # All rows are merged into one row to display

6. Tools for analyzing text

---

Text Statistics ：wc

Organize the text ：sort

Compare files ：diff and patch

6.1 Collect text Statistics wc

---

wc The command can be used to count the total number of lines in a file 、 The total number of words 、 The total number of bytes and the total number of characters

It can be done on files or STDIN Data statistics in

Common options ：

• -l        Just count the number of lines
• -w      Just count the total number of words
• -c      Count only the total number of bytes
• -m    Count only the total number of characters
• -L    Displays the length of the longest line in the file

6.2 Text sorting sort

Display the collated text in STDOUT, Don't change the original file

Format

sort [options] file(s)

Common options ：

• -r Execute in the opposite direction （ From top to bottom ） Arrangement
• -R Stochastic ranking
• -n Perform sorting by number
• -h Human readable sequencing , Such as : 2K 1G
• -f Option ignored （fold） The case of characters in a string
• -u Options （ unique ,unique）, Merge duplicates , That is to say, to remove the weight
• -t c Option to use c As a field qualifier
• -k # Options according to use c Character separated # Column can be used more than once

6.3 duplicate removal uniq

uniq The command removes duplicate lines from input

Format ：

uniq [OPTION]... [FILE]...

Common options ：

• -c: Shows how many times each line repeats
• -d: Show only repeated lines
• -u: Show only lines that have not been repeated

uniq Constant harmony sort Commands are used together

6.4 Compare files

6.4.1diff

diff Command to compare the differences between two files

-u  Option to output “ A unified （unified）”diff Format file , Best for patch files 

6.4.2 patch

patch Copy changes made in other files （ Use... With caution ）

-b  Option to automatically back up changed files 

6.4.3 vimdiff

amount to vim -d

6.4.4 cmp

Example ： Look at the different binary files

---