当前位置:网站首页>Shell_ 01_ data processing
Shell_ 01_ data processing
2022-07-06 16:50:00 【Coke loving w】
Shell_01_ Data processing
Data processing
Redirect / The Conduit
Redirect
File descriptor (File Descriptor):Linux A positive integer representing each file object
1) A single process can have at most 9 File descriptors ;
2)Linux Default hold 0、1 and 2 File descriptor ( Users can customize 6~8);
Temporary redirection
Output redirection : Save the execution result of the command to a specific file or device
1)Linux The default command in is output to the terminal after execution
2) Command format : Instructions Redirect Path to file
standard output (standard output)stdout: The code is 1
> | >> |
---|---|
Cover the output Overwrite the contents of the original file | Additional output Continue to add at the end of the existing file content |
1) It can also be expressed as 1> or 1>>, By default “1” Omit
2) If the specified file does not exist , The system will create this file
Such as : Use ls -la Inquire about “/” Post import ls.txt file
Standard error output (standard error output)stderr: The code is 2
2> | 2>> |
---|---|
Cover the output Overwrite the contents of the original file | Additional output Continue to add at the end of the existing file content |
// Numbers 2 Don't omit ( Used to distinguish standard output )
Such as : Inquire about /home All of them .bashrc file , And output to list1 and list2 file
stdout | stderr | |
---|---|---|
difference | After the command is executed successfully Correct information returned | After the command fails Error message returned |
//stdout and stderr You cannot use the output redirection symbol of the other party
// If the data of successful and failed operation are written in the same file , Special syntax is required “&>”
Such as : Inquire about /home All of them .bashrc File and enter the results into list file
// If two pieces of data are written to one at the same time , Special syntax should be used , If not “&>” It will lead to cross write or data loss
/dev/null:Linux The recycle bin ( Black holes )
1) equipment /dev/null Can digest any information directed to the device
Input redirection : Input data on the specified file or device
The standard input (standard input)stdin: The code is 0
< | << Delimiter |
---|---|
File content replaces keyboard input | Stop inputting when you encounter a delimiter |
Such as : Send using input redirection mail to mwl user
tee command : Two way redirection
Command format :tee Options File path
1) Send data streams to terminals and files respectively ( By default, the file is written in the form of overwrite )
Options | meaning |
---|---|
-a | Write files in append mode |
-i | Ignore interrupt signal |
Such as : Output $PATH Address the variable and save it to /root/test/test2.txt file
Permanent redirection
exec command : Realization Shell Permanent redirection of fragments in the script
Command format :exec Redirect format
1) stay exec The command specifies the output information after resetting , Save to exec Designated location ;
// stay exec Before the command is specified , Then save according to the default output ( Or follow the previous exec Specify save )
Such as : adopt exec Command to achieve permanent output redirection
1) To write test19.sh Script files
2) call test19.sh Script files
Such as : adopt exec The command implements permanent input redirection
1) To write test20.sh Script files
2) call test20.sh Script files
Custom redirection
exec command : Custom output / Input redirection
Command format :exec Numeric descriptor and redirection
1) File descriptors can store redirection attributes of other file descriptors
Such as : Create descriptor 3 Output redirection for
1) To write test21.sh Script files ;
2) call test21.sh Script files
// Custom output redirection is not specified , Output to by default stdout( Or the last permanent redirect )
Such as : adopt exec Command to record the attributes of other file descriptors
1) To write test22.sh Script files
2) call test22.sh Script files
mktemp command : Create a document in the current directory
Command format :mktemp Options Document prefix name .XXXXXX
Options | meaning |
---|---|
-t | stay /tmp Create documents under the directory |
-d | Create directory ( Create file by default ) |
1) The created document is not affected by umask Limit , Only the owner owns rw jurisdiction ;
2) The system will randomly fill “XXXXXX”, To achieve the uniqueness of creating documents ;
Such as : adopt mktemp Command create file
The Conduit
** The Conduit :** be used for “ Filtering treatment ”、“ Special treatment ”、“ Extended processing ”
1) Symbolic form “|”
2) Pipes cannot be used alone , Must be used with instructions ( The essence is auxiliary )
// Pipelines can be understood as dividing lines . The output in front of the pipeline is the input of the following instructions
Such as : The query root directory contains “y” The document name of the letter
Such as : Count the total number of documents in a directory
xargs command (x arguments): Filters that pass parameters to other commands
Command format :xargs Options command
1) No command , The default command is echo
Options | meaning |
---|---|
-n | Execute the command with several parameters |
-p | Execute each command and ask whether to execute |
-e | Set end character |
-0 | Restore special characters to general characters |
Such as : adopt id Command query /etc/passwd Information of the first three users in the file
1)xargs The newline character and space character in the output information through the pipeline will be replaced by space character
// Many commands do not support pipelines , But through xargs Provides command parameters to realize multiple commands in a single line
format
file
join command : Specify the same field to connect according to the two files
Command format :join Options File path 1 File path 2
Options | meaning |
---|---|
-t | Specify the delimiter of the data |
-i | Ignore case |
-1 | Specified file 1 Field of |
-2 | Specified file 2 Field of |
1) The default is the second of two files 1 Compare fields
// Field : Data divided by separator in each row
Such as : Will file /etc/passwd And documents /etc/group Integrate
// because GID stay /etc/passwd pass the civil examinations 4 Fields , stay /etc/group The third field in
paste command : Merge multiple files in a queue
Command format :paste file 1 file N
1) Whether the contents of the document are the same or not , Will be merged according to the number of rows
split command : In size / The number of lines divides a file into multiple data
Command format :split Options File path Prefix
Options | meaning |
---|---|
-b | Split data by file size |
-l | Split the file by the number of lines |
1) After the file is split , Its name is : Prefix aa、 Prefix ab、 Prefix ac And so on
// If there is no prefix name , Default to aa、ab、ac Name and so on
2) After segmentation, the original file will be saved
Such as : take /root/etc/passwd File by 10 Split rows and save to /root/test Under the folder
Format
tr command : Modify the specified characters in the data
Command format 1:tr Options character // Modify character
Options | meaning |
---|---|
-d | Delete keywords in the data |
-s | Delete consecutive repeated characters to , And keep one |
Command format 2:tr Original character Target character // Replace character
Such as : Use tr adjustment echo Output
expand command : Put the tabs in the data (Tab) Convert to space
Command format :expand Options Document path
Options | meaning |
---|---|
-t | Define a Tab How many blank characters does the key equal ( The default is 8 Characters ) |
1)unexpand The effect of the command is the opposite ( The usage is the same )
col command : Adjust input / Output content
Command format :col Options
Options | meaning |
---|---|
-x | Replace the skip character with multiple space characters (Tab) |
-b | Delete all control characters ( Include RLF and HRLF) |
uniq command : Show / Ignore duplicate rows in data ( Default ignore )
Command format :uniq Options
Options | meaning |
---|---|
-i | Ignore case |
-c | Displays the number of repetitions |
-d | Show only recurring rows and columns |
1) Generally speaking, it is related to sort Command in combination with
Output
grep command : Find the data containing keywords in the data and output
Command format :grep Options key word Document path
1)grep After the keyword is found, the data is intercepted in behavioral units for output ;
2)egrep The order is grep Command upgrade , It supports POSIX Extended regular expression ;
Options | meaning |
---|---|
-a | Find data in the form of binary file as text file |
-c | Calculate the number of rows with keywords |
-i | Ignore case differences |
-n | Output line number |
-v | Find data without keywords |
-E | Support extended expression |
sort command : Sort the data in the file and output
Command format :sort Options File path
Options | meaning |
---|---|
-f | Ignore case ( By default, uppercase letters are placed first ) |
-b | Ignore the space at the beginning of the line |
-n | Sort numbers in numerical form ( By default, numbers are sorted as characters ) |
-M | Sort by month |
-r | Reverse sorting |
-t | Specify the delimiter of the data ( The separator must be ‘’ Cover up ) |
-k | Specify which interval to sort |
1) The default is ASCII Code value comparison for sorting , And the sorted characters are related to the coding of language family
2)-t and -k It makes sense to use them together ( similar cut Of -d and -f)
echo command : Output the extracted value of the specified string or variable to the terminal
Command format :echo String or ${ Variable }
1) If the string contains spaces , It needs to be enclosed in double quotation marks ;
2) If the string contains single quotation marks or double quotation marks , Use different quotation marks
// When using single quotation marks , Use double quotes ( conversely , Use single quotation marks )
Such as : Output Hello,World and SHELL Variable content to terminal
Such as : Output the contents with quotation marks to the terminal
printf command : Format and output the result to stdout
Command format :printf ‘ Output format ’ Print the content
Format | meaning |
---|---|
%s | String format output |
%i | Output in integer format |
%f | Floating point format output |
%% | Output % |
1)printf Not a pipeline command
2) There are some special escape characters ( Keyboard can't input ) Use envoy format instead
Special format | meaning |
---|---|
\a | Output warning sound |
\b | Backspace |
\f | Clear the screen |
\n | Output a new line |
\r | The next line ( Enter key ) |
\t | level Tab Key |
\v | vertical Tab Key |
Such as : Format conversion output /root/test/test1.txt The contents of the document
command | difference |
---|---|
printf | Ksh Built in commands , regular expression Don't wrap |
Bash Built in commands Word wrap |
边栏推荐
- Audio and video development interview questions
- 「博士毕业一年,我拿下 ACL Best Paper」
- Saw local status change event StatusChangeEvent [timestamp=1644048792587, current=DOWN, previous=UP]
- One hundred questions of image processing (11-20)
- Research Report on market supply and demand and strategy of China's tetraacetylethylenediamine (TAED) industry
- ~70 row high
- ~85 transition
- Research Report on hearing health care equipment industry - market status analysis and development prospect prediction
- ~68 Icon Font introduction
- Eureka single machine construction
猜你喜欢
Ffmpeg command line use
「博士毕业一年,我拿下 ACL Best Paper」
~Introduction to form 80
字节跳动技术新人培训全记录:校招萌新成长指南
7-10 punch in strategy
~83 form introduction
7-4 harmonic average
The concept of spark independent cluster worker and executor
LeetCode 1560. The sector with the most passes on the circular track
~69 other ways to use icon fonts
随机推荐
LeetCode 1020. Number of enclaves
~68 Icon Font introduction
Market trend report, technical innovation and market forecast of double-sided foam tape in China
~81 long table
图像处理一百题(1-10)
LeetCode 1984. Minimum difference in student scores
The most lost road I have ever walked through is the brain circuit of ByteDance programmers
【锟斤拷】的故事:谈谈汉字编码和常用字符集
~79 Movie card exercise
业务系统从Oracle迁移到openGauss数据库的简单记录
第一章 MapReduce概述
~77 linear gradient
解决Intel12代酷睿CPU单线程只给小核运行的问题
LeetCode 1640. Can I connect to form an array
Audio and video development interview questions
LeetCode 1550. There are three consecutive arrays of odd numbers
力扣leetcode第 280 场周赛
Research Report on hearing health care equipment industry - market status analysis and development prospect prediction
Submit several problem records of spark application (sparklauncher with cluster deploy mode)
音视频开发面试题