当前位置:网站首页>Shell_ 01_ data processing
Shell_ 01_ data processing
2022-07-06 16:50:00 【Coke loving w】
Shell_01_ Data processing
Data processing
Redirect / The Conduit
Redirect
File descriptor (File Descriptor):Linux A positive integer representing each file object
1) A single process can have at most 9 File descriptors ;
2)Linux Default hold 0、1 and 2 File descriptor ( Users can customize 6~8);
Temporary redirection
Output redirection : Save the execution result of the command to a specific file or device
1)Linux The default command in is output to the terminal after execution
2) Command format : Instructions Redirect Path to file
standard output (standard output)stdout: The code is 1
> | >> |
---|---|
Cover the output Overwrite the contents of the original file | Additional output Continue to add at the end of the existing file content |
1) It can also be expressed as 1> or 1>>, By default “1” Omit
2) If the specified file does not exist , The system will create this file
Such as : Use ls -la Inquire about “/” Post import ls.txt file
Standard error output (standard error output)stderr: The code is 2
2> | 2>> |
---|---|
Cover the output Overwrite the contents of the original file | Additional output Continue to add at the end of the existing file content |
// Numbers 2 Don't omit ( Used to distinguish standard output )
Such as : Inquire about /home All of them .bashrc file , And output to list1 and list2 file
stdout | stderr | |
---|---|---|
difference | After the command is executed successfully Correct information returned | After the command fails Error message returned |
//stdout and stderr You cannot use the output redirection symbol of the other party
// If the data of successful and failed operation are written in the same file , Special syntax is required “&>”
Such as : Inquire about /home All of them .bashrc File and enter the results into list file
// If two pieces of data are written to one at the same time , Special syntax should be used , If not “&>” It will lead to cross write or data loss
/dev/null:Linux The recycle bin ( Black holes )
1) equipment /dev/null Can digest any information directed to the device
Input redirection : Input data on the specified file or device
The standard input (standard input)stdin: The code is 0
< | << Delimiter |
---|---|
File content replaces keyboard input | Stop inputting when you encounter a delimiter |
Such as : Send using input redirection mail to mwl user
tee command : Two way redirection
Command format :tee Options File path
1) Send data streams to terminals and files respectively ( By default, the file is written in the form of overwrite )
Options | meaning |
---|---|
-a | Write files in append mode |
-i | Ignore interrupt signal |
Such as : Output $PATH Address the variable and save it to /root/test/test2.txt file
Permanent redirection
exec command : Realization Shell Permanent redirection of fragments in the script
Command format :exec Redirect format
1) stay exec The command specifies the output information after resetting , Save to exec Designated location ;
// stay exec Before the command is specified , Then save according to the default output ( Or follow the previous exec Specify save )
Such as : adopt exec Command to achieve permanent output redirection
1) To write test19.sh Script files
2) call test19.sh Script files
Such as : adopt exec The command implements permanent input redirection
1) To write test20.sh Script files
2) call test20.sh Script files
Custom redirection
exec command : Custom output / Input redirection
Command format :exec Numeric descriptor and redirection
1) File descriptors can store redirection attributes of other file descriptors
Such as : Create descriptor 3 Output redirection for
1) To write test21.sh Script files ;
2) call test21.sh Script files
// Custom output redirection is not specified , Output to by default stdout( Or the last permanent redirect )
Such as : adopt exec Command to record the attributes of other file descriptors
1) To write test22.sh Script files
2) call test22.sh Script files
mktemp command : Create a document in the current directory
Command format :mktemp Options Document prefix name .XXXXXX
Options | meaning |
---|---|
-t | stay /tmp Create documents under the directory |
-d | Create directory ( Create file by default ) |
1) The created document is not affected by umask Limit , Only the owner owns rw jurisdiction ;
2) The system will randomly fill “XXXXXX”, To achieve the uniqueness of creating documents ;
Such as : adopt mktemp Command create file
The Conduit
** The Conduit :** be used for “ Filtering treatment ”、“ Special treatment ”、“ Extended processing ”
1) Symbolic form “|”
2) Pipes cannot be used alone , Must be used with instructions ( The essence is auxiliary )
// Pipelines can be understood as dividing lines . The output in front of the pipeline is the input of the following instructions
Such as : The query root directory contains “y” The document name of the letter
Such as : Count the total number of documents in a directory
xargs command (x arguments): Filters that pass parameters to other commands
Command format :xargs Options command
1) No command , The default command is echo
Options | meaning |
---|---|
-n | Execute the command with several parameters |
-p | Execute each command and ask whether to execute |
-e | Set end character |
-0 | Restore special characters to general characters |
Such as : adopt id Command query /etc/passwd Information of the first three users in the file
1)xargs The newline character and space character in the output information through the pipeline will be replaced by space character
// Many commands do not support pipelines , But through xargs Provides command parameters to realize multiple commands in a single line
format
file
join command : Specify the same field to connect according to the two files
Command format :join Options File path 1 File path 2
Options | meaning |
---|---|
-t | Specify the delimiter of the data |
-i | Ignore case |
-1 | Specified file 1 Field of |
-2 | Specified file 2 Field of |
1) The default is the second of two files 1 Compare fields
// Field : Data divided by separator in each row
Such as : Will file /etc/passwd And documents /etc/group Integrate
// because GID stay /etc/passwd pass the civil examinations 4 Fields , stay /etc/group The third field in
paste command : Merge multiple files in a queue
Command format :paste file 1 file N
1) Whether the contents of the document are the same or not , Will be merged according to the number of rows
split command : In size / The number of lines divides a file into multiple data
Command format :split Options File path Prefix
Options | meaning |
---|---|
-b | Split data by file size |
-l | Split the file by the number of lines |
1) After the file is split , Its name is : Prefix aa、 Prefix ab、 Prefix ac And so on
// If there is no prefix name , Default to aa、ab、ac Name and so on
2) After segmentation, the original file will be saved
Such as : take /root/etc/passwd File by 10 Split rows and save to /root/test Under the folder
Format
tr command : Modify the specified characters in the data
Command format 1:tr Options character // Modify character
Options | meaning |
---|---|
-d | Delete keywords in the data |
-s | Delete consecutive repeated characters to , And keep one |
Command format 2:tr Original character Target character // Replace character
Such as : Use tr adjustment echo Output
expand command : Put the tabs in the data (Tab) Convert to space
Command format :expand Options Document path
Options | meaning |
---|---|
-t | Define a Tab How many blank characters does the key equal ( The default is 8 Characters ) |
1)unexpand The effect of the command is the opposite ( The usage is the same )
col command : Adjust input / Output content
Command format :col Options
Options | meaning |
---|---|
-x | Replace the skip character with multiple space characters (Tab) |
-b | Delete all control characters ( Include RLF and HRLF) |
uniq command : Show / Ignore duplicate rows in data ( Default ignore )
Command format :uniq Options
Options | meaning |
---|---|
-i | Ignore case |
-c | Displays the number of repetitions |
-d | Show only recurring rows and columns |
1) Generally speaking, it is related to sort Command in combination with
Output
grep command : Find the data containing keywords in the data and output
Command format :grep Options key word Document path
1)grep After the keyword is found, the data is intercepted in behavioral units for output ;
2)egrep The order is grep Command upgrade , It supports POSIX Extended regular expression ;
Options | meaning |
---|---|
-a | Find data in the form of binary file as text file |
-c | Calculate the number of rows with keywords |
-i | Ignore case differences |
-n | Output line number |
-v | Find data without keywords |
-E | Support extended expression |
sort command : Sort the data in the file and output
Command format :sort Options File path
Options | meaning |
---|---|
-f | Ignore case ( By default, uppercase letters are placed first ) |
-b | Ignore the space at the beginning of the line |
-n | Sort numbers in numerical form ( By default, numbers are sorted as characters ) |
-M | Sort by month |
-r | Reverse sorting |
-t | Specify the delimiter of the data ( The separator must be ‘’ Cover up ) |
-k | Specify which interval to sort |
1) The default is ASCII Code value comparison for sorting , And the sorted characters are related to the coding of language family
2)-t and -k It makes sense to use them together ( similar cut Of -d and -f)
echo command : Output the extracted value of the specified string or variable to the terminal
Command format :echo String or ${ Variable }
1) If the string contains spaces , It needs to be enclosed in double quotation marks ;
2) If the string contains single quotation marks or double quotation marks , Use different quotation marks
// When using single quotation marks , Use double quotes ( conversely , Use single quotation marks )
Such as : Output Hello,World and SHELL Variable content to terminal
Such as : Output the contents with quotation marks to the terminal
printf command : Format and output the result to stdout
Command format :printf ‘ Output format ’ Print the content
Format | meaning |
---|---|
%s | String format output |
%i | Output in integer format |
%f | Floating point format output |
%% | Output % |
1)printf Not a pipeline command
2) There are some special escape characters ( Keyboard can't input ) Use envoy format instead
Special format | meaning |
---|---|
\a | Output warning sound |
\b | Backspace |
\f | Clear the screen |
\n | Output a new line |
\r | The next line ( Enter key ) |
\t | level Tab Key |
\v | vertical Tab Key |
Such as : Format conversion output /root/test/test1.txt The contents of the document
command | difference |
---|---|
printf | Ksh Built in commands , regular expression Don't wrap |
Bash Built in commands Word wrap |
边栏推荐
- Educational Codeforces Round 122 (Rated for Div. 2)
- LeetCode 1557. The minimum number of points that can reach all points
- LeetCode 1636. Sort the array in ascending order by frequency
- ~Introduction to form 80
- Chapter 6 rebalance details
- 第6章 Rebalance详解
- Chapter 2 shell operation of hfds
- LeetCode 1641. Count the number of Lexicographic vowel strings
- LeetCode 1447. Simplest fraction
- Jedis
猜你喜欢
Hbuilder x format shortcut key settings
Redis standalone startup
视频压缩编码和音频压缩编码基本原理
ByteDance new programmer's growth secret: those glittering treasures mentors
Chapter 7__ consumer_ offsets topic
The most lost road I have ever walked through is the brain circuit of ByteDance programmers
SQL quick start
One hundred questions of image processing (11-20)
ByteDance 2022 school recruitment R & D advance approval publicity meeting, students' top 10 issues
Mp4 format details
随机推荐
~68 Icon Font introduction
Research Report on market supply and demand and strategy of China's four seasons tent industry
Spark's RDD (elastic distributed data set) returns a large result set
Market trend report, technological innovation and market forecast of desktop electric tools in China
LeetCode 1636. Sort the array in ascending order by frequency
~76 sprite map
第五章 Yarn资源调度器
(lightoj - 1349) Aladdin and the optimal invitation (greed)
谢邀,人在工区,刚交代码,在下字节跳动实习生
The 116 students spent three days reproducing the ByteDance internal real technology project
力扣leetcode第 280 场周赛
~74 JD top navigation bar exercise
7-5 blessing arrived
JS time function Daquan detailed explanation ----- AHAO blog
7-4 harmonic average
LeetCode 1558. Get the minimum number of function calls of the target array
字节跳动新程序员成长秘诀:那些闪闪发光的宝藏mentor们
第三章 MapReduce框架原理
LeetCode1556. Thousand separated number
Market trend report, technical innovation and market forecast of double-sided foam tape in China