当前位置：网站首页>Chapter 6 text processing tools for shell programming (awk)

Chapter 6 text processing tools for shell programming (awk)

2022-07-05 04:32:00 【Follow the wind and die】

One 、awk Introduce

1. awk summary

awk It's a kind of programing language , Mainly used in linux/unix Next pair Text and data To deal with , yes linux/unix Next tool . Data can come from standard inputs 、 One or more files , Or other command output .
awk How to process text and data ： progressive scanning file , Default from the first line to the last line , Look for a match Specific patterns The line of , And do what you want on these lines .
awk The first letter of the author's surname . Because its author is three people , Namely Alfred Aho、Brian Kernighan、Peter Weinberger.
gawk yes awk Of GNU edition , It provides Bell Laboratory and GNU Some extensions of .
What follows awk In order to GNU Of gawk As an example of , stay linux The system has awk link to gawk, So all of the following are awk To introduce .

2. awk What can I do ?

awk Used to process files and data Of , It's a class unix Next tool , It's also a programming language
Can be used to statistics , For example, website visits , Access to the IP Quantity and so on
Support conditional judgment , Support for and while loop

Two 、awk Usage mode

1. Command line mode uses

㈠ Grammatical structure

awk  Options  ' Command part '  file name 


 In particular ：
 quote shell Variables are enclosed in double quotes

㈡ Introduction to common options

-F Define field segmentation symbols , The default separator is Space
-v Define variables and assign values

###㈢ ' Name part description ’

Regular expressions , Address location

'/root/{awk sentence }'				sed in ： '/root/p'
'NR==1,NR==5{awk sentence }'			sed in ： '1,5p'
'/^root/,/^ftp/{awk sentence }'  	sed in ：'/^root/,/^ftp/p'

{awk sentence 1**;awk sentence 2;**…}

'{print $0;print $1}'		sed in ：'p'
'NR==5{print $0}'				sed in ：'5p'
 notes ：awk Use semicolon interval between command statements

BEGIN…END…

'BEGIN{awk sentence };{ In processing };END{awk sentence }'
'BEGIN{awk sentence };{ In processing }'
'{ In processing };END{awk sentence }'

2. Script mode uses

㈠ scripting

#!/bin/awk -f  Define magic characters 
 Here are awk List of commands in quotes , Don't use quotes to protect commands , Multiple commands are separated by semicolons 
BEGIN{
    FS=":"}
NR==1,NR==3{
    print $1"\t"$NF}
...

㈡ Script execution

 Method 1：
awk  Options  -f awk Script file for    The text file to be processed 
awk -f awk.sh filename

sed -f sed.sh -i filename

 Method 2：
./awk Script file for ( Or the absolute path )	 The text file to be processed 
./awk.sh filename

./sed.sh filename

3、 ... and 、 awk Internal related variables

Variable	Variable description	remarks
$0	All records of the current processing line
$1,$2,$3…$n	Each line in the document is marked with Space symbol Different fields split	awk -F: ‘{print $1,$3}’
NF	The number of fields in the current record （ Number of columns ）	awk -F: ‘{print NF}’
$NF	The last column	$(NF-1) Represents the penultimate column
FNR/NR	Line number
FS	Define the separator	‘BEGIN{FS=":"};{print $1,$3}’
OFS	Define output field separator , Default space	‘BEGIN{OFS="\t"};print $1,$3}’
RS	Enter the record separator , Default newline	‘BEGIN{RS="\t"};{print $0}’
ORS	Output record separator , Default newline	‘BEGIN{ORS="\n\n"};{print $1,$3}’
FILENAME	The file name currently entered

1、 Common examples of built-in variables

# awk -F: '{print $1,$(NF-1)}' 1.txt
# awk -F: '{print $1,$(NF-1),$NF,NF}' 1.txt
# awk '/root/{print $0}' 1.txt
# awk '/root/' 1.txt
# awk -F: '/root/{print $1,$NF}' 1.txt 
root /bin/bash
# awk -F: '/root/{print $0}' 1.txt 
root:x:0:0:root:/root:/bin/bash
# awk 'NR==1,NR==5' 1.txt 
# awk 'NR==1,NR==5{print $0}' 1.txt
# awk 'NR==1,NR==5;/^root/{print $0}' 1.txt 
root:x:0:0:root:/root:/bin/bash
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin

2、 Examples of built-in variable separators

FS and OFS:
# awk 'BEGIN{FS=":"};/^root/,/^lp/{print $1,$NF}' 1.txt
# awk -F: 'BEGIN{OFS="\t\t"};/^root/,/^lp/{print $1,$NF}' 1.txt 
root            /bin/bash
bin             /sbin/nologin
daemon          /sbin/nologin
adm             /sbin/nologin
lp              /sbin/nologin
# awk -F: 'BEGIN{OFS="@@@"};/^root/,/^lp/{print $1,$NF}' 1.txt 
[email protected]@@/bin/bash
[email protected]@@/sbin/nologin
[email protected]@@/sbin/nologin
[email protected]@@/sbin/nologin
lp@@@/sbin/nologin
[[email protected] shell07]# 

RS and ORS：
 Before modifying the source file 2 Add tabs and contents to rows ：
vim 1.txt
root:x:0:0:root:/root:/bin/bash hello   world
bin:x:1:1:bin:/bin:/sbin/nologin        test1   test2

# awk 'BEGIN{RS="\t"};{print $0}' 1.txt
# awk 'BEGIN{ORS="\t"};{print $0}' 1.txt

Four 、 awk working principle

awk -F: '{print $1,$3}' /etc/passwd

awk Use a line as input , And assign this line to the internal variable $0, Each line can also be called a record , With a line break (RS) end
Each line is spaced **:**( The default is space or tab ) Break down into fields ( Or domain ), Each field is stored in a numbered variable , from $1 Start
ask ：awk How to know how to separate fields with spaces ？
answer ： Because there's an internal variable FS To determine the field separator . At the beginning ,FS Assign as space
awk Use print Function print fields , The printed field will be in The blank space to separate , because $1,$3 There is a comma between . Commas are special , It maps to another internal variable , be called Output field separator OFS,OFS Default is space
awk After processing a line , Will get another line from the file , And store it in $0 in , Cover the original content , The new string is then separated into fields and processed . This process will continue until all lines are processed

5、 ... and 、awk Use advanced

1. Format output `print` and `printf`

print function 		 similar echo "hello world"
# date |awk '{print "Month: "$2 "\nYear: "$NF}'
# awk -F: '{print "username is: " $1 "\t uid is: "$3}' /etc/passwd


printf function 		 similar echo -n
# awk -F: '{printf "%-15s %-10s %-15s\n", $1,$2,$3}' /etc/passwd
# awk -F: '{printf "|%15s| %10s| %15s|\n", $1,$2,$3}' /etc/passwd
# awk -F: '{printf "|%-15s| %-10s| %-15s|\n", $1,$2,$3}' /etc/passwd

awk 'BEGIN{FS=":"};{printf "%-15s %-15s %-15s\n",$1,$6,$NF}' a.txt

%s  Character type   strings			%-20s
%d  value type 	
 Occupy 15 character 
-  Indicates left alignment , The default is right alignment 
printf The default is not to wrap at the end of the line , Add \n

2. awk Variable definitions

# awk -v NUM=3 -F: '{ print $NUM }' /etc/passwd
# awk -v NUM=3 -F: '{ print NUM }' /etc/passwd
# awk -v num=1 'BEGIN{print num}' 
1
# awk -v num=1 'BEGIN{print $num}' 
 Be careful ：
awk The variables defined in the call do not need to add $

3. awk in BEGIN…END Use

①BEGIN： It means that Before the program starts perform

②END ： Means all documents After processing perform

③ usage ：'BEGIN{ Before you start processing };{ In processing };END{ After processing }'

㈠ Illustrate with examples 1

Print the last and last Columns （ Sign in shell And home directory ）

awk -F: 'BEGIN{ print "Login_shell\t\tLogin_home\n*******************"};{print $NF"\t\t"$(NF-1)};END{print "************************"}' 1.txt

awk 'BEGIN{ FS=":";print "Login_shell\tLogin_home\n*******************"};{print $NF"\t"$(NF-1)};END{print "************************"}' 1.txt

Login_shell		Login_home
************************
/bin/bash		/root
/sbin/nologin		/bin
/sbin/nologin		/sbin
/sbin/nologin		/var/adm
/sbin/nologin		/var/spool/lpd
/bin/bash		/home/redhat
/bin/bash		/home/user01
/sbin/nologin		/var/named
/bin/bash		/home/u01
/bin/bash		/home/YUNWEI
************************************

㈡ Illustrate with examples 2

Print /etc/passwd The user name in 、 Home directory and login shell

u_name      h_dir       shell
***************************

***************************

awk -F: 'BEGIN{OFS="\t\t";print"u_name\t\th_dir\t\tshell\n***************************"};{printf "%-20s %-20s %-20s\n",$1,$(NF-1),$NF};END{print "****************************"}'


# awk -F: 'BEGIN{print "u_name\t\th_dir\t\tshell" RS "*****************"} {printf "%-15s %-20s %-20s\n",$1,$(NF-1),$NF}END{print "***************************"}' /etc/passwd

 Format output ：
echo		print
echo -n	printf

{
    printf "%-15s %-20s %-20s\n",$1,$(NF-1),$NF}

###4. awk And regular application

Operator	explain
==	be equal to
!=	It's not equal to
>	Greater than
<	Less than
>=	Greater than or equal to
<=	Less than or equal to
~	matching
!~	Mismatch
!	Logic is not
&&	Logic and
\|\|	Logic or

㈠ Illustrate with examples

 Match from the first line to lp Start line 
awk -F: 'NR==1,/^lp/{print $0 }' passwd  
 From the first line to the 5 That's ok           
awk -F: 'NR==1,NR==5{print $0 }' passwd
 From the lp The first line matches to 10 That's ok        
awk -F: '/^lp/,NR==10{print $0 }' passwd 
 From the root The first line matches with lp Beginning line        
awk -F: '/^root/,/^lp/{print $0}' passwd
 Print to root Start with or with lp Beginning line             
awk -F: '/^root/ || /^lp/{print $0}' passwd
awk -F: '/^root/;/^lp/{print $0}' passwd
 Show 5-10 That's ok    
awk -F':' 'NR>=5 && NR<=10 {print $0}' /etc/passwd     
awk -F: 'NR<10 && NR>5 {print $0}' passwd 

 Print 30-39 Line to bash Ending content ：
[[email protected] shell06]# awk 'NR>=30 && NR<=39 && $0 ~ /bash$/{print $0}' passwd 
stu1:x:500:500::/home/stu1:/bin/bash
yunwei:x:501:501::/home/yunwei:/bin/bash
user01:x:502:502::/home/user01:/bin/bash
user02:x:503:503::/home/user02:/bin/bash
user03:x:504:504::/home/user03:/bin/bash

[[email protected] shell06]# awk 'NR>=3 && NR<=8 && /bash$/' 1.txt 
stu7:x:1007:1007::/rhome/stu7:/bin/bash
stu8:x:1008:1008::/rhome/stu8:/bin/bash
stu9:x:1009:1009::/rhome/stu9:/bin/bash

 In a printed document 1-5 And take root Beginning line 
[[email protected] shell06]# awk 'NR>=1 && NR<=5 && $0 ~ /^root/{print $0}' 1.txt
root:x:0:0:root:/root:/bin/bash
[[email protected] shell06]# awk 'NR>=1 && NR<=5 && $0 !~ /^root/{print $0}' 1.txt
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin


 understand ; Number and || The meaning of ：
[[email protected] shell06]# awk 'NR>=3 && NR<=8 || /bash$/' 1.txt
[[email protected] shell06]# awk 'NR>=3 && NR<=8;/bash$/' 1.txt


 Print IP Address 
# ifconfig eth0|awk 'NR>1 {print $2}'|awk -F':' 'NR<2 {print $2}' 
# ifconfig eth0|grep Bcast|awk -F':' '{print $2}'|awk '{print $1}'
# ifconfig eth0|grep Bcast|awk '{print $2}'|awk -F: '{print $2}'


# ifconfig eth0|awk NR==2|awk -F '[ :]+' '{print $4RS$6RS$8}'
# ifconfig eth0|awk -F"[ :]+" '/inet addr:/{print $4}'

4. Practice cases

Display all information of users who can log in to the operating system From 7 Columns match with bash ending , Output line （ All columns in the current row ）

[[email protected] ~] awk '/bash$/{print $0}'    /etc/passwd
[[email protected] ~] awk '/bash$/{print $0}' /etc/passwd
[[email protected] ~] awk '/bash$/' /etc/passwd
[[email protected] ~] awk -F: '$7 ~ /bash/' /etc/passwd
[[email protected] ~] awk -F: '$NF ~ /bash/' /etc/passwd
[[email protected] ~] awk -F: '$0 ~ /bash/' /etc/passwd
[[email protected] ~] awk -F: '$0 ~ /\/bin\/bash/' /etc/passwd

Display the user name that can log in to the system

# awk -F: '$0 ~ /\/bin\/bash/{print $1}' /etc/passwd

Print out the UID And the user name

500	stu1
501	yunwei
502	user01
503	user02
504	user03


# awk -F: 'BEGIN{print "UID\tUSERNAME"} {if($3>=500 && $3 !=65534 ) {print $3"\t"$1} }' /etc/passwdUID USERNAME


# awk -F: '{if($3 >= 500 && $3 != 65534) print $1,$3}' a.txt 
redhat 508
user01 509
u01 510
YUNWEI 511

5. awk Script programming for

㈠ Flow control statement

① if structure

if sentence ：

if [ xxx ];then
xxx
fi

 Format ：
awk  Options  ' Regular , Address location {awk sentence }'   file name 

{
     if( expression )｛ sentence 1; sentence 2;...｝}

awk -F: '{if($3>=500 && $3<=60000) {print $1,$3} }' passwd

# awk -F: '{if($3==0) {print $1" It's the administrator "} }' passwd 
root It's the administrator 

# awk 'BEGIN{if('$(id -u)'==0) {print "admin"} }'
admin

② if…else structure

if...else sentence :
if [ xxx ];then
	xxxxx
	
else
	xxx
fi

 Format ：
{
    if( expression )｛ sentence ; sentence ;...｝else｛ sentence ; sentence ;...}}

awk -F: '{ if($3>=500 && $3 != 65534) {print $1" It's the average user "} else {print $1," Not ordinary users "}}' passwd 

awk 'BEGIN{if( '$(id -u)'>=500 && '$(id -u)' !=65534 ) {print " It's the average user "} else {print " Not ordinary users "}}'

③ if…elif…else structure

if [xxxx];then
	xxxx
elif [xxx];then
	xxx
....
else
...
fi


if...else if...else sentence ：

 Format ：
{
     if( expression 1)｛ sentence ; sentence ;...｝else if( expression 2)｛ sentence ; sentence ;...｝else if( expression 3)｛ sentence ; sentence ;...｝else｛ sentence ; sentence ;...｝}

awk -F: '{ if($3==0) {print $1,": It's the administrator "} else if($3>=1 && $3<=499 || $3==65534 ) {print $1,": It's the system user "} else {print $1,": It's the average user "}}'


awk -F: '{ if($3==0) {i++} else if($3>=1 && $3<=499 || $3==65534 ) {j++} else {k++}};END{print " The number of administrators is :"i "\n The number of system users is :"j"\n The number of ordinary users is :"k }'


# awk -F: '{if($3==0) {print $1,"is admin"} else if($3>=1 && $3<=499 || $3==65534) {print $1,"is sys users"} else {print $1,"is general user"} }' a.txt 

root is admin
bin is sys users
daemon is sys users
adm is sys users
lp is sys users
redhat is general user
user01 is general user
named is sys users
u01 is general user
YUNWEI is general user

awk -F: '{ if($3==0) {print $1": Administrators "} else if($3>=1 && $3<500 || $3==65534 ) {print $1": It's the system user "} else {print $1": It's the average user "}}'   /etc/passwd


awk -F: '{if($3==0) {i++} else if($3>=1 && $3<500 || $3==65534){j++} else {k++}};END{print " The number of administrators is :" i RS " The number of system users is :"j RS " The number of ordinary users is :"k }' /etc/passwd
 The number of administrators is :1
 The number of system users is :28
 The number of ordinary users is :27


# awk -F: '{ if($3==0) {print $1": It's the administrator "} else if($3>=500 && $3!=65534) {print $1": It's the average user "} else {print $1": It's the system user "}}' passwd 

awk -F: '{if($3==0){i++} else if($3>=500){k++} else{j++}} END{print i; print k; print j}' /etc/passwd

awk -F: '{if($3==0){i++} else if($3>999){k++} else{j++}} END{print " Number of Administrators : "i; print " It's common to count : "k; print " System users : "j}' /etc/passwd 

 If it is the normal user printing default shell, If the system user prints the user name 
# awk -F: '{if($3>=1 && $3<500 || $3 == 65534) {print $1} else if($3>=500 && $3<=60000 ) {print $NF} }' /etc/passwd

㈡ Loop statement

① for loop

 Print 1~5
for ((i=1;i<=5;i++));do echo $i;done

# awk 'BEGIN { for(i=1;i<=5;i++) {print i} }'
 Print 1~10 The odd number in 
# for ((i=1;i<=10;i+=2));do echo $i;done|awk '{sum+=$0};END{print sum}'
# awk 'BEGIN{ for(i=1;i<=10;i+=2) {print i} }'
# awk 'BEGIN{ for(i=1;i<=10;i+=2) print i }'

 Calculation 1-5 And 
# awk 'BEGIN{sum=0;for(i=1;i<=5;i++) sum+=i;print sum}'
# awk 'BEGIN{for(i=1;i<=5;i++) (sum+=i);{print sum}}'
# awk 'BEGIN{for(i=1;i<=5;i++) (sum+=i);print sum}'

② while loop

 Print 1-5
# i=1;while (($i<=5));do echo $i;let i++;done

# awk 'BEGIN { i=1;while(i<=5) {print i;i++} }'
 Print 1~10 The odd number in 
# awk 'BEGIN{i=1;while(i<=10) {print i;i+=2} }'
 Calculation 1-5 And 
# awk 'BEGIN{i=1;sum=0;while(i<=5) {sum+=i;i++}; print sum }'
# awk 'BEGIN {i=1;while(i<=5) {(sum+=i) i++};print sum }'

③ Nested loop

 Nested loop ：
#!/bin/bash
for ((y=1;y<=5;y++))
do
	for ((x=1;x<=$y;x++))
	do
		echo -n $x	
	done
echo
done

awk 'BEGIN{ for(y=1;y<=5;y++) {for(x=1;x<=y;x++) {printf x} ;print } }'


# awk 'BEGIN { for(y=1;y<=5;y++) { for(x=1;x<=y;x++) {printf x};print} }'
1
12
123
1234
12345

# awk 'BEGIN{ y=1;while(y<=5) { for(x=1;x<=y;x++) {printf x};y++;print}}'
1
12
123
1234
12345

 Try printing... In three ways 99 Pithy formula table ：
#awk 'BEGIN{for(y=1;y<=9;y++) { for(x=1;x<=y;x++) {printf x"*"y"="x*y"\t"};print} }'

#awk 'BEGIN{for(y=1;y<=9;y++) { for(x=1;x<=y;x++) printf x"*"y"="x*y"\t";print} }'
#awk 'BEGIN{i=1;while(i<=9){for(j=1;j<=i;j++) {printf j"*"i"="j*i"\t"};print;i++ }}'

#awk 'BEGIN{for(i=1;i<=9;i++){j=1;while(j<=i) {printf j"*"i"="i*j"\t";j++};print}}'

 Cycle control ：
break		 Break the loop when the conditions are met 
continue	 Skip the loop when conditions are met 
# awk 'BEGIN{for(i=1;i<=5;i++) {if(i==3) break;print i} }'
1
2
# awk 'BEGIN{for(i=1;i<=5;i++){if(i==3) continue;print i}}'
1
2
4
5

6. awk Arithmetic operation

+ - * / %( model ) ^( power 2^3)
 You can perform calculations in patterns ,awk Will perform arithmetic operations as floating-point numbers 
# awk 'BEGIN{print 1+1}'
# awk 'BEGIN{print 1**1}'
# awk 'BEGIN{print 2**3}'
# awk 'BEGIN{print 2/3}'

6、 ... and 、awk Statistical cases

1、 All kinds of shell

# awk -F: '{ shells[$NF]++ };END{for (i in shells) {print i,shells[i]} }' /etc/passwd

books[linux]++
books[linux]=1
shells[/bin/bash]++
shells[/sbin/nologin]++

/bin/bash 5
/sbin/nologin 6

shells[/bin/bash]++			a
shells[/sbin/nologin]++		b
shells[/sbin/shutdown]++	c

books[linux]++
books[php]++

2、 Statistics website visit status

# ss -antp|grep 80|awk '{states[$1]++};END{for(i in states){print i,states[i]}}'
TIME_WAIT 578
ESTABLISHED 1
LISTEN 1

# ss -an |grep :80 |awk '{states[$2]++};END{for(i in states){print i,states[i]}}'
LISTEN 1
ESTAB 5
TIME-WAIT 25

# ss -an |grep :80 |awk '{states[$2]++};END{for(i in states){print i,states[i]}}' |sort -k2 -rn
TIME-WAIT 18
ESTAB 8
LISTEN 1

3、 Count every... Visit the site IP The number of

# netstat -ant |grep :80 |awk -F: '{ip_count[$8]++};END{for(i in ip_count){print i,ip_count[i]} }' |sort


# ss -an |grep :80 |awk -F":" '!/LISTEN/{ip_count[$(NF-1)]++};END{for(i in ip_count){print i,ip_count[i]}}' |sort -k2 -rn |head

4、 Statistics website log PV The amount

 Statistics Apache/Nginx One day in the Journal PV The amount  　< Statistics log >
# grep '27/Jul/2017' mysqladmin.cc-access_log |wc -l
14519

 Statistics Apache/Nginx One day in the journal is different IP Of visits 　< Statistics log >
# grep '27/Jul/2017' mysqladmin.cc-access_log |awk '{ips[$1]++};END{for(i in ips){print i,ips[i]} }' |sort -k2 -rn |head

# grep '07/Aug/2017' access.log |awk '{ips[$1]++};END{for(i in ips){print i,ips[i]} }' |awk '$2>100' |sort -k2 -rn

A term is used to explain ：

Website views （PV）
Noun ：PV=PageView ( Website views )
explain ： Refers to the number of page views , To measure the number of pages visited by website users . If you open the same page several times, the total number of views will be . Every time a user opens a page, he records 1 Time PV.

Noun ：VV = Visit View（ Number of visits ）
explain ： All pages from visitors to your site to the final closure of the site leave , Counted as 1 visit . If visitors continue 30 There is no new page or refresh page in minutes , Or the visitor closes the browser , Is calculated as the end of this visit .

Independent visitor （UV）
Noun ：UV= Unique Visitor（ Number of unique visitors ）
explain ：1 The same visitors visit your website more than once in a day 1 individual UV.

Independent IP（IP）
Noun ：IP= Independent IP Count
explain ： finger 1 Different use in days IP Number of sites visited by users of the address . same IP No matter how many pages you visit , Independent IP The numbers are all 1

# 7、 ... and 、 Homework after class

Homework 1：
1、 Write a script that automatically detects disk usage , When the disk space reaches 90% When above , You need to send an email to relevant personnel
2、 Write a script to monitor system memory and swap partition usage

Homework 2：
Enter a IP Address , Use the script to determine its legitimacy ：
Must conform to ip Address specification , The first 1、4 Bits cannot be in the form of 0 start , Not greater than 255 Not less than 0

# 8、 ... and 、 Enterprise actual combat cases

1. Mission / background

web There are a total of... In the server cluster 9 Taiwan machine , The above deployment is Apache service . As the business continues to grow , A large number of access logs are generated on each machine every day , Now we need to put each web On the server apache Access log Keep recent 3 God Of ,3 Days ago dump To a special log server , Follow up analysis has been done . How to keep only on each server 3 Within days of the log ？

2. specific requirement

Each station web The log of the server corresponds to the directory of the log server . Such as ：web1——>web1.log（ On the log server ）
Each station web Keep the latest... On the server 3 Days of access logs ,3 The log days ago, every morning 5:03 Sub dump to the log server
If script dump fails , The operation and maintenance personnel need to select manual log cleaning through the menu of the springboard machine