Record that the server has been invaded by viruses: the SSH password has been changed, the login fails, the malicious program runs full of CPU, the jar package fails to start automatically, and you ha

Problem phenomenon ：

ssh The login password ： The server password was changed without any reason , I also asked my colleagues that no one has touched the server , Then login to the server again by changing the password . jar Package start Failure ： start-up jar package , Soon afterwards, they were directly kill

[[email protected] discovery]# java -jar discovery-1.0.0.RELEASE.jar 

################################################
# #
# ## # # ## ### ### ## ### #
# # # # # # # # # # # # # #
# ### # # ### # # # ## # #
# # # ### ### # # # ### # # ### #
# #
# Obfuscation by Allatori Obfuscator v6.4 DEMO #
# #
# http://www.allatori.com #
# #
################################################

2021-02-18 09:08:01.045  INFO 19907 --- [           main] s.c.a.AnnotationConfigApplicationContex: Refreshing org.spring[email protected]1376c05c: artup date [Thu Feb 18 09:08:01 CST 2021]; root of context hierarchy
...
Killed

Copy

Keep popping up You have new mail in /var/spool/mail/root： Check the relevant contents of this and you will find that there has always been ERROR

|| ERROR || already running…

From [email protected]  Thu Feb 18 09:12:01 2021
Return-Path: <[email protected]>
X-Original-To: root
Delivered-To: [email protected]
Received: by k8s-n4.localdomain (Postfix, from userid 0)
	id 9ABDD43E1F; Thu, 18 Feb 2021 09:12:01 +0800 (CST)
From: "(Cron Daemon)" <[email protected]>
To: [email protected]
Subject: Cron <[email protected]> /root/.sshd/sshd
Content-Type: text/plain; charset=UTF-8
Auto-Submitted: auto-generated
Precedence: bulk
X-Cron-Env: <XDG_SESSION_ID=5360>
X-Cron-Env: <XDG_RUNTIME_DIR=/run/user/0>
X-Cron-Env: <LANG=en_US.UTF-8>
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <HOME=/root>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=root>
X-Cron-Env: <USER=root>
Message-Id: <[email protected]>
Date: Thu, 18 Feb 2021 09:12:01 +0800 (CST)

||  ERROR  || already running...

Copy

The first reaction is that the server resources are insufficient

Look at the running memory 、 There is enough disk space

Was invaded , Set up a regular task , hold CPU All the resources are occupied . In order to keep the system stable , So automatically kill process

top # View occupancy CPU , etc.

 Go straight ahead kill This sshd Your program will regenerate new 
kill -9 2172 # Name this sshd、pid by 2172 The program kill fall 

Copy

crontab -l # View scheduled tasks Sure enough, this is the fundamental problem Is written to the scheduled task

solve

Directly clear the scheduled task modification

crontab -e # Modify scheduled tasks # After entering the edit mode, delete it directly, and then wq Save and exit

modify root password

passwd root # modify root password It is recommended to modify the complex password You can search for a random password

Delete virus files

rm -rf /root/.sshd # Delete the suspicious directory directly

View process details ll /proc/7289

I also deleted /usr/bin/sshd Program And pass yum reinstall 、 start-up And then modified ssh Port number Default 22 Modify the port number

View scheduled tasks , Because it's not just through crontab -e This method is modified Check cron. Is there a new program in the directory

Checked /etc/cron All directories , I see several of them have added this pwnrig I deleted all the programs Direct deletion cannot delete Because of the addition of —-ia attribute

To delete Then I'm afraid there are cron Planned tasks have been modified , I'll go through yum Uninstall and then reinstall .

Check whether there are new users , Check out /home Add a new user The user was also recently added Use userdel user Delete the user and You don't succeed You can modify it directly /etc/passwd as well as /etc/shadow file See at the bottom of the file is about user The user's information is direct Delete and save exit

It is probably after the above steps , I tried this again kill Program , This program will be used by me kill fall , But when I reconnect to the server , That is to say linux When the server user logs in, the program will still generate , That is, the environment variables will be loaded when the user logs in , The attacker modified the environment variables of the server , This program will be generated as soon as I log in ！

Login is loading environment variable Of Check the documents cat /etc/profile cat /root/.bash_profile As expected, we found the clue

You can't modify and save directly , File properties have been modified

Delete the added items and save them

From the file of the environment variable just now, we can know that he put /bin/ The next program is copied Enter into /bin Under the table of contents , Because I knew that the approximate time was Feb Monthly screening

You can see that several suspicious files have been deleted

[[email protected] bin]# rm -rf bprofr 
rm: cannot remove ‘bprofr’: Operation not permitted
[[email protected] bin]# rm -rf crondr 
rm: cannot remove ‘crondr’: Operation not permitted
[[email protected] bin]# rm -rf initdr
rm: cannot remove ‘initdr’: Operation not permitted
[[email protected] bin]# rm -rf sysdr
rm: cannot remove ‘sysdr’: Operation not permitted
[[email protected] bin]# chattr -ia bprofr crondr initdr sysdr
[[email protected] bin]# rm -rf bprofr crondr initdr sysdr

Copy

Finally try again kill This process , This is called sshd My program was kill 了 , And will not regenerate , And this program will not be generated when you log in to the server again Before solving ：

After the solution ：

Publisher ： Full stack programmer stack length , Reprint please indicate the source ：https://javaforall.cn/101819.html Link to the original text ：https://javaforall.cn