ORIGINAL SOURCE: https://gist.github.com/rushipkar90/26934e53c0274c6acca3
MODIFIED VERSION BELOW:-
#If you are receiving complaints against your server for outgoing xmlrpc brute-force attack, then you can easily find-out culprit account by using below steps:
1) Run continous tcpdump on the server.
i) Prepare / cleanup /home/logs folder first.
mkdir /home/logs
killall tcpdump
rm -vf /home/logs/*
ii) Run below command in 'screen', it will dump all the monitored logs to /home/logs/tcpdump14022016 file.
tcpdump -A -s 0 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)' > /home/logs/tcpdump14022016
2) Set up below crons on the server. You can either set up these crons in the server crons i.e at crontab -e or you can set it up at /etc/cron.d/filename, for e.g. as in below case filename is "dumplogs".
OPTION 1: Put in /etc/cron.d/dumplogs:-
root@sv2[/etc/cron.d]# cat dumplogs
* * * * * root /home/logs/writelogs.sh >> /home/logs/writelogs.log 2>&1
* * * * * root sleep 30; /home/logs/writelogs.sh >> /home/logs/writelogs.log 2>&1
OPTION 2: Put in /var/spool/cron/root:-
* * * * * /home/logs/writelogs.sh >> /home/logs/writelogs.log 2>&1
* * * * * sleep 30; /home/logs/writelogs.sh >> /home/logs/writelogs.log 2>&1
Note that all cron output will be logged to /home/logs/writelogs.log . If face any problems below, check this file first for clues.
Before setting up the cron job, please make sure that file: /home/logs/writelogs.sh is exist with the below script inside:-
root@sv2[/etc/cron.d]# cat /home/logs/writelogs.sh
#!/bin/bash
netstat -antpe > /home/logs/`date +%F_%T`-ns.txt
ps auxfwww > /home/logs/`date +%F_%T`-ps.txt
top ccbn1 -c > /home/logs/`date +%F_%T`-top.txt
/etc/init.d/httpd fullstatus > /home/logs/`date +%F_%T`-apache.txt
3) Monitor the tcpdump log file launched earlier, look for outbound connections from server:-
root@sv2[/home/logs]# tail -f tcpdump14022016 | grep 'sv2\.nocser\.net\..* > .*\.http'
( NOTE: Replace sv2\.nocser\.net with the hostname of the server displayed by 'hostname' command )
Look for spurrious high frequency ( e.g. multiple lines per second ) outbound requests matching the filter above (If only one or two lines in a minute, ignore for now, since normally malware activity is high frequency). Once found, find out the IP address of the destination.
Example:-
10:07:19.228009 IP sv2.nocser.net.46725 > 69-195-89-225.unifiedlayer.com.http: Flags [.], seq 19776:22672, ack 1, win 115, options [nop,nop,TS val 2782745216 ecr 594492584], length 2896
10:07:19.228095 IP sv2.nocser.net.46725 > 69-195-89-225.unifiedlayer.com.http: Flags [.], seq 22672:25568, ack 1, win 115, options [nop,nop,TS val 2782745216 ecr 594492584], length 2896
10:07:19.228110 IP sv2.nocser.net.46725 > 69-195-89-225.unifiedlayer.com.http: Flags [.], seq 25568:28464, ack 1, win 115, options [nop,nop,TS val 2782745216 ecr 594492584], length 2896
10:07:19.228408 IP sv2.nocser.net.46725 > 69-195-89-225.unifiedlayer.com.http: Flags [.], seq 28464:31360, ack 1, win 115, options [nop,nop,TS val 2782745216 ecr 594492584], length 2896
.....
10:07:19.041943 IP sv2.nocser.net.46725 > 69-195-89-225.unifiedlayer.com.http: Flags [.], seq 4150649726:4150652622, ack 2058331274, win 115, options [nop,nop,T
S val 2782745029 ecr 594492565], length 2896
E....t@.@...*..FE.Y....P.e.~z......s.`.....
..Q.#o<.POST /xmlrpc.php HTTP/1.1
Host: stratxology.info
Cookie: income=1
Content-Type: application/x-www-form-urlencoded
Content-Length: 176171
Connection: Close
Find out the IP of the attacked host:-
root@sv2 [/home/logs]# ping 69.195.89.225
PING 69.195.89.225 (69.195.89.225) 56(84) bytes of data.
64 bytes from 69.195.89.225: icmp_seq=1 ttl=46 time=181 ms
64 bytes from 69.195.89.225: icmp_seq=2 ttl=46 time=181 ms
So in this example, destination IP being attacked is 69.195.89.225 .
4) Once we got the IP address of remote domain, then just do below. For e.g., if remote IP we got is: 69.195.89.225 then:
root@sv2 [/home/logs]# grep -lr 69.195.89.225 *.txt
2016-06-08_10:07:31-ns.txt
( Can be multiple files matched, if so, repeat the following on each file ):-
root@sv2 [/home/logs]# grep 69.195.89.225 "2016-06-08_10:07:31-ns.txt"
tcp 425 0 42.0.28.70:46725 69.195.89.225:80 CLOSE_WAIT 693 240086820 16385/php
From 16385/php : 16385 was the process id responsible for the attack above
5) Once we got the process ID, then just do below and which will show up the result with the exact domain name which was acting as the source of xmlrpc brute-force attack.
root@sv2 [/home/logs]# grep 16385 *-ps.txt
2016-06-08_10:07:32-ps.txt:693 16385 0.3 0.1 108804 11572 ? S 10:07 0:00 \_ /usr/bin/php /home/somedomain/public_html/wp-admin/scripts/script1.php
2016-06-08_10:08:02-ps.txt:693 16385 1.5 0.1 110212 13612 ? S 10:07 0:00 \_ /usr/bin/php /home/somedomain/public_html/wp-admin/scripts/script1.php
2016-06-08_10:08:31-ps.txt:693 16385 1.0 0.1 110724 14136 ? S 10:07 0:00 \_ /usr/bin/php /home/somedomain/public_html/wp-admin/scripts/script1.php
2016-06-08_10:09:01-ps.txt:693 16385 0.7 0.1 111236 14824 ? S 10:07 0:00 \_ /usr/bin/php /home/somedomain/public_html/wp-admin/scripts/script1.php
2016-06-08_10:09:31-ps.txt:693 16385 0.6 0.1 111236 15024 ? S 10:07 0:00 | \_ /usr/bin/php /home/somedomain/public_html/wp-admin/scripts/script1.php
Now, we have the exact script which performed the attack!
Here, we can see that culprit account is "somedomain". (actual name removed for privacy)
6) In order to resolve this issue:-
i) Quickly disable the website:-
cd /home/somedomain; mv -vi public_html public_html.DISABLED
ii) Kill all the somedomain processes found above to ensure no lingering attacks
kill 16385
ps -aef | grep somedomain
( kill any more matching process, repeat until no more process running for this user )
iii) Kindly inform website owner of this breach and abuse (Use Predefined reply --> [ATTENTION] Attacks Launched from Your Website), and ask them to refer the below URL's for information on mitigating xmlrpc attacks.
https://wordpress.org/plugins/stop-xmlrpc-attack/
https://wordpress.org/support/topic/recent-new-xmlrpcphp-brute-force-password-guessing-attack-details
In meantime, we can disable xmlrpc.php for the culprit account by inserting below code inside .htaccess file.
<Files xmlrpc.php>
order deny,allow
deny from all
</Files>
Then cleanup the website folder, and lock it:-
mkdir /home/somedomain/MALWARE
cd /home/somedomain/public_html.DISABLED
mv -vi wp-admin/scripts/script1.php ../MALWARE/
find . -mtime -1 (-10, -100, etc, until find other scripts, move them to ../MALWARE ).
cd /home/somedomain
chmod -R a-w public_html.DISABLED
Then re-enable the website if required:-
cd /home/somedomain; mv -vi public_html.DISABLED public_html
chattr+i public_html
REMINDERS:-
- Remember to disable back (e.g. add comment prefix '#') to the cron entries added earlier.
- If this action is prompted by abuse report, remember to reply back to the report, summarizing what has been done to ensure no more attacks from this server.
DONE.
