Here is a reasonablyt quick procedure on how to early detection, prevention, and removal of spam mails from our servers' email queue. Quick action is needed to avoid servers being blacklisted by antispam blocklists in the Internet.
1. Keep monitoring interface open (nagios)
- Keep an eye on items marked WARNING (in yellow) or CRITICAL (in red). If found/alerted, click on it to see the details.
- A critical alert related to possible spam in email queue is as follows:-
SMTP Mail Queue [MAILSERVER] Mailqueue CRITICAL - 119 messages on queue
- In this example, mailserver has 119 messages in queue, which means possible spam activities.
2. SSH into affected server (in this case, mailserver), and browse through the e-mail queue to find the spammy e-mail:-
# exim -bp | more
- The queue contents will be displayed in the form of
*duration* *size* *queue ID* *sender address*
*recipient address*
- Scroll through the queue until you find an item with large amount of recipient addresses, and seemingly random internet addresses, and sometimes with sender address not hosted in our servers. e.g.:-
24m 2.1M 1WDQrg-0001Wf-1W <SampadaGodse@hotmail.com>
D jyunwa@dream.com
msherifs@comercio.mineco.es
D info@agifish.com.vn
D khollatz@alaskaseafood.org
D tgkallis@australfisheries.com.au
D info@peispa.com
D asian2@asianseafoods.co.th
...
- Copy the queue ID ( in this case 1WDQrg-0001Wf-1W )
3. Find the hijacked account details in the queue details:-
- Browse through the contents of this queue ID, using this command:-
# find /var/spool/exim/input -name "*1WDQrg-0001Wf-1W*" -exec cat {} \; | more
- Remember to substitute the above example queue ID with actual queue ID you found in step 2 above.
- You should see the queue details which reflect the details you see in #2, with some more details, e.g.:-
1WDQrg-0001Wf-1W-H
mailnull 47 12
<SampadaGodse@hotmail.com>
1392177120 0
-helo_name webmail.customerdomain.com
-host_address 127.0.0.1.39633
-host_name localhost
-host_auth dovecot_login
-interface_address 127.0.0.1.25
-received_protocol esmtpa
-body_linecount 28060
-max_received_linelength 76
-auth_id user@customerdomain.com <------------------------------------ keep an eye on the '-auth_id' line
YY menzai-auto1354@vesta.ocn.ne.jp YY
geral@ancipa.pt YY bdurin@almarsa-fisheries.com YY almarr@marsea.co.uk YY
acope@netcabo.pt ...
- Verify that the queue details is the same spammy message you see in step 2. If so, copy the "-auth_id" line:-
-auth_id user@customerdomain.com
- That is the account hijacked by the spammer for use in sending the spam e-mail.
4. Reset the hijacked account password to stop spammer from hijacking the account.
- Go to server's WHM interface --> List Accounts --> enter account domain name (in this case 'customerdomain.com'), and click on the account's CPanel icon.
- In account's CPanel, click on "E-Mail Accounts". Find the hijacked user account (in this case, user@customerdomain.com) and click "Change Password". Enter any arbitrary, reasonably complex password and note down the new password.
5. Restart exim mail service in the affected server, to immediately cut off the spammer's connection, to avoid further spamming using past authentication session:-
# /etc/init.d/exim restart
6. Delete all remaining spam using this hijacked account:- # grep -rHi 'auth_id user@customerdomain.com' /var/spool/exim/input | awk -F: '{ print $1 }' | awk -F/ '{ print $7 }'
- Make sure the email address after the auth_id part matches the hijacked email address
- Make sure you get list of queue IDs like so:-
1WDQrg-0001Wf-1W-H
..
- Only then proceed with deletion below:- # grep -rHi 'auth_id user@customerdomain.com' /var/spool/exim/input | awk -F: '{ print $1 }' | awk -F/ '{ print $7 }' | sed -E 's/-(H|D)$//g' | sort | uniq | xargs exim -Mrm
- The results should be as follows:-
Message 1WDQrg-0001Wf-1W has been removed ..
7. Repeat step #2-#6 until there are no more spammy messages in queue.
8. Sometimes, exim -bp yields 'backscatter' messages, i.e. server bounce in response to failed spam mail deliveries. For example:-
42m 5.8K 1WDQtF-000223-KY <> *** frozen ***
somestrangerperson@live.com
42m 7.1K 1WDQtH-00022I-Qu <> *** frozen ***
somestrangerperson@live.com
42m 5.1K 1WDQtK-00022o-1p <> *** frozen ***
somestrangerperson@live.com
..
- Confirm that this is backscatter by analyzing one sample of the file:-
# find /var/spool/exim/input -name "*1WDQtF-000223-KY*" -exec cat {} \; | more
- Check if the details of intended message in bounce message is spammy (thus confirming it is backscatter), if so, you can remove all such messages by:-
# exiqgrep -i -r somestrangerperson@live.com | xargs exim -Mrm
9. Via WHMCS, notify the customer / owner of the account/server of this incident, along with the new password set to the hijacked account. (Use suitable template in Predefined Replies).
