Extracting recipient addresses from the Postfix maillog for a given authenticated SMTP sender

May 18, 2015 under Main

This is something I’ve had to do frequently enough, and end up having to reinvent the wheel each time as I never remember exactly how I did it last time.

The problem: given a particular authenticated SMTP username, extract a list of all recipients to which mail was sent from the current maillog file. You may want to first extract the section of the log(s) that you need to cover a specific time period.


grep sasl_username=sender@domain.tld maillog | cut -d " " -f 6 | perl -pe 's/://g' | xargs -I MSGID grep MSGID /var/log/maillog | grep to=\< | perl -pe 's/^.+to=\<(.+)\>,.+$/$1/g'

Replace sender@domain.tld with the username your sender authenticated as.

The tricky part is that the line Postfix logs when authenticating an SMTP connection only contains the message ID, not the recipient address(es). So we have to extract those message IDs and then re-parse the maillog to search for the line which contains the message ID and the string “to=”. Then extract the address from the¬†matching log entry.

You can pipe through uniq and sort if you want a sorted list of unique addresses, or leave as-is if you just want the list in the order logged by Postfix.

comments: Comments Off on Extracting recipient addresses from the Postfix maillog for a given authenticated SMTP sender

Why we no longer list a phone number on our Contact Us page

December 24, 2014 under Main

About a month ago I was making some updates to www.anu.net. We had put in a new support system and I was updating our email and Twitter contact details. I got to the section of our Contact Us page where our phone and fax numbers were listed, and thought to myself “does anyone still use these”?

Since setting up the company in 2004 we have never had an actual fixed line, but we have had incoming VoIP numbers. Initially we forwarded incoming calls to a receptionist service who would direct calls to the appropriate person, or take a message if nobody was available. As the years went by call volume steadily decreased (even though our customer base has grown steadily, as has the volume of support requests). Around 2011 (the exact date escapes me) we found the receptionist service, while good, was excessively expensive and wasn’t really needed, so we installed Kerio Operator (an IP PBX) and set it up to do much the same as the receptionist service had previously done through an IVR system. Customers didn’t complain.

Looking back over the years, the way we interact with our customers has slowly been changing. We are using phones less and less, with Skype, iMessage and Google Talk largely replacing their function. Email volume has remained fairly steady and is still a favourite method of communication.

Smart phones, and more recently the Phablet (iPhone 6 Plus anyone?) have been instrumental in this shift. Many people now seem to prefer picking up their phone to write a message than to make a call. The other factor at play here is the outdated pricing models of the traditional telcos. With customers spread out across the globe, traditional phone call charges can quickly rack up significant bills, making the free alternatives much more attractive.

We are still reachable by phone, and any customers who already have our number can continue to call it and will reach us. Don’t get me wrong, I don’t want to make it hard for our customers to get hold of us. Phones are however in my view old school tech and unless the traditional carriers make some radical changes to their business models (and do it quickly), they will soon become obsolete, replaced by cheaper, richer methods of communicating.

To date I haven’t had a single complaint from a customer that they could not find our phone number or figure out how to reach us when they needed to, which validates the decision to remove the phone number. How it works out longer term remains to be seen.

Personally I can imagine within 3-5 years being able to ditch my mobile phone completely and instead rely on a small tablet with wifi and 4G connectivity. I already exclusively use email, Facebook and FaceTime for keeping in touch with friends and family. Anyone stuck on an old school phone could reach me via an incoming VoIP a line. I already rely on VoIP for 90% of outbound calls and could probably switch over 100% already.

Here’s to the disruptive power of the Internet and new technologies. Happy Holidays!

comments: Comments Off on Why we no longer list a phone number on our Contact Us page tags: , , , , ,

Blocking http attacks with mod_evasive + DirectAdmin Brute Force Monitor + iptables

July 10, 2014 under Main

One of the joys (and yes, I mean that, I find this stuff fun) of running a shared hosting server with hundreds of accounts and thousands of domains on it is the constant monitoring and tweaking needed to keep it running smoothly.

We are already monitoring our hosting control panel, FTP server, SSH, IMAP and SMTP server logs for brute force attacks using the DirectAdmin control panel’s Brute Force Monitor system, tied in to the iptables firewall which blocks any IPs with too many invalid login attempts within a set period of time. This system works well and blocks dozens of IPs on a daily basis.

Another common attack vector which previously hasn’t been adequately covered is Web based brute force attacks, for example there are frequent attacks targeting WordPress sites on the server, trying to brute force the admin account password.

I’ve previously used Jon Zdziarski’s mod_evasive to detect DoS attacks against Apache, and remembered the DOSSystemCommand config parameter, which is able to execute an external script when an attack is detected. Using this I’ve tied mod_evasive into DirectAdmin’s brute force monitor system, so when a DoS attack is detected by mod_evasive the IP is blocked in the iptables firewall. The blocked IP can be viewed in the DirectAdmin Brute Force Monitor system where you can view details such as IP info and date/time when it was blocked, and remove it from the block list if desired.

The installation and integration was fairly straight forward on our CentOS based DirectAdmin server:

1. Download mod_evasive here, unpack, install with ‘apxs -i -a -c mod_evasive20.c’
2. Set up the config. I’ve started with the following config, and will keep an eye on the IPs it blocks over the next few hours/days and tweak as needed. See the README file included with mod_evasive for details on the various config parameters.
file: /etc/httpd/conf/extra/httpd-mod_evasive.conf

DOSHashTableSize	12289
DOSPageCount		10
DOSSiteCount		25
DOSPageInterval		1
DOSSiteInterval		3
DOSBlockingPeriod	300
DOSEmailNotify		you@company.com
DOSSystemCommand	"sudo /usr/local/bin/block_ip.sh %s"
DOSLogDir		"/var/log/mod_evasive"

And add ‘Include /etc/httpd/conf/extra/httpd-mod_evasive.conf’ to /etc/httpd/conf/httpd.conf

3. Create the script to tie into DirectAdmin’s brute force monitor file and to reload iptables:

file: /usr/local/bin/block_ip.sh


echo "Blocking $1 ...";
echo "$1=dateblocked=`date +%s`" >> $BF;
echo "Restarting iptables ...";
/etc/init.d/iptables restart

‘chmod +x /usr/local/bin/block_ip.sh’ to make it executable.

4. Set up sudoers to allow the apache user that the module runs as to execute the block_ip.sh script. Run ‘visudo’ and add ‘apache my.host.name = NOPASSWD: /usr/local/bin/block_ip.sh’, right under the default ‘root ALL=(ALL) ALL’ entry is a good place. Replace my.host.name with the output from ‘uname -n’ on your system. On our system all virtual users can only run CGIs and PHP scripts under their own UID so this could not be exploited to create a DoS by a user on our server, if you are not using suexec/suPHP or similar, think again before allowing this command.

5. Restart Apache with ‘service httpd restart’ and watch it work! After only a few minutes it has already blocked a few IPs on our server. I grep’s the IPs in /var/log/httpd/domains/*log to try to determine if they were legit users or were actually attacks. The first IP blocked was repeatedly requesting “GET /life/?action=lostpassword”, which definitely looks suspicious to me, so looks like it’s working nicely.

Hopefully this technique will be useful to others running DirectAdmin’s brute force monitoring system. Any questions, comments or suggestions for improvement would be welcome in the comments.

comments: Comments Off on Blocking http attacks with mod_evasive + DirectAdmin Brute Force Monitor + iptables

Running post-install/update scripts with yum-plugin-post-transaction-actions

June 6, 2013 under Main

Ever had the problem where periodic security/bug fix updates installed via yum end up breaking things because the RPM packages run post-install scripts that do annoying things like change ownership or permissions on files or directories?

For example the httpd package on CentOS always resets the owner of /var/www/html post-update. If you’ve changed this directory to be owned by another user on your system, chances are it’ll get reverted every time httpd is updated. Same goes for the tomcat log and temp directories when updating the tomcat6 package.

CentOS 6 includes a helpful new plugin for yum called yum-plugin-post-transaction-actions. It doesn’t get installed by default but a simple ‘yum -y install yum-plugin-post-transaction-actions’ will install it for you.

To use it all you have to do is create a new file in /etc/yum/post-actions/ like this, which resets the owner of the Tomcat log and temp dirs post-update: (file: /etc/yum/post-actions/tomcat6.action)

tomcat6*:update:/bin/chown -R myuername /var/cache/tomcat6/temp /var/log/tomcat6

For further information on the syntax and variables available to your script, see the docs at fedoraproject.org.

comments: Comments Off on Running post-install/update scripts with yum-plugin-post-transaction-actions