One of the joys (and yes, I mean that, I find this stuff fun) of running a shared hosting server with hundreds of accounts and thousands of domains on it is the constant monitoring and tweaking needed to keep it running smoothly.
We are already monitoring our hosting control panel, FTP server, SSH, IMAP and SMTP server logs for brute force attacks using the DirectAdmin control panel’s Brute Force Monitor system, tied in to the iptables firewall which blocks any IPs with too many invalid login attempts within a set period of time. This system works well and blocks dozens of IPs on a daily basis.
Another common attack vector which previously hasn’t been adequately covered is Web based brute force attacks, for example there are frequent attacks targeting WordPress sites on the server, trying to brute force the admin account password.
I’ve previously used Jon Zdziarski’s mod_evasive to detect DoS attacks against Apache, and remembered the DOSSystemCommand config parameter, which is able to execute an external script when an attack is detected. Using this I’ve tied mod_evasive into DirectAdmin’s brute force monitor system, so when a DoS attack is detected by mod_evasive the IP is blocked in the iptables firewall. The blocked IP can be viewed in the DirectAdmin Brute Force Monitor system where you can view details such as IP info and date/time when it was blocked, and remove it from the block list if desired.
The installation and integration was fairly straight forward on our CentOS based DirectAdmin server:
1. Download mod_evasive here, unpack, install with ‘apxs -i -a -c mod_evasive20.c’
2. Set up the config. I’ve started with the following config, and will keep an eye on the IPs it blocks over the next few hours/days and tweak as needed. See the README file included with mod_evasive for details on the various config parameters.
DOSHashTableSize 12289 DOSPageCount 10 DOSSiteCount 25 DOSPageInterval 1 DOSSiteInterval 3 DOSBlockingPeriod 300 DOSEmailNotify email@example.com DOSWhitelist 127.0.0.1 DOSSystemCommand "sudo /usr/local/bin/block_ip.sh %s" DOSLogDir "/var/log/mod_evasive"
And add ‘Include /etc/httpd/conf/extra/httpd-mod_evasive.conf’ to /etc/httpd/conf/httpd.conf
3. Create the script to tie into DirectAdmin’s brute force monitor file and to reload iptables:
#!/bin/sh BF=/root/blocked_ips.txt echo "Blocking $1 ..."; echo "$1=dateblocked=`date +%s`" >> $BF; echo "Restarting iptables ..."; /etc/init.d/iptables restart
‘chmod +x /usr/local/bin/block_ip.sh’ to make it executable.
4. Set up sudoers to allow the apache user that the module runs as to execute the block_ip.sh script. Run ‘visudo’ and add ‘apache my.host.name = NOPASSWD: /usr/local/bin/block_ip.sh’, right under the default ‘root ALL=(ALL) ALL’ entry is a good place. Replace my.host.name with the output from ‘uname -n’ on your system. On our system all virtual users can only run CGIs and PHP scripts under their own UID so this could not be exploited to create a DoS by a user on our server, if you are not using suexec/suPHP or similar, think again before allowing this command.
5. Restart Apache with ‘service httpd restart’ and watch it work! After only a few minutes it has already blocked a few IPs on our server. I grep’s the IPs in /var/log/httpd/domains/*log to try to determine if they were legit users or were actually attacks. The first IP blocked was repeatedly requesting “GET /life/?action=lostpassword”, which definitely looks suspicious to me, so looks like it’s working nicely.
Hopefully this technique will be useful to others running DirectAdmin’s brute force monitoring system. Any questions, comments or suggestions for improvement would be welcome in the comments.