Setting up a CentOS 6 server to host a secure site

January 10, 2013 under Main

I recently had the task of setting up a CentOS 6 server which will be hosting a secure site. The site will be storing sensitive customer data including billing and credit card information, so security is critical.

The site will be Apache/PHP/MySQL, so a fairly standard LAMP stack. It will need to pass PCI compliance scans which involve a lot of port scanning, fingerprinting and attempts to break the web server by sending it unexpected requests.

While this list is in no way complete or authoritative, I thought I’d share a few of the steps I took to configure the server. The very first step was to install a clean CentOS 6 x86_64 and apply all the latest updates available by running ‘yum update’. I then installed mysql-server, php, php-mysql (and other php-* modules we need), httpd and mod_ssl. I enabled the servers I needed and disabled everything else, until I ended up with just:

# chkconfig --list|grep :on
crond              0:off    1:off    2:on    3:on    4:on    5:on    6:off
httpd              0:off    1:off    2:on    3:on    4:on    5:on    6:off
iptables           0:off    1:off    2:on    3:on    4:on    5:on    6:off
mysqld             0:off    1:off    2:on    3:on    4:on    5:on    6:off
network            0:off    1:off    2:on    3:on    4:on    5:on    6:off
rsyslog            0:off    1:off    2:on    3:on    4:on    5:on    6:off
sshd               0:off    1:off    2:on    3:on    4:on    5:on    6:off
udev-post          0:off    1:on    2:on    3:on    4:on    5:on    6:off

When it comes to security, the fewer servers running the better, as less code = fewer potential vulnerabilities and less servers = fewer potential exploitation points.

1. Firewall
CentOS ships with iptables which is a very capable and flexible firewall. There are lots of ways to configure iptables, but my personal favourite is to just open up the config file and write out the rules by hand. This way I can be completely confident that the firewall is doing what I intended it to do, nothing more and nothing less, and not what some configuration tool thinks will be a good configuration for me. The config file on CentOS is /etc/sysconfig/iptables and I ended up with the following rules:

-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m state -m tcp --dport 22 --state NEW -s -j ACCEPT
-A INPUT -p tcp -m state -m tcp --dport 443 --state NEW -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited

Enable the firewall using ‘chkconfig iptables on; service iptables start’

2. SSH
You may have noticed the only 2 ports I opened in the firewall are 443 for https (SSL web server) and 22 for SSH, but that the rule for port 22 has a source network restriction in place. This restriction limits access to the SSH server to connections from the local LAN. In this case that LAN is in a remote datacenter, and we can access the LAN using a VPN connection. This adds a layer of security to the already secure SSH server. If you don’t have a VPN available another good option is to restrict access to your static IP(s). If you don’t have a static IP, see Convenient And Secure Temporary Firewall Exceptions for another good solution.

The next step we took is to set up key based authentication by generating a keypair on our workstation using ‘ssh-keygen -t rsa’, then copying the public key part of the keypair to ~/.ssh/authorized_keys on the server. The .ssh directory, if it doesn’t already exist, must be chmod to 700 and authorized_keys to 600 – if you forget this step SSH will fail to use the keys as the permissions are insecure!

On our workstation we set up an alias for the host by opening (creating if it doesn’t exist) the file ~/.ssh/config and inserting:

Host secureserver
IdentityFile ~/.ssh/mynewprivatekey
Port 22
User mylogin

(real values obscured)

We can now log in to the server simply by typing ‘ssh secureserver’. Once key based authentication was working, I disabled password authentication entirely by editing /etc/ssh/sshd_config and setting PasswordAuthentication to ‘no’, and restarting sshd. SSH is now locked down to our local LAN IP range and to users in possession of the correct private key. This should be pretty secure, and the PCI scanner won’t even pick up on the existence of the SSH server as the firewall will block any connection attempts from the WAN.

In addition to using SSH for administering the server, we’ll also use the SFTP subsystem to upload files to our website. This means we don’t need to run a separate FTP server – one less server is only a good thing from a security perspective. We’ll also use the SSH server to tunnel connections to our MySQL server. MySQL Workbench (free from supports this type of connection of out the box, as do some other MySQL clients.

3. Apache and PHP
One of the things the PCI vulnerability scanners tend to do is try to fingerprint your Web server to see what software you have installed and what versions you are running. They then compare the version numbers against a database of known vulnerabilities. There are 2 problems with this approach: 1. it does not make any attempt to verify whether you are actually vulnerable and 2. the RHEL/CentOS philosophy of freezing software versions at release time then backporting security fixes means that the stock versions of things like Apache and PHP available from the CentOS yum repositories are never the latest ones available. This causes the vulnerability scanner to go wild saying you’re vulnerable to dozens of things that have actually been patched long ago. You can verify this if you are curious by looking at the changelogs which list the CVE numbers of security fixes which have been backported, eg. ‘rpm -q –changelog httpd’.

So let’s make their job a little harder by limiting what information our Web server discloses:
1. Edit /etc/php.ini and set “expose_php = Off”. This prevents PHP from adding a line to the HTTP response headers declaring its presence on the server.
2. Edit /etc/httpd/conf/httpd.conf and set:
– “ServerTokens Prod” – hide the version number from the HTTP response headers
– “ServerSignature Off” – hide the server name and version from server generated responses such as errors, directory listings, etc.

The HTTP headers now look like this:

HTTP/1.1 200 OK
Date: Thu, 10 Jan 2013 09:58:33 GMT
Server: Apache
Last-Modified: Tue, 08 Jan 2013 21:58:40 GMT
ETag: "85af-348-4d2ce0c66c400"
Accept-Ranges: bytes
Content-Length: 840
Connection: close
Content-Type: text/html; charset=UTF-8

Good – the version number is no longer reported and the existence of PHP is also omitted.

Also in /etc/httpd/conf/httpd.conf, I commented out all the modules which we don’t need, such as ldap, webdav, usertrack, userdir, status, info, vhost_alias, autoindex, speling, proxy, cache and version. This step will depend on what features of Apache you intend to use. I also removed all the configuration stuff we don’t need such as (in our case) the entire virtual hosting and proxying sections.

The SSL server configuration lives in /etc/httpd/conf.d/ssl.conf on a default install and this is where I set up the SSL virtual host. Of particular note in the SSL server configuration is to disable all weak encryption and outdated SSL implementations. This is important for any secure site and also something the PCI scan will pick up on if you leave the insecure defaults. I ended up with the following virtual host definition:

        DocumentRoot "/var/www/mysite/public_html"
        ServerName myhostname:443

        <Directory "/var/www/mysite/public_html">
                Options FollowSymLinks
                AllowOverride None

        # Use separate log files for the SSL virtual host; note that LogLevel
        # is not inherited from httpd.conf.
        ErrorLog logs/ssl_error_log
        TransferLog logs/ssl_access_log
        LogLevel warn

        #   SSL config
        SSLEngine on
        SSLProtocol -ALL +SSLv3 +TLSv1
        SSLCertificateFile ssl/mysslcert.crt
        SSLCACertificateFile ssl/
        SSLCertificateKeyFile ssl/myprivate.key

        CustomLog logs/portal_request_log "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"

4. MySQL
MySQL by default includes an anonymous user with access to the ‘test’ database, and a root user without a password! (granted, access is limited to the local server, but still not very secure). So the first thing I did after installing mysql-server was log in as root and secure the server:

- USE mysql
- DELETE FROM user WHERE user='';
- UPDATE user SET Password=PASSWORD('mysupersecretpassword');

Then proceed to set up any databases and appropriate access control lists as required.

I also copied /usr/share/doc/mysql-server-5.1.66/my-large.cnf to /etc/my.cnf and used this as a basis for my MySQL server configuration, I found this a reasonable basis for our requirements on this server.

This is not an exhaustive list of steps we took to secure this server and will probably not apply in full to you as every deployment is unique. I hope however that some of the information in this post is useful, and as always I welcome any comments, suggestions and feedback.

comments: Comments Off on Setting up a CentOS 6 server to host a secure site