In part 4 of this series I configured Microsoft Exchange to work with nginx.

In this final part of the series I tidy up the loose ends so it can be put live.

Other articles in the series:

1. Installing and configuring keepalived
2. Installing nginx+
3. Configuring nginx+ for Microsoft Exchange
4. Configuring Microsoft Exchange
5. Tidying up

The first thing to configure is synchronise the nginx+ configs between both VMs.  To do this we will use rsync over SSH.

Create a new user on both VMs to run the rsync copy.  Insert your own password as desired:

useradd -s /bin/bash -p $(echo mysecretpassword | openssl passwd -1 -stdin) sa_copyconf

On HA1, login as the user and create the SSH keys:

mkdir .ssh
chmod 700 .ssh
cd .ssh
ssh-keygen -t rsa -N '' -b 2048

Accept the default file name for the private key. Add the public key to the list of authorized keys:

cat id_rsa.pub > authorized_keys2
chmod 644 authorized_keys2

Copy the public key over to HA2:

cat id_rsa.pub | ssh ha2.mail.mdb-lab.com "mkdir .ssh && chmod 700 .ssh && cat > .ssh/authorized_keys2"

On HA2, login is as sa_copyconf and set the permissions to /home/sa_copyconf/.ssh/authorized_keys2:

chmod 644 /home/sa_copyconf/.ssh/authorized_keys2

Also on HA2, copy across the id_rsa file from HA1 and place in .ssh:

sftp ha1.mail.mdb-lab.com:.ssh/id_rsa .ssh/id_rsa

On each VM, add permission to /etc/nginx/ for sa_copyconf:

setfacl -m u:sa_copyconf:rwx /etc/nginx/

Next, install rsync (if it isn’t already):

yum install rsync -y --nogpgcheck

Create the following script on each host (replace the hostname as needed – on HA1, it should reference HA2 and vice-versa):

cat <<EOF> /home/sa_copyconf/copyconf.sh
#!/bin/bash
rsync -avuz -e ssh ha2.mail.mdb-lab.com:/etc/nginx/nginx.conf /etc/nginx
EOF

Make the script executable:

chmod +x /home/sa_copyconf/copyconf.sh

Add a cron job to run the script every five minutes:

crontab -l | { cat; echo "*/5 * * * * /home/sa_copyconf/copyconf.sh"; } | crontab -

To test, delete the config on HA2:

rm -f /etc/nginx/nginx.conf

Wait ten minutes and the config should now reappear on HA2. To check this:

diff /etc/nginx/nginx.conf <(ssh ha1.mail.mdb-lab.com 'cat /etc/nginx/nginx.conf')

Next, restrict VRRP (the protocol keepalived uses) to the IPs of the two hosts. On HA1:

iptables -D INPUT -p 112 -j ACCEPT
iptables -I INPUT -p 112 -s 172.17.80.12 -j ACCEPT
service iptables save

On HA2:

iptables -D INPUT -p 112 -j ACCEPT
iptables -I INPUT -p 112 -s 172.17.80.11 -j ACCEPT
service iptables save

Test this by pausing the VM currently owning the cluster addresses and verifying they have transferred.

Finally, SELinux needs to be modified so nginx can run.  To demonstrate this, enable SELinux:

setenforce 1

Then restart the nginx service:

service nginx restart

You will get the following error:

nginx: [emerg] bind() to 172.17.80.13:135 failed (13: Permission denied)
nginx: configuration file /etc/nginx/nginx.conf test failed

This is because with SELinux enabled nginx is unable to bind to tcp/25, tcp/135 and tcp/139. To work around this:

grep nginx /var/log/audit/audit.log | audit2allow -m nginx > nginx.te
grep nginx /var/log/audit/audit.log | audit2allow -M nginx
semodule -i nginx.pp

To test, restart the service again:

service nginx restart

nginx should now start without issue.  On each VM run the following as root:

sudo sed -i "/SELINUX=permissive/c\SELINUX=enforcing" /etc/selinux/config

I would like to thank the technical guys at Nginx for help with the SELinux component.  More information regarding this can be found on their blog at http://nginx.com/blog/nginx-se-linux-changes-upgrading-rhel-6-6/.

Nick Shadrin at Nginx has also put together a comprehensive Exchange configuration guide on their site.  I highly recommend checking it out – http://nginx.com/blog/load-balancing-microsoft-exchange-nginx-plus-r6/.

Now that mainstream support for Microsoft Threat Management Gateway 2010 has ended (extended support is available till 14 April 2020), there is an opportunity to leverage technologies such as nginx+ to load-balance and publish Microsoft Exchange 2013 externally when the time comes.  If there is, I’ll be sure to document it!

In this article we have provided a method of syncing the configs, tightened security and re-enabled SELinux.

That completes the series on how to configure nginx+ to load-balance Microsoft Exchange.

In part 2 of this series I installed nginx+ on both HA1 and HA2.

In this part, I configure nginx+ to support Microsoft Exchange 2010/13.

Other articles in the series:

1. Installing and configuring keepalived
2. Installing nginx+
3. Configuring nginx+ for Microsoft Exchange
4. Configuring Microsoft Exchange
5. Tidying up

First, find your Exchange front-end SSL certificate and its serial number:

certutil -store my

Export the certificate (along with the private key) so it can be imported onto the nginx+ VMs:

certutil -exportpfx -p "password" -privatekey serialnumber mail.mdb-lab.com.pfx

Copy the PFX file to HA1 and HA2. Check the file came across okay:

openssl pkcs12 -info -in mail.mdb-lab.com.pfx

Import the certificate (you will be asked for the password you specified in the preceding step):

openssl pkcs12 -in mail.mdb-lab.com.pfx -nocerts -nodes -out mail.mdb-lab.com.key.enc
openssl pkcs12 -in mail.mdb-lab.com.pfx -clcerts -nokeys -out mail.mdb-lab.com.cer
openssl pkcs12 -in mail.mdb-lab.com.pfx -out cacerts.crt -nodes -nokeys -cacerts

The first command extracts the private key, the second the certificate, and the third the CA certificate(s).  Next make the private key ready for nginx+

openssl rsa -in mail.mdb-lab.com.key.enc -out mail.mdb-lab.com.key

Check the private key is correct:

openssl rsa -in mail.mdb-lab.com.key -check

Move the certificate, private key and CA certificates to /etc/nginx/ssl/

rm -f mail.mdb-lab.com.key.enc
rm -f mail.mdb-lab.com.pfx
mkdir -p /etc/nginx/ssl
mv -f mail.mdb-lab.com.* /etc/nginx/ssl/

Edit /etc/nginx/nginx.conf and make sure the following global settings are in place:

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log info;
pid /var/run/nginx.pid;
events {
     worker_connections 1024;
}

Add the following lines to the http block in /etc/nginx/nginx.conf, replacing values for your CAS servers where necessary:

http {
     log_format main '$remote_addr - $remote_user [$time_local] '
          '"$request'' $status $body_bytes_sent '
          '"$http_user_agent" "$upstream_addr"';
     #set the log
     access_log /var/log/nginx/access.log main;
     keepalive_timeout 3h;
     proxy_read_timeout 3h;
     tcp_nodelay on;

     upstream exchange {
          zone exchange-general 64k;
          server 172.17.80.21:443; # Replace with IP address of a your CAS
          server 172.17.80.22:443; # Replace with IP address of a your CAS
          sticky learn create=$remote_addr lookup=$remote_addr
                    zone=client_sessions:10m timeout=3h;
     }

     server {
          # redirect to HTTPS
          listen 80;
          location / {
               return 301 https://$host$request_uri;
               }
     }

     server {
          listen 443 ssl;
          client_max_body_size 2G;
          ssl_certificate /etc/nginx/ssl/mail.mdb-lab.com.cer;
          ssl_certificate_key /etc/nginx/ssl/mail.mdb-lab.com.key;
          ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
          status_zone exchange-combined;
          # redirect from main page to /owa/
          location = / {
               return 301 "/owa/";
          }
     }

     location = /favicon.ico {
          empty_gif;
          access_log off;
     }

     location / {
          proxy_pass https://exchange;
          proxy_buffering off;
          proxy_http_version 1.1;
          proxy_request_buffering off;
          proxy_set_header Connection ''Keep-Alive'';
     }
}

Add the stream block to /etc/nginx/nginx.conf also:

stream {

     upstream exchange-smtp {
          zone exchange-smtp 64k;
          server 172.17.80.31:25; # Replace with IP address of a your Hub Transport
          server 172.17.80.32:25; # Replace with IP address of a your Hub Transport
     }

     upstream exchange-smtp-ssl {
          zone exchange-smtp-ssl 64k;
          server 172.17.80.31:465; # Replace with IP address of a your Hub Transport
          server 172.17.80.32:465; # Replace with IP address of a your Hub Transport
     }

     upstream exchange-smtp-submission {
          zone exchange-smtp-submission 64k;
          server 172.17.80.31:587; # Replace with IP address of a your Hub Transport
          server 172.17.80.32:587; # Replace with IP address of a your Hub Transport
     }

     upstream exchange-imaps {
          zone exchange-imaps 64k;
          server 172.17.80.21:993; # Replace with IP address of a your CAS
          server 172.17.80.22:993; # Replace with IP address of a your CAS
     }

     server {
          listen 25; #SMTP
          status_zone exchange-smtp;
          proxy_pass exchange-smtp;
     }

     server {
          listen 465; #SMTP SSL
          status_zone exchange-smtp-ssl;
          proxy_pass exchange-smtp-ssl;
     }

     server {
          listen 587; #SMTP submission
          status_zone exchange-smtp-submission;
          proxy_pass exchange-smtp-submission;
     }
}

Test the configuration before putting it live:

nginx -t

If everything is correct, it will yield the following:

Modify iptables to allow traffic through the host firewall:

for i in {25,80,135,139,443,465,587,60000,60001}; do iptables -I INPUT -p tcp --dport $i -m state --state NEW,ESTABLISHED -j ACCEPT; done

Save the new iptables rulebase:

service iptables save

To get nginx+ running we need to disable SELinux temporarily:

setenforce 0

Edit /etc/selinux/config:

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=enforcing
# SELINUXTYPE= can take one of these two values:
#     targeted - Targeted processes are protected,
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

Change SELINUX=enforcing to SELINUX=permissive

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=permissive
# SELINUXTYPE= can take one of these two values:
#     targeted - Targeted processes are protected,
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

Start the service:

service nginx start

Make sure the config is the same on both HA1 and HA2. In part 5 I’ll configure rsync to ensure the configs are kept in sync.

That’s it for configuring nginx+.  In part 4 I’ll configure Exchange to support our nginx+ configuration.

In part 1 of this series I installed and configured keepalived in preparation for installing nginx+.

In this part, I install nginx+.

Other articles in the series:

1. Installing and configuring keepalived
2. Installing nginx+
3. Configuring nginx+ for Microsoft Exchange
4. Configuring Microsoft Exchange
5. Tidying up

On each VM, create the /etc/ssl/nginx directory:

mkdir -p /etc/ssl/nginx

Download CA.crt to /etc/ssl/nginx:

wget https://cs.nginx.com/static/files/CA.crt -P /etc/ssl/nginx

As part of your nginx+ trial or when you bought the software, you will have been provided a link to nginx-repo.key and nginx.repo.crt. Download these and place in /etc/ssl/nginx:

mv nginx-repo.key nginx.repo.crt /etc/ssl/nginx

Next, create a yum repository to install nginx+:

wget https://cs.nginx.com/static/files/nginx-plus-6.repo -P /etc/yum.repos.d

Then use yum to install:

yum install -y nginx-plus

If there is any issue with CA.crt (ie. it is missing or the permissions are not set correctly) then yum will not install the software.  The same goes for nginx.repo.crt.

Finally, enable the service on both VMs:

chkconfig nginx on

That’s all there is to the nginx+ installation.

In part 3, I configure nginx+ to load-balance the Microsoft Exchange environment.