Moving Production to Amazon Web Services

For over twenty years I’ve run my own servers at home. For slightly less time I also ran my lab from there, so these servers that ran my email, DNS, web services etc I referred to as “production”. Recently I decided that my home was no longer the place to keep my lab, as heating and noise were becoming increasingly more irritating. Shortly after I decided 

that production had to go as well. The question was “to where”?

To me, the Cloud was the obviously choice.

The estate

At first I ran everything on Windows NT4 on HP tin. As I was originally a Windows Administrator it seemed like the obvious choice. As time went on I moved over to RedHat Linux (starting with 6.0 in 1999) before finding NetBSD a few years later. In 2001 my friend Dan Higham introduced me to the Cobalt Qube2 microserver – the ideal platform for NetBSD. Sixteen years later they’re still in use.

The servers run a variety of software including Postfix, Dovecot, Apache, OpenLDAP, Bind, Dhcpd, Freeradius and VSftpd. A number of these services are needed to run the home LAN, but away from home, only a handful are needed – meaning I could potentially decommission one or two boxes and migrate the rest.

There are also two Qubes at my secondary site which serve as a backup for production. These also run Postfix, Bind etc, as well as being a replication partner for my primary Dovecot mail server. An IPSEC VPN is in place between both sites to ensure traffic is protected.

The Bookseller

As an architect designing solutions which involve cloud services, it’s important I’m familiar with as many offerings as possible. As I already spend a lot of time with Azure and vCloud Air, I decided it was time to work with the largest IaaS provider out there: Amazon Web Services.

The first step was to create a VPC with a subnet mirroring the existing one. This would simplify the VPN configuration as it means I would only have to rebuild one side. Once the sites were connected I began building my machines.

After a bit of searching I discovered that the Amazon Machine Image (Amazon’s equivalent of a template) I needed for NetBSD/i386 wasn’t available in London, so I needed to run my EC2 instances in Ireland. Whilst this wasn’t a major issue, I would have preferred to use London.

I deployed two t1.micro instances of the NetBSD/i386 AMI and imported my SSH public key to each one. Once built I began configuring each machine with the necessary software.

You’ve Got Mail

The first machine to configure was my mail server. After stopping Postfix in production all mail was being routed to the higher MX – my secondary site. This would continue to queue here until production was ready.

After installing and configuring Dovecot, I reversed the replication so all mailboxes at the secondary site were copied across the VPN to the new production server. I then installed and configured Postfix, and then repointed my DNS to the instance public IP. Finally I asked Amazon to update the reverse DNS on the IP to ensure my mail would not be bounced by other mail servers.

Web server

Installing and configuring Apache on my second machine was a trivial affair. After uploading my content, I created a number of rsync scripts to copy data to the secondary site and added them to my crontab.

Infrastructure

One of the boxes on the original production network handled infrastructure services such as DNS, DHCP and 802.1X authentication for my wireless access point. Whilst the latter two would no longer be needed, I used BIND to host the DNS zone for my lab network, as I found this easier to maintain than using my ISP. However I didn’t want to run this on an existing instance, and creating a new one just for that seemed like overkill.

The solution was to create a zone in Amazon’s hosted DNS system, Route 53. Once created, I exported all existing records from my existing BIND installation and imported them into Route 53. All in the process took less that two minutes.

Website, simplified

Whilst researching possible web server solutions I could host in AWS, I discovered that simple sites with static content could be hosted in S3 – Amazon’s storage service. Whilst this wouldn’t be possible for my WebDAV calendar, I did have another use for it.

In the lab I run Apache on CentOS 7 purely for hosting my internal two-tier certificate authority CRL. After creating a bucket in S3 and making it publicly accessible, I uploaded the CRL and redirected the DNS record to it. One less lab virtual machine to maintain…

Finishing off

Moving my production services to AWS has been a painless experience. The Cloud offers companies a number of benefits, including flexibility, agility and the ability to provision IT services with transparent costs. It has enabled me to consolidate my workloads and reduce the complexity of my existing estate.

More importantly it has removed four mildly noisy and hot boxes off my desk. Although I can’t lie… it’ll be hard to see them go.

2 thoughts on “Moving Production to Amazon Web Services

  1. Pingback: Newsletter: July 22, 2017 | Notes from MWhite

  2. Pingback: Getting the hell out of Amazon Web Services | virtualhobbit

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s