Infrastructure as code using AWS Cloudformation and Chef : Introduction

This post discusses how we automated infrastructure setup and app installation at Logic Square Technologies. The goal was to be able to bring up EC2 instances and auto configure them to install application code and start the application. We have broken this down in 3 parts.

1. Introduction: High level set up
2. CloudFormation: Setting up CloudFormation
3. Chef: Automating using Chef

Let us start with what goes into launching an EC2 instance and running an application on it?

1. Select a base image
2. Install all the software’s you need (nginx, node, etc.)
3. Update configurations for these software’s (server config, app config)
4. Download and install application code on it
5. Start the application

While doing this, developers usually resolve conflicts (between packages) pin to specific versions of installed software, discover, resolve and tune configurations.

Now what happens if this instance goes down? If you did not keep notes or missed noting down something, you do all that again. What happens if the project is paused for a while or if you work on it say 2 hours a day, you are still paying for 22 hours you are not using it.

What if you could press a button and in 3-5 minute you can have the instance up and running with all the things you installed on it and configured correctly, and when you are done you shut it down, or if you start getting traffic, you can launch a bunch of instances with a button click and they all come up configured correctly, ready to take traffic.

Enter AWS Cloudformation and Chef. Cloudformation () allows developers to write down all the resources of a “stack” in a json (or yaml) file. A typical web application stack would consist of a load balancer, auto scaling group, scale up and down policies, alarms, caching layer (memcache, redis, etc.) and permanent storage layer (RDS, DynamoDB, etc.) developers can write all the resources as code and check it into source control like git. If something does not work, you revert. It’s that easy.

Chef () allows you to easily configure the system, install software, put the right configurations in place, and start all the services needed. You can do all this in CloudFormation too, but there are some advantages of doing this in Chef: it’s easier to manage, it’s also not tied to AWS and moving vendors (google cloud or Azure) is pretty painless.

So, CloudFormation can bring up the instance, install chef, download your chef recipes and run chef. Chef can do all the involved installations, copying configuration files, etc.

Terraform () is another alternative to CloudFormation, one major advantage of using it is that it works with multiple vendors and services. You can have your configuration in one place instead of having it with each vendor or service. It also has a feature called “plan” which is more like a dry run, it tells you what resources will be added, changed or modified if you run “execute” on your configuration. For the scope of this series, we will limit to CloudFormation and Chef. If you are only using AWS services, CloudFormation should be fine.

At Logic Square Technologies, we have set up CloudFormation and Chef for a few projects, the upside of having it for our client was they could use the config file and create as many tests, staging or production environments they wanted. Once we delivered, we did not just give them the application code, we also gave them everything else they needed to run that code on a machine from scratch. They could have a working instance with click of a button.