Infrastructure as Code | Azure Resource Manager Templates with Automated Testing | Part 1

In this blog series, we will be looking at how we can apply DevOps principles to the creation, maintenance, testing, and deployment of Azure ARM templates.

Azure Resource Manager is a core management layer, allowing for the deployment of grouped solutions to Azure, rather than individual services. At scale, customers can’t rely on manual, individual deployments through the Azure portal, or via imperative command line; it makes much more sense to define infrastructure as code, to ensure that deployments are repeatable and consistent.

Treating infrastructure as code means declaring upfront what your infrastructure deployment should look like; virtual networks, virtual machines, subnets, security rules, extensions, load balancers.. etc. The general rule of thumb is, if it can be configured in the Azure portal, you can usually declare it in an ARM template.

ARM templates are declarative templates, written in JSON, that allow you to deploy resources to Azure. You can also use templates to update resources and to delete resources, depending on the deployment mode you choose. I won’t blog any further about this, as ARM is very well documented and there are numerous other posts out there that do an excellent job of explaining the concept. What we will focus on here is:

How do we apply other DevOps principles to deploying our Azure resouces? Namely:

Source Control and Continuous Integration: How can we, as a team, ensure that we can collaborate on organisational ARM templates effectively?

Automated Testing: How can we ensure that the templates we want to deploy are valid, and how can we be sure what we just deployed is what we defined in our template?

Continuous Deployment: How can we speed up our pipeline, from authoring a new template, to making that template available to the organisation?

I worked with a customer last summer who were asking all of the above questions, and we went on a bit of a journey through each of these. I’ll take each in turn.

Source Control and Continuous Integration

So you have decided that infrastructure as code is right for you. I’ll assume you have landed on ARM as your tool of choice, but it is worth noting that there are options out there; Terraform and Ansible to name a couple. Everything we talk about in this section is relevant to those; you still need to maintain state in Terraform files and ensure your team is working off the same, and you still need to host your Ansible playbooks somewhere.

The customer I worked with was quite early in their journey with Azure DevOps (then VSTS). They are a small team, with predominantly infrastructure backgrounds, so a lot of these concepts were fairly new to them, however they were very comfortable working with ARM.

Problem:

ARM templates were being authored on individual workstations, then copied manually into the TFVC repo in Azure DevOps. This was causing problems with duplicated effort, errors creeping into templates, no change history, risk of loss of work in progress etc. This team are co-located, so finding out what someone else is working on is pretty easy, however back then there was no real ownership of tasks or overall view of team progress.

Solution:

I’ll break this down into a few areas:

Migration to Git for source control

I won’t get into the TFVC vs Git comparison here; ultimately, they could have used either. I happened to use a Git project to demo a CI/CD pipeline for ARM templates that went down quite well, and I personally haven’t used TFVC. They also weren’t very mature with TFVC, so they made the decision early to migrate. Azure DevOps makes this fairly easy with an import tool, but do read up on this before you take the plunge. The team didn’t have much history to carry over, so they weren’t too bothered.

The learning curve for Git is pretty steep, in my opinion. Sure, there are a few commands I think most of us have used and know well:

git clone
git commit
git push

But if you’re anything like me, my first few months looked more like:

I spent time really trying to learn how Git works under the hood, and what to do if something goes wrong (aside from the above advice!) so that I could give the customer adequate training. You can try the lab out here.

After the training, and with more practice using Git, the team saw a drastic decline in how many conflicts they had to deal with.

Another key consideration here is branching strategy. Initially, the team were pushing all local commits to master branch, which can be fine for small teams, but isn’t ideal. My guidance in choosing a branch strategy is generally.. start simple, and go from there. There is some great documentation out there which goes into detail on popular strategies, and I personally favour trunk based development with short lived branches. In this scenario in particular, with a team of 4 or 5 cloud infrastructure engineers, feature branches are only needed for as little as a few hours in a day, potentially a day at most.

Workflow looks something like this:

1. Developer performs a pull from master
2. Developer creates a local dev branch and codes a new feature
3. Developer performs a speculative pull from master to incorporate any recent remote changes (and addresses merge conflicts if necessary)
4. Developer pushes dev branch to the remote repo
5. Developer creates a pull request from dev into master for the team to review

Visual Studio Code for template development

Now that they had decided on a source control solution, the next decision was an authoring tool. Again, I developed the demos using VS Code, and in my opinion it is one of the best code editors out there, particularly when using Git and developing for Azure. Here’s why:

• Built in Git Support
• Side by side merge comparison (this was important for the team who didn’t want to navigate file changes using vim)
• Git Status Bar actions (the synchronize button pulls down the latest changes from the remote repo, then pushes your local commits)
• Extensions (so many extensions!)
• It’s FREE

There are some very good extensions that work well in this scenario. I would suggest Azure Resource Manager Tools, a JSON prettifier (a few to choose from) and PowerShell.

Based on the above, VS Code was chosen for developing the templates.

Azure DevOps Boards for tracking progress

Whilst not strictly falling under the umbrella of source control and integration, tracking progress on the ARM template development in the team was essential. Not only that, it allowed for new ideas to be captured and prioritised in the backlog.

That’s it for this post. In the next one, I will cover off:

• Automated ARM testing using the Pester framework in a CI/CD pipeline
• Continuous Deployment of ARM templates:
  • Pushing validated ARM templates to Azure Storage as part of a build pipeline
  • Creating a release pipeline to deploy Azure resources by calling ARM templates from Azure Storage using SAS tokens stored in Azure KeyVault

I’ll also provide details of a hands on lab which covers off key topics discussed in the blog series.