Azure Infrastructure Automation with Gitlab CI
Creating new resources in Azure is very easy thanks to Azure Portal, but what if you want to create an infrastructure with code. One of the options is to use ARM templates for infrastructure deployment. If you want to implement a proper CI/CD against your ARM templates, Gitlab CI can be a very strong candidate. Let's assume if you have a pipeline to create the infrastructure and delete it upon a successful deployment in order to save cost. Wouldn't it be perfect? If you say "yes", let's go deeper and see how I implemented the solution.
I am not going into the details of how you can create a Gitlab project as it is done by simply clicking a few buttons, but the most important part of the automation is to create variables which we will use in Gitlab shared runners to authenticate to Azure Cloud.
You can add variables by clicking "Settings" and then "CI/CD" in your project. The "Variables" blade should be visible and you can introduce such variables:
Do not forget to mark your variables as "Masked". Otherwise they will be visible in the Gitlab CI pipeline and they contain sensitive information about your Azure subscription
If you wish to mark your variables protected, please remember that they can be only accessed from protected branches
VARIABLE NAME | DESCRIPTION |
SP_ID | Azure Service Principal account to run the deployment |
SP_SECRET | Secret of the service principal account |
SUBSCRIPTION_ID | Azure subscription where resources will be created |
TENANT_ID | Tenant ID of your Azure subscription |
Now it is time to set up your pipeline to login to the Azure service principal account and deploy resources defined in your subscription. I am using Gitlab shared runners and "mcr.microsoft.com/azure-cli" container image to run the pipeline.
One of the important things is to use a dedicated resource group for the deployment tests so we can destroy whatever is inside that resource group to save costs.
variables:
DEFAULT_RG:
description: "Default resource group to deploy the resources for testing"
value: "test-rg"
DEFAULT_LOCATION:
description: "Default location of the testing resource group"
value: "northeurope"
default:
image: mcr.microsoft.com/azure-cli
before_script:
- az login --service-principal --username $SP_ID --password $SP_SECRET --tenant $TENANT_ID
- az account set --subscription $SUBSCRIPTION_ID
- set -euo pipefail
set -euo pipefail will prevent pipeline to run further steps if one of the variables is emptyThe remaining of the pipeline can change based on what you want to test but let me paste you the full content here so you can see what I am testing under that small scope. The respective Gitlab project can also be found here.
variables:
DEFAULT_RG:
description: "Default resource group to deploy the resources for testing"
value: "test-rg"
DEFAULT_LOCATION:
description: "Default location of the testing resource group"
value: "northeurope"
default:
image: mcr.microsoft.com/azure-cli
before_script:
- az login --service-principal --username $SP_ID --password $SP_SECRET --tenant $TENANT_ID
- az account set --subscription $SUBSCRIPTION_ID
- set -euo pipefail
stages:
- validate
- deploy
- destroy
validate virtual network:
stage: validate
script:
- az deployment group validate -g ${DEFAULT_RG} -f virtual-network/virtual-network.json -p virtual-network/virtual-network.parameters.json
validate virtual machine:
stage: validate
script:
- az deployment group validate -g ${DEFAULT_RG} -f virtual-machine/linux-vm.json -p virtual-machine/linux-vm.parameters.json -p adminPassword=$VM_INITIAL_PASSWORD
deploy virtual network:
stage: deploy
script:
- az deployment group create -g ${DEFAULT_RG} -f virtual-network/virtual-network.json -p virtual-network/virtual-network.parameters.json
deploy virtual machine:
stage: deploy
script:
- az deployment group create -g ${DEFAULT_RG} -f virtual-machine/linux-vm.json -p virtual-machine/linux-vm.parameters.json -p adminPassword=$VM_INITIAL_PASSWORD
needs:
- job: "deploy virtual network"
destroy test environment:
stage: destroy
script:
- az deployment group create -g ${DEFAULT_RG} -f resource-group/empty-resource-group.json --mode "Complete"
This pipeline will run validation of the resource to be created under the test resource group and will deploy them. After the successful deployment, it will use a very good trick to delete all resources in the resource group by doing a complete deployment with an empty Azure resource.
The above approach can let you test your ARM templates with Gitlab CI pipeline at a minimal cost. There are various ways of testing Azure deployment including Azure DevOps pipeline but I wanted to test it under Gitlab CI.