Protecting tfstate in a remote backend

When Terraform handles resources, it writes the state of these resources in a tfstate file. This file is in JSON format and preserves the resources and their properties throughout the execution of Terraform.

By default, this file, called terraform.tfstate, is created locally when the first execution of the apply command is executed. It will then be used by Terraform each time the plan command is executed in order to compare its state (written in this tfstate) with that of the target infrastructure, and hence return the preview of what will be applied.

When using Terraform in an enterprise, this locally stored tfstate file poses many problems:

• Knowing that this file contains the status of the infrastructure, it should not be deleted. If deleted, Terraform may not behave as expected when it is executed.
• It must be accessible at the same time by all members of the team handling resources on the same infrastructure.
• This file can contain sensitive data, so it must be secure.
• When provisioning multiple environments, it is necessary to be able to use multiple tfstate files.

With all of these points, it is not possible to keep this tfstate file locally or even to archive it in an SCM.

To solve this problem, Terraform allows this tfstate file to be stored in a shared and secure storage called the remote backend.

In our case, we will use an azurerm remote backend to store our tfstates files with a storage account and a blob for the tfstate file.

We will, therefore, implement and use a remote backend in three steps:

1. The creation of the storage account
2. Terraform configuration for the remote backend
3. The execution of Terraform with the use of this remote backend

Let's look in detail at the execution of these steps:

1. To create an Azure Storage Account and a blob container, we can use either the Azure portal (https://docs.microsoft.com/en-gb/azure/storage/common/storage-quickstart-create-account?tabs=azure-portal) or an az cli script:

# 1-Create resource group
az group create --name MyRgRemoteBackend --location westeurope

# 2-Create storage account
az storage account create --resource-group MyRgRemoteBackend --name storageremotetf --sku Standard_LRS --encryption-services blob

# 3-Get storage account key
ACCOUNT_KEY=$(az storage account keys list --resource-group MyRgRemoteBackend --account-name storageremotetf --query [0].value -o tsv)

# 4-Create blob container
az storage container create --name tfbackends --account-name storageremotetf --account-key $ACCOUNT_KEY

This script creates a MyRgRemoteBackend resource group and a storage account, storageremotetf.

Then, the script retrieves the key account from the storage account and creates a blob container, tfbackends, in this storage account.

This script can be run in Azure Cloud Shell, and the advantage of using a script rather than using the Azure portal is that this script can be integrated into a CI/CD process.

1. Then, to configure Terraform to use the previously created remote backend, we must add the configuration section in the Terraform.tf file:

terraform {
    backend "azurerm" {
        storage_account_name  = "storageremotetfdemo"
        container_name        = "tfbackends"
        key                   = "myappli.tfstate" 
    }
}

The storage_account_name property contains the name of the storage account, the container_name property contains the container name, and the key property contains the name of the blob tfstate object.

However, there is still one more configuration information to be provided to Terraform so that it can connect and have permissions on the storage account. This information is the access key, which is a private authentication and authorization key on the storage account. To provide the storage key to Terraform, as with the Azure SP information, set an ARM_STORAGE_KEY environment variable with its value.

The following is a screenshot of the Azure storage access key:

1. Finally, once the Terraform configuration is completed, Terraform can be run with this new remote backend. It is during init that Terraform initializes the context of the tfstate file and, by default, the init command remains unchanged with terraform init.

However, if several tfstates in several environments are used, it is possible to create several remote backend configurations with the simplified code in the .tf file:

terraform {
    backend "azurerm" {}
}

Then, create several backend.tfvars files that only contain the properties of the backends.

These backend properties are the storage account name, the name of the blob container, and the blob name of the tfstate:

storage_account_name  = "storageremotetf"
container_name        = "tfbackends"
key                   = "myappli.tfstate"

In this case, when executing the init command, we can specify the backend.tfvars file to use with the following command:

terraform init -backend-config="backend.tfvars"

The -backend-config argument is the path to the backend configuration file.

Personally, I prefer this way of doing things as it allows me to decouple the code by externalizing the values of the backend properties and for better readability of the code. So, here is the execution of Terraform:

In this execution, we can see the export of the ARM_ACCESS_KEY environment variable, as well as the Terraform init command that determines the backend configuration with the -backend-config option.

With this remote backend, the tfstate file will no longer be stored locally, but on a storage account, which is a shared space. Therefore, it can be used at the same time by several users. This storage account offers, at the same time, security to protect the sensitive data of the tfstate but also the possibilities of backups and restoration of the tfstate files, which are an essential and critical element of Terraform as well.