Terraform

Motivation

For me there are several levels of getting infrastructure

Buy your own server
- Rent a server from somebody else who bought a server
  - Rent a virtual server with no 1:1 mapping to real servers
    - Pay for services you want to use, e.g. a database
      - Use infrastructure as code (IaC) to acquire the services you need.

Installation

https://developer.hashicorp.com/terraform/tutorials/azure-get-started/install-cli

wget -O- https://apt.releases.hashicorp.com/gpg | gpg --dearmor | sudo tee /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] \
https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt-get update
sudo apt-get install terraform
terraform -install-autocomplete

Providers

You need to configure Terraform for your cloud provider

Azure

https://developer.hashicorp.com/terraform/tutorials/azure-get-started/azure-build

Login with your Azure credentials (you need to have an Azure account already for this)

az login

You get something like this

...
{
"cloudName": "AzureCloud",
"homeTenantId": "aa-aa-aa-aa-aa",
"id": "bb-bb-bb-bb-bb",
"isDefault": false,
"managedByTenants": [],
"name": "ccc",
"state": "Enabled",
"tenantId": "dd-dd-dd-dd-dd",
"user": {
"name": "thorsten@example.com",
"type": "user"
}
},
...

Create a Service Principal using the value of the id field

az account set --subscription bb-bb-bb-bb-bb
az ad sp create-for-rbac --role="Contributor" --scopes="/subscriptions/bb-bb-bb-bb-bb"

You get something like this and need to remember it

{
"appId": "ee-ee-ee",
"displayName": "fffff",
"password": "ggggg",
"tenant": "hh-hh-hh-hh"
}

You could in theory but this into your configuration files but then you hard code the cloud environment and also the password in the in your version control. Better to put it into variables

# Configure Terraform to use Azure (Emirates) test-env
export ARM_CLIENT_ID="ee-ee-ee"
export ARM_CLIENT_SECRET="ggggg"
export ARM_SUBSCRIPTION_ID="bb-bb-bb-bb-bb"
export ARM_TENANT_ID="hh-hh-hh-hh"

How to use it

General Idea

You create an empty folder, go inside that folder, create

main.tf
terraform.tfvars

where main contains that infrastructure you want to have and then tfvars contains variables you use inside the main one. With the following commands in the directory everything will be created and delete again

# only needed once per folder
terraform init

# changes the format of all files to look nice
terraform fmt

# Are the files ok?
terraform validate

# What would be done?
terraform plan

# Do it
terraform apply

# Show what we have
terraform show
terraform state list

# delete what was created by us (be careful)
terraform delete

Variables

But careful to not hardcode to much in you configuration. For example if you want to create your infrastructure in a different region it sucks if you mention the location in lots of lines. Better define a variable for the location and use it everywhere. There are multiple ways to define the value then

In the main.cf, but at least only in one line

variable "my_location" {
description = "This is the Azure region that we use"
default = "westus2"
}

In the terraform.tfvars so you can have multiple versions of that file

my_location = "westus2"

In the command line e.g.

terraform plan -var "location=westus"

As environment variables

export TF_VAR_location="westus"

We can use a locals block to combine multiple variables into a new one.
We can use an output block to use the output of things we create as variable
We can use an data block to retrieve values of infrastructure that already exists
We can use an external block to run code locally to retrieve data

There is much more about variables here https://developer.hashicorp.com/terraform/language/values/variables

Having multiple variations of your infrastructure

It is very common to have the same infrastructure multiple times, e.g. production and test or for disaster recovery or for multiple clients. One way to solve this:

In the a folder have the usual files main.tf and variables.tf.
Per environment have a subfolder with terraform.tfvars
For each terraform command explicitly name the tfvars to be used

terraform plan -var-file="dev/terraform.tfvars"

You can use conditions in your configuration to differentiate between environments

name = var.environment == "PROD" ? "Production" : "No production"

Workspaces

If your environments are pretty much the same and don't need a lot of different variables you can also use workspaces to switch between them

# terraform workspace list
* default
# terraform workspace new DEV
# terraform workspace new PROD
# terraform workspace select DEV
# terraform workspace delete PROD

You can access the current workspace like this in your configuration

terraform.workspace

Create multiple instances of the same thing

With the count attribute you can create multiple instances of something.

variables.tf

variable "how_many_you_want" {
description = "This controls how many you want"
type = number
}

terraform.tfvars

how_many_you_want = 3

main.tf

...

# Create a new resource group
resource "azurerm_resource_group" "my_groups" {
count = var.how_many_you_want
name = "loop_demo_${count.index+1}"
location = "westus2"
}

A nice trick is to use either 0 or 1 to toggle things on and of, e.g.

count = var.environment == "PROD" ? 1 : 0

An alternative to count is the for_each which iterates through the content of a variable

Build in functions

There are some build in functions to manipulate variables in your configuration, e.g. make something uppercase https://developer.hashicorp.com/terraform/language/functions

Sync with what really exist

When you create your infrastructure with terraform it knows exactly what infrastructure exist.

When some smaller parts of your infrastructure was creates without terraform you can still define it in your configuration and then sync with

terraform import IDENTIFIER_IN_YOUR_CONFIG REMOTE_IDENTIFIER

When you have complex infrastructure already that was not defined in terraform you can create the configuration with such tools

Modules

You can create inside the folder that holds your Terraform files a subfolder modules/FOO and inside the FOO directory you can just start a new Terraform project. That project can then be called from the main folder like this

module "this_is_my_foo_module" {
source = "./modules/FOO"
some_variable = 42
}

You can even refer to remote sources like a git repository for the module source.

Modules can be used to group things that belong together (e.g. creating a VM and that network + storage it is using). You can find many ready to be used modules online so you don't have to reinvent the wheel all the time

Limit deletes

In order to apply changes sometimes infrastructure may be deleted and that may be unwanted

This will prevent the resource to be deleted but it also may stop you from changing some things with the resource

resouce ... ... {
lifecycle {
prevent_destroy = true
}
}

This will try to first create a new resource before deleting the old. But does not work well when the resource name is important and needs to be unique

resouce ... ... {
lifecycle {
create_before_destroy = true
}
}

Examples

Most examples are for Azure
For each example create an empty directory, go inside the directory and create the files
Run the standard terraform commands

Create Resource Group

variables.tf

variable "my_location" {
description = "This is the Azure region that we use"
default = "westus2"
type = string
}

variable "my_resource_group_name" {
description = "The resource group we use"
default = "thorsten_terraform_test_1"
type = string
}

terraform.tfvars

my_location = "westus2"
my_resource_group_name = "thorsten_terraform_test_1"

provider.tf

# Configure the Azure provider
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~> 3.60.0"
}
}

required_version = ">= 1.1.0"
}

provider "azurerm" {
features {}
}

main.tf

# Create a new resource group
resource "azurerm_resource_group" "my_rg_01" {
name = var.my_resource_group_name
location = var.my_location
}

Create DNS entries

Create a set of DNS entries.

Define variables

variables.tf

variable "my_location" {
description = "This is the Azure region that we use"
default = "westus2"
type = string
}

variable "my_resource_group_name" {
description = "The resource group we use"
default = "thorsten_terraform_test_2"
type = string
}

variable "dns_records_www" {
type = list(string)
default = ["1.2.3.4", "1.2.3.5"]
description = "List of IPv4 addresses."
}

variable "dns_zone_name" {
type = string
default = "thorsten.example.com"
description = "Name of the DNS zone"
}

variable "client_ids" {
description = "List of all clients"
type = any
}

Then a list of values for the variable

terraform.tfvars

client_ids = {
a01234 = {
name = "a01234"
ttl = 15
},
b56789 = {
name = "b56789"
ttl = 16
}
}

And now create the DNS entries

provider.tf

# Configure the Azure provider
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~> 3.60.0"
}
}

required_version = ">= 1.1.0"
}

provider "azurerm" {
features {}
}

main.tf

# Create a new resource group
resource "azurerm_resource_group" "my_rg_01" {
name = var.my_resource_group_name
location = var.my_location
}

# DNS zone for all our dns entries
resource "azurerm_dns_zone" "my_dns_zone" {
name = "thorsten.example.com"
resource_group_name = var.my_resource_group_name
}

# A record
resource "azurerm_dns_a_record" "record" {
name = "www"
resource_group_name = var.my_resource_group_name
zone_name = var.dns_zone_name
ttl = 15
records = var.dns_records_www
}

# CNAME
resource "azurerm_dns_cname_record" "cnamerecord" {
name = "www2"
zone_name = var.dns_zone_name
resource_group_name = var.my_resource_group_name
ttl = 20
record = "www"
depends_on = [azurerm_dns_a_record.record]
}

# API aliases
resource "azurerm_dns_cname_record" "api_record" {
for_each = var.client_ids
name = "api-${each.value["name"]}"
zone_name = var.dns_zone_name
resource_group_name = var.my_resource_group_name
ttl = each.value["ttl"]
record = "www.example.com"
depends_on = [azurerm_dns_a_record.record]
}

Create a local file with content

resource "local_file" "my_local_file" {
content = "Hello World"
filename = "demo.txt"
}

Run local commands

Maybe not a good idea, restricts OS where you can run your terraform

This gets the uptime of the local computer on every apply, the trigger triggers every time otherwise it would be executed only once

main.tf

resource "null_resource" "my_command" {
triggers = {
trigger = timestamp()
}
provisioner "local-exec" {
command = "/usr/bin/uptime"
interpreter = ["/bin/bash", "-c"]
}
}

Deploy something to a local Docker instead of Azure

As long as Docker works on your computer, you can instantly deploy services there

provider.tf

terraform {
required_providers {
docker = {
source = "terraform-providers/docker"
}
}
}

provider "docker" {}

main.tf

resource "docker_image" "nginx" {
name = "nginx:latest"
keep_locally = false
}

resource "docker_container" "nginx" {
image = docker_image.nginx.latest
name = "tutorial"
ports {
internal = 80
external = 8000
}
}

Define a Linux server with SSH access

providers.tf

# Configure the Microsoft Azure Provider.
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = ">= 2.26"
}
}
}

provider "azurerm" {
features {}
}

main.tf

# Create a resource group
resource "azurerm_resource_group" "rg" {
name = "${var.prefix}TFRG"
location = var.location
tags = var.tags
}

# Create virtual network
resource "azurerm_virtual_network" "vnet" {
name = "${var.prefix}TFVnet"
address_space = ["10.0.0.0/16"]
location = var.location
resource_group_name = azurerm_resource_group.rg.name
tags = var.tags
}

# Create subnet
resource "azurerm_subnet" "subnet" {
name = "${var.prefix}TFSubnet"
resource_group_name = azurerm_resource_group.rg.name
virtual_network_name = azurerm_virtual_network.vnet.name
address_prefixes = ["10.0.1.0/24"]
}

# Create public IP
resource "azurerm_public_ip" "publicip" {
name = "${var.prefix}TFPublicIP"
location = var.location
resource_group_name = azurerm_resource_group.rg.name
allocation_method = "Dynamic"
tags = var.tags
}

# Create Network Security Group and rule
resource "azurerm_network_security_group" "nsg" {
name = "${var.prefix}TFNSG"
location = var.location
resource_group_name = azurerm_resource_group.rg.name
tags = var.tags

security_rule {
name = "SSH"
priority = 1001
direction = "Inbound"
access = "Allow"
protocol = "Tcp"
source_port_range = "*"
destination_port_range = "22"
source_address_prefix = "*"
destination_address_prefix = "*"
}
}

# Create network interface
resource "azurerm_network_interface" "nic" {
name = "${var.prefix}NIC"
location = var.location
resource_group_name = azurerm_resource_group.rg.name
tags = var.tags

ip_configuration {
name = "${var.prefix}NICConfg"
subnet_id = azurerm_subnet.subnet.id
private_ip_address_allocation = "dynamic"
public_ip_address_id = azurerm_public_ip.publicip.id
}
}

# Create a Linux virtual machine
resource "azurerm_virtual_machine" "vm" {
name = "${var.prefix}TFVM"
location = var.location
resource_group_name = azurerm_resource_group.rg.name
network_interface_ids = [azurerm_network_interface.nic.id]
vm_size = "Standard_DS1_v2"
tags = var.tags

storage_os_disk {
name = "${var.prefix}OsDisk"
caching = "ReadWrite"
create_option = "FromImage"
managed_disk_type = "Premium_LRS"
}

storage_image_reference {
publisher = "Canonical"
offer = "UbuntuServer"
sku = lookup(var.sku, var.location)
version = "latest"
}

os_profile {
computer_name = "${var.prefix}TFVM"
admin_username = var.admin_username
admin_password = var.admin_password
}

os_profile_linux_config {
disable_password_authentication = false
}

}

data "azurerm_public_ip" "ip" {
name = azurerm_public_ip.publicip.name
resource_group_name = azurerm_virtual_machine.vm.resource_group_name
depends_on = ["azurerm_virtual_machine.vm"]
}

output "os_sku" {
value = lookup(var.sku, var.location)
}

Variables to be used in the config

variables.tf

variable "location" {}

variable "admin_username" {
type = string
description = "Administrator user name for virtual machine"
}

variable "admin_password" {
type = string
description = "Password must meet Azure complexity requirements"
}

variable "prefix" {
type = string
default = "my"
}

variable "tags" {
type = map

default = {
Environment = "Terraform GS"
Dept = "Engineering"
}
}

variable "sku" {
default = {
westus2 = "16.04-LTS"
eastus = "18.04-LTS"
}
}

Values for variables

terraform.tfvars

location = "westus2"
prefix = "tf"

terraform init
terraform plan
terraform apply
terraform show
terraform state list

More examples from Azure

Misc

Show dependencies as graph

terraform graph | dot -Tsvg > /tmp/graph.svg

Other