The Terraform Chronicles— A Personal View

Skipping Terragrunt: Why I Prefer My Terraform Scripts Extra Crispy (Not Too DRY, as They Say)

🏗️ Terraform repository Structure

There are many ways to organize your terraform repository , one of the simplest way is when you don’t have any environment needs

🚲 Mono Approach

terraform-exp1
├── backend.tf       # Type of backend to use to store Terraform state.(bucket GCS,S3,Consul)
├── main.tf          # Main file where all of teh resources are there like VM,network,...
├── iam.tf           # IAM definition for the project (rights management).
├── locals.tf        # Variables built during runtime.
├── project.tf       # Project creation/update using project-factory module.
├── variables.tf     # Variables definition (name, type, description).
└── versions.tf      # Version of terraform and providers to use.(Like google or consule or aws)

🛵 ENV Approach

terraform-exp2
├── env             # Folder that hold all the variables based on the env 
  ├── dev.tfvars    # Dev env  variables
  ├── shared.tfvars # SHared varable between all the envs
  ├── {ENV}.tfvars  # it could be any other env prod.tfvar or lab.tfvars
├── backend.tf       # Type of backend to use to store Terraform state.(bucket GCS,S3,Consul)
├── compute.tf       # File that will contains all the copute resources like VM
├── network.tf       # File contains network resources 
├── security.tf      # File contains security resources
├── iam.tf           # IAM definition for the project (rights management).
├── locals.tf        # Variables built during runtime.
├── project.tf       # Project creation/update using project-factory module.
├── variables.tf     # Variables definition (name, type, description).
└── versions.tf      # Version of terraform and providers to use.(Like google or consule or aws)

🚀 Layered Approach(recommended)

In a layered Terraform repository structure, resources are grouped by functionality or service type into separate directories. Each layer encapsulates related resources and configurations, allowing for better readability, manageability, and modularity.

terraform-exp3
├── gke
   ├── backend.tf
   ├── main.tf
   ├── provider.tf
   ├── tfvars
      ├── dev.tfvars
      ├── prod.tfvars
      ├── shared.tfvars
   ├── variables.tf
├── cloudrun
   ├── backend.tf
   ├── main.tf
   ├── iam.tf
   ├── tfvars
      ├── dev.tfvars
      ├── prod.tfvars
      ├── shared.tfvars
   ├── variables.tf
├── bootstarp 
   ├── backend.tf
   ├── project.tf
   ├── network.tf
   ├── tfvars
      ├── dev.tfvars
      ├── prod.tfvars
      ├── shared.tfvars
   ├── variables.tf

Benefits of Layered Structure

• Modularity: Each layer represents a specific aspect of infrastructure, promoting code reusability and separation of concerns.
• Readability: Resources are logically grouped, making it easier to understand and navigate the codebase.
• Isolation: Changes or operations (like apply or destroy) are scoped to specific layers, minimizing unintended impacts across the entire project.
• Environment-specific Configuration: Environment variables (dev.tfvars, prod.tfvars) are isolated per layer, simplifying environment-specific configurations.

🔡 Naming

🥋 Code Formatting

Indentation should be 2 spaces (soft tabs). No hard tabs.

Attribute assignments (=) should be aligned for clarity.

// bad
resource "aws_security_group" "main" {
    name = "${var.name}"
    description = "Security Group ${var.name}"
    vpc_id = "${var.vpc_id}"
    tags {
        Name = "${var.name}"
    }
}

// good
resource "aws_security_group" "main" {
  name        = "${var.name}"
  description = "Security Group ${var.name}"
  vpc_id      = "${var.vpc_id}"
  tags {
    Name = "${var.name}"
  }
}

🤬 Naming Conventions

File Names

Create a separate resource file for each type of resource. Similar resources should be defined in the same file and named accordingly.

network.tf
data.tf
iam.tf
compute.tf
providers.tf
variables.tf

Parameter, Meta-parameter and Variable Naming

Only use an underscore (_) when naming Terraform resources like TYPE/NAME parameters and variables.

resource "google_memcache_instance" "memcache_instance_project" {
...

Resource Naming

Choose the name of resources carefully

resource "google_memcache_instance" "private_google_memcache_instance" #bad
resource "google_memcache_instance" "private" #good
resource "google_memcache_instance" "private_memcache_instance" #bad

why because in your tfstate the resource name will be google_memcache_instance.private and it is easier to manipulated(delete, search,update…)

Only use a hyphen (-) when naming the component being created.

resource "google_memcache_instance" "memcache_instance_project" {
 count   = var.vm_number
 region  = var.region
 project = "${var.project_name}-${var.env}-demo"

Variables

Variables should be provided in a variables.tf file at the root of your project.

A description and type should be provided for each declared variable.

// bad
variable "vpc_id" {}

// good
variable "vpc_id" {
  description = "The VPC this security group will go in"
  type = string/number/bool....
}

Comments

Only comment what is necessary.

Single line comments: # or //.

Multi-line comments: /* followed by `*

🔢 Terraform Versioning

Always specify the exact version of terraform to use in version.tf

terraform {
  required_version = "1.14.0" #whatever teh version , you are using 
}

Always commit the .terraform.lock.hcl tou your github repo,it will keep all the dependency management and versioning sync across all platform running your IAC.

Always pin your provider version to not get unexpected upgrade and break changes

terraform {
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = ">= 4.0"
    }
    google-beta = {
      source  = "hashicorp/google-beta"
      version = ">= 4.0"
    }
  }
  required_version = ">= 1.0"
}

never pin a module like this

module "security_policy" {
  source = "GoogleCloudPlatform/cloud-armor/google"
  version = "~> 2.0" # may be upgraded unexpectedly

but like this

module "security_policy" {
  source = "GoogleCloudPlatform/cloud-armor/google"
  version = "2.0.1" # Good Good

To change between terraform versions you can use tfswitch

Terraform tooling

Install Terraform

• Linux
• Download Terraform archive here
• Unzip terraform archive
• Move terraform binary /usr/local/bin

mv ~/Downloads/terraform /usr/local/bin/

Install terraform Tools

• Linux
• Install tflint here for linting (unused vars,naming convention)
• Install tfsec here for secrity check
• Install terraform docs here to generate docs for terraform modules

🍠 Bonus

How to migrate state

1- Enter project path
2- Pull state (Optional)
terraform state pull > dev.tfsate
3- Init with new backend
terraform init -backend-config=”bucket=bucket_name” -backend-config=”prefix=migrate” — migrate-state
4- In case if we want to rename resources, Use:
terraform mv old new
OR
add move block
s moved { from = oldrsrc_name to = newsrc_name }
OR use this opensource tool
tfautomv