State Management and Backends

Authors

K19G

grok-bot

Updated

October 31, 2026

State Management and Backends

State is the memory of object ids. A backend is where that memory lives and how it is locked. This chapter moves the pet example’s local file to a design you can hand to a second operator.

Learning goals

Say what a state file stores, including values marked sensitive
Configure a remote backend with a lock
Read another project’s outputs without reading its resources
Move a state between backends, and recover from a lock that outlived its apply

Understanding State: What It Stores and Why Local Is Not Enough

terraform.tfstate is JSON. For each resource address it stores the provider, the type, the id the provider returned, and the attributes from the last refresh. It also stores outputs and a serial number. Sensitive values are stored in full. The sensitive flag changes the CLI, not the file.

Local state fails as soon as two actors exist. Two engineers, or an engineer and a pipeline, apply two copies and each believes it owns the objects. The second apply creates duplicates or destroys the first world’s objects because they are absent from the second state. A laptop disk is also a poor backup.

Local state is still correct for a throwaway directory and for learning the commands. The moment the objects cost money or someone else must change them, the state moves.

Backend Types: Local, S3, Azure Blob, GCS, Consul, and More

A backend is declared inside the terraform block. It is read at init, and changing it means tofu init -migrate-state or a fresh init, not a normal plan.

Backend	Typical use	Locking
`local`	Learning, throwaway	None that helps a team
`s3`	AWS accounts	From OpenTofu 1.10, a lock file in the bucket (`use_lockfile = true`). Older setups use a DynamoDB table
`azurerm`	Azure	Blob lease
`gcs`	Google Cloud	Object generation / lock in the bucket
`consul`	Environments that already run Consul	Session lock
`http`, `pg`, and others	When the platform standard is something else	Only if that backend implements it. A backend without a lock is a hazard

Pick the backend that sits next to the account you are managing, so the failure domains match a story you can tell: the AWS state lives in an AWS account dedicated to state, not in the same bucket the configuration is free to delete. Access to the state backend is access to every secret the state holds and to the power to unlock or overwrite it. Chapter 12 narrows that IAM.

An example shape for S3, not a bucket this book creates for you:

terraform {
  backend "s3" {
    bucket       = "example-tofu-state"
    key          = "pets/terraform.tfstate"
    region       = "eu-west-1"
    encrypt      = true
    use_lockfile = true
  }
}

encrypt = true is server-side encryption at the bucket. It is not OpenTofu’s client-side encryption. You can use both. Client-side encryption is the encryption block below. Backend blocks cannot use variables or locals in older OpenTofu. Since 1.8 they can, which is useful and also a way to point production at the wrong bucket with a typo in a local. Prefer a partial backend config (tofu init -backend-config=backend.hcl) for the key and bucket, so the committed configuration does not contain an account-specific name you copy between environments by accident.

Remote State Configuration: Locking, Encryption, and Versioning

Locking makes a second apply wait or fail while the first holds the lock. Always turn it on. A backend that cannot lock is the wrong backend for production.

Versioning on the bucket, blob container, or GCS bucket keeps previous state objects. That is your backup when a bad apply writes a bad state. Lifecycle-expire old versions on a schedule you have tested a restore from. A versioned bucket nobody can read is not a backup.

Client-side encryption (OpenTofu 1.7 and later) encrypts the state and, if you ask, the plan, before they leave the process. The backend then stores ciphertext. A passphrase of at least 16 characters, fed from TF_VAR_, is the small-team form:

variable "state_passphrase" {
  type      = string
  sensitive = true
}

terraform {
  encryption {
    key_provider "pbkdf2" "main" {
      passphrase = var.state_passphrase
    }

    method "aes_gcm" "main" {
      keys = key_provider.pbkdf2.main
    }

    state {
      method   = method.aes_gcm.main
      enforced = true
    }

    plan {
      method   = method.aes_gcm.main
      enforced = true
    }
  }
}

enforced = true refuses to write plaintext. Lose the passphrase and the state is gone. Store the passphrase in the same secret manager the pipeline uses, and store a recovery copy under the same controls as any other encryption key. For a team, prefer a KMS key provider (aws_kms, gcp_kms, Azure Key Vault, or OpenBao) so access is IAM, not a shared password. Rotate by writing a new method and a fallback block that can still decrypt the old one. Take an unencrypted backup, offline and access-controlled, before the first migration onto encryption.

Reading Remote State: Cross-Referencing Infrastructure Between Projects

terraform_remote_state is a data source that reads another state and returns that state’s outputs.

data "terraform_remote_state" "network" {
  backend = "s3"

  config = {
    bucket = "example-tofu-state"
    key    = "network/terraform.tfstate"
    region = "eu-west-1"
  }
}

output "vpc_id" {
  value = data.terraform_remote_state.network.outputs.vpc_id
}

Only outputs are part of the contract. Reaching for anything else couples you to the other project’s resource names. The reader needs permission to read that state, which means the reader can also see every secret in it. Prefer passing an id through a narrow output over sharing a whole state with a wide team. Where the dependency is permanent, a data source on the real object (data "aws_vpc") is often a smaller permission than read access to someone else’s state.

Remote state creates an apply order. The network project must have been applied, and its outputs must exist, before the consumer plans. Write that order down. A pipeline that applies consumers in parallel with their producers will fail, or worse, will succeed against stale outputs.

State Migration: Moving Between Backends Safely

Changing the backend block and running tofu init -migrate-state copies state from the old backend to the new one and offers to delete the old copy. Do this with a lock, with a fresh backup of the source state, and with nobody else applying.

The safe sequence:

Stop pipelines that apply this state
tofu state pull > backup.tfstate and store that file somewhere encrypted, not in git
Change the backend configuration in a review
tofu init -migrate-state
tofu plan and confirm the plan is empty
Re-enable pipelines pointed at the new backend
Only then delete the backup, after the retention window you promised yourself

tofu state mv moves an address inside one state, or between states with pull and push. Renaming a resource without moved or state mv destroys and recreates. A moved block in the configuration is the reviewable form:

moved {
  from = random_pet.service
  to   = random_pet.api
}

Leave the block in place through one release so every state has applied it, then remove it.

State Troubleshooting: Corruption, Drift, and Recovery Procedures

A stale lock. The apply died and the lock remains. Read the lock. If the id matches a dead run and no apply is in progress, tofu force-unlock LOCK_ID removes it. Unlocking a live apply is how two writers corrupt state. Look at the pipeline before you unlock.

Drift. tofu plan shows a change you did not make. Either the console edit is wrong and you apply to put the object back, or the console edit is the new truth and you import it into configuration. Refresh-only (tofu apply -refresh-only) updates state to match the world without changing the world. Use it when you have decided the world is right and the state is stale. Do not use it as a weekly habit to hide drift you should have configured.

A corrupt or empty state. Stop. Restore the previous version from the backend’s object versioning, or from the backup you pulled. tofu state push replaces remote state with a file you have inspected. Pushing the wrong file is a worse incident than the one you were fixing. Inspect with tofu show -json backup.tfstate and confirm resource addresses before you push.

Resources in state and not in configuration. The next apply destroys them. That is either the change you wanted or an accidental deletion of a .tf file. Read a destroy plan from the bottom. terraform.tfstate.backup on a local backend is a single generation. It is not a history.

Never edit state JSON by hand unless you are following a documented recovery and you have a copy of the previous bytes. The CLI subcommands (state rm, state mv, import, force-unlock) exist so the serial number and the checksum stay consistent.

Summary

State stores ids and attributes, including sensitive ones, in full
A remote backend with a lock and versioning is the minimum for two operators
Server-side encryption and OpenTofu’s client-side encryption solve different problems. Production can use both
Remote state consumers depend on outputs only
Migrate with a backup, an empty plan, and pipelines paused
Unlock only a dead lock. Restore state from a version, do not hand-edit it