{ backend "s3" { bucket = "terraform-state-prod-us-east-1" key = "networking/vpc/terraform.tfstate" region = "us-east-1" encrypt = true dynamodb_table = "terraform-locks"

# Versioning is managed via bucket configuration, not backend block
# but the backend block ensures the state file is written correctly.

} }

### Step 2: State Partitioning Strategy

Avoid a single state file per environment. Partition state based on **independent deployability** and **ownership**.

**Recommended Topology:**
1.  **Per-Component:** Separate state files for VPC, IAM, EKS, RDS.
2.  **Per-Environment:** Distinct buckets or prefixes for `dev`, `staging`, `prod`.
3.  **Avoid Workspaces for Isolation:** Terraform workspaces share the same state structure and are designed for parametric variation (e.g., `dev` vs `prod` with different variable values), not for isolating components. Use directory structures for isolation.

**Directory Structure Example:**
```text
infrastructure/
├── networking/
│   ├── vpc/
│   │   ├── main.tf
│   │   └── backend.tf  # Points to networking/vpc/terraform.tfstate
├── compute/
│   ├── eks/
│   │   ├── main.tf
│   │   └── backend.tf  # Points to compute/eks/terraform.tfstate
└── data/
    ├── rds/
    │   ├── main.tf
    │   └── backend.tf

Step 3: State Migration and Import

Moving existing resources into managed state requires precise operations.

Migrating to Remote Backend: When switching from local to remote, Terraform prompts to copy state. Ensure no other processes are running.

terraform init -migrate-state

Importing Existing Resources: If resources exist outside Terraform, use terraform import to bring them under management. Avoid manual state manipulation.

# Import an existing S3 bucket
terraform import aws_s3_bucket.data_bucket my-existing-bucket-name

Refactoring State: When renaming resources or moving modules, use terraform state mv to update the state file without destroying and recreating resources.

terraform state mv aws_instance.old aws_instance.new

Step 4: Security and Compliance Controls

State files must be treated as secrets.

1. IAM Policies: Restrict access to the state bucket and DynamoDB table to CI/CD roles only. Developers should have read-only access for debugging.
2. Versioning: Enable S3 bucket versioning to retain history of state changes.
3. Lifecycle Rules: Implement non-current version expiration to manage storage costs while retaining recovery windows.

# s3_bucket_policy.tf
resource "aws_s3_bucket_policy" "state_policy" {
  bucket = aws_s3_bucket.state.id
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Sid       = "DenyUnencryptedTransport"
        Effect    = "Deny"
        Principal = "*"
        Action    = "s3:*"
        Resource  = "${aws_s3_bucket.state.arn}/*"
        Condition = {
          Bool = { "aws:SecureTransport" = "false" }
        }
      },
      {
        Sid       = "DenyUnencryptedState"
        Effect    = "Deny"
        Principal = "*"
        Action    = "s3:PutObject"
        Resource  = "${aws_s3_bucket.state.arn}/*"
        Condition = {
          StringNotEquals = { "s3:x-amz-server-side-encryption" = "aws:kms" }
        }
      }
    ]
  })
}

Step 5: Automated Drift Detection

State drift occurs when infrastructure changes outside Terraform. Implement scheduled drift detection.

Implementation:

• Run terraform plan in read-only mode on a schedule.
• Alert on non-empty diffs.
• Integrate with Slack/PagerDuty for critical resources.

# .github/workflows/drift-detection.yml
name: Drift Detection
on:
  schedule:
    - cron: '0 2 * * *' # Daily at 2 AM
jobs:
  detect-drift:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: hashicorp/setup-terraform@v2
      - run: terraform init
      - run: terraform plan -detailed-exitcode
        continue-on-error: true
        id: plan
      - if: steps.plan.exitcode == 2
        run: echo "::warning::Drift detected in state"

Pitfall Guide

1. Using Workspaces for Environment Isolation

Mistake: Using terraform workspace new prod to separate dev and prod. Reality: Workspaces share the same configuration and state structure. They are intended for deploying the same architecture with different variables, not for isolating distinct environments with different resource counts or security requirements. This leads to configuration complexity and accidental cross-environment operations. Best Practice: Use separate directories or repositories with distinct backend configurations for environments.

2. Storing Secrets in State Without Encryption

Mistake: Relying on default storage without KMS or bucket encryption. Reality: State files store plaintext values of sensitive attributes. If the bucket is exposed, credentials are compromised. Best Practice: Enforce KMS encryption and strict bucket policies. Use sensitive = true in outputs to prevent secrets from leaking in logs, though they remain in state.

3. Ignoring State File Size and Performance

Mistake: Allowing state files to grow indefinitely with thousands of resources. Reality: Large states slow down plan/apply and increase the risk of timeouts or corruption during writes. Best Practice: Partition state aggressively. Keep state files under 10MB where possible. Use terraform state list to audit resource density.

4. Manual State Manipulation

Mistake: Using terraform state rm or editing the JSON file directly to fix errors. Reality: Manual edits bypass Terraform's validation and locking,极易导致 state corruption and orphaned resources. Best Practice: Use terraform import to add resources and terraform state mv to rename. If state is corrupted, restore from backup rather than editing.

5. Missing Locking Mechanism

Mistake: Using a backend that doesn't support locking or disabling it for speed. Reality: Concurrent applies without locking can result in partial updates, resource duplication, or state corruption. Best Practice: Always configure a locking mechanism (e.g., DynamoDB for S3 backend). Never disable locking in production.

6. No Backup or Recovery Strategy

Mistake: Assuming versioning is sufficient and never testing recovery. Reality: Versioning helps, but without a tested restoration procedure, teams may fail to recover quickly during incidents. Best Practice: Enable versioning and implement a recovery runbook. Periodically test restoring state from a previous version.

7. Cross-State Dependencies Without Data Sources

Mistake: Hardcoding IDs or outputs across separate state files. Reality: This creates implicit coupling and breaks if resource IDs change. Best Practice: Use terraform_remote_state data sources to read outputs from dependent states. This formalizes dependencies and allows Terraform to track them.

data "terraform_remote_state" "network" {
  backend = "s3"
  config = {
    bucket = "terraform-state-prod-us-east-1"
    key    = "networking/vpc/terraform.tfstate"
    region = "us-east-1"
  }
}

resource "aws_instance" "web" {
  subnet_id = data.terraform_remote_state.network.outputs.private_subnet_id
  # ...
}

Production Bundle

Action Checklist

• Configure Remote Backend: Migrate all state to a remote backend with locking enabled (e.g., S3 + DynamoDB).
• Enable Versioning: Turn on S3 bucket versioning to retain state history and enable rollback.
• Implement Partitioning: Refactor monolithic state into component-based state files based on deployability boundaries.
• Enforce Encryption: Apply KMS encryption at rest and TLS for transit; validate with bucket policies.
• Restrict IAM: Limit state access to CI/CD roles; grant developers read-only access via terraform force-unlock or state pull only if necessary.
• Set Up Drift Detection: Schedule daily terraform plan checks and alert on non-empty diffs.
• Test Recovery: Perform a tabletop exercise or live drill to restore state from backup.
• Audit Secrets: Review state files for sensitive data; use sensitive = true and external secret managers where possible.

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
Small Team / Startup	S3 + DynamoDB Backend	Low cost, full control, standard AWS integration.	Low (Storage + DynamoDB write capacity)
Enterprise / Compliance	Terraform Cloud / Enterprise	Built-in audit trails, policy enforcement, SSO, no infra to manage.	High (License costs)
Multi-Account / Multi-Region	Partitioned Remote Backends	Isolates blast radius, enables parallel deployments, aligns with account boundaries.	Medium (Increased S3/DynamoDB overhead)
High Velocity CI/CD	Atlantis / Spacelift	Automated PR planning, locking, and approval workflows integrated with VCS.	Medium (Runner costs + Tool licensing)
Legacy Monolith State	Incremental Partitioning	Reduces risk by migrating components one at a time using terraform state mv.	Low (Engineering time only)

Configuration Template

Robust S3 Backend Module:

# modules/terraform-state/main.tf
variable "environment" { type = string }
variable "region"      { type = string }
variable "component"   { type = string }

locals {
  bucket_name = "tf-state-${var.environment}-${var.region}"
  key_path    = "${var.component}/terraform.tfstate"
}

resource "aws_s3_bucket" "state" {
  bucket = local.bucket_name
  # Force destroy disabled in prod to prevent accidental data loss
  lifecycle {
    prevent_destroy = true
  }
}

resource "aws_s3_bucket_versioning" "state" {
  bucket = aws_s3_bucket.state.id
  versioning_configuration {
    status = "Enabled"
  }
}

resource "aws_s3_bucket_server_side_encryption_configuration" "state" {
  bucket = aws_s3_bucket.state.id
  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm     = "aws:kms"
      kms_master_key_id = aws_kms_key.state.arn
    }
  }
}

resource "aws_dynamodb_table" "locks" {
  name         = "tf-locks-${var.environment}"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "LockID"
  
  attribute {
    name = "LockID"
    type = "S"
  }
  
  server_side_encryption {
    enabled = true
  }
}

output "backend_config" {
  value = {
    bucket         = aws_s3_bucket.state.id
    key            = local.key_path
    region         = var.region
    dynamodb_table = aws_dynamodb_table.locks.name
    encrypt        = true
  }
}

Quick Start Guide

1. Create Storage Infrastructure: Run the configuration template above to provision an S3 bucket with versioning/encryption and a DynamoDB table for locking.

   terraform init
   terraform apply -var="environment=dev" -var="region=us-east-1" -var="component=shared"
   

2. Configure Your Module: Add the backend block to your module's backend.tf using the outputs from step 1.

   terraform {
     backend "s3" {
       bucket         = "tf-state-dev-us-east-1"
       key            = "shared/terraform.tfstate"
       region         = "us-east-1"
       dynamodb_table = "tf-locks-dev"
       encrypt        = true
     }
   }
   

3. Initialize and Migrate: Run terraform init. Terraform will detect the new backend and prompt to copy existing state. Confirm the migration.

   terraform init
   # Select "yes" to copy state
   

4. Verify Locking: Check that the DynamoDB table contains a lock entry during operations.

   aws dynamodb scan --table-name tf-locks-dev
   

5. Enable Drift Detection: Add a scheduled job in your CI/CD pipeline to run terraform plan and alert on changes.

   terraform plan -detailed-exitcode
   # Exit code 2 indicates drift
   

By implementing these practices, you transform Terraform state from a liability into a reliable, scalable foundation for infrastructure management. Prioritize partitioning, security, and automation to maintain velocity as your infrastructure grows.