Skip to content

feat: CodeBuild+Terraform runtime for EKS repo creation #17

Merged
merged 14 commits into from
Apr 21, 2026

Conversation

arnol377
Copy link
Collaborator

@arnol377 arnol377 commented Apr 7, 2026

Summary

This workspace is executed by the eks-terragrunt-repo-creator CodeBuild project (triggered by the ECA Lambda) to create EKS cluster GitHub repositories via Terraform.

Changes

buildspec.yml

  • Added INSTALL phase: downloads Terraform 1.9.1 from S3 (hashicorp.com blocked on Census), installs Census CA cert via update-ca-trust
  • Added HTTPS_PROXY/HTTP_PROXY for registry.terraform.io provider downloads
  • Added github.e.it.census.gov to NO_PROXY for direct GHE connections
  • Added GIT_SSL_NO_VERIFY=true as belt-and-suspenders

providers.tf

  • Widened GitHub provider constraint to >= 6.11.0 (required by CSVD module's ~> 6.11 constraint)
  • Added provider "github" { insecure = true } for Census GHE TLS

variables.tf / locals.tf

  • Accepts cluster_config as JSON-encoded string from CodeBuild env vars

.github/copilot-instructions.md

  • Rewrites from 'Do NOT suggest CodeBuild' to accurate architecture documentation
  • Documents Census-specific networking (proxy, CA cert, NO_PROXY)
  • Documents token split: PAT (ghe-runner/github-token) for Terraform; App token for Lambda

callnotes.md

  • Step 6: documents all 8 fixes applied during e2e test debug cycle

Runtime Environment

HTTPS_PROXY=http://proxy.tco.census.gov:3128   # registry.terraform.io
NO_PROXY=...,github.e.it.census.gov            # direct to GHE
TF binary: s3://csvd-packer-pipeline-assets/terraform/terraform_1.9.1_linux_amd64.zip
CA cert:   s3://csvd-packer-pipeline-assets/certs/census-ca.pem

Test Result

sc-e2e-test-20260407-1402 — ✔ PASS (207s)

Your Name added 13 commits April 6, 2026 13:39
- Replace placeholder paths (environment/region/vpc/cluster/) with
  var.environment / var.region / var.cluster_config.vpc_name / var.name
  so managed_extra_files land in the correct Terragrunt hierarchy
- Change CSVD/terraform-github-repo source from SSH (git@) to HTTPS
  (git::https://) to work inside CodeBuild without SSH agent
Buildspec used by the 'eks-terragrunt-repo-creator' CodeBuild project
triggered by the Lambda function. Downloads Terraform from S3 assets bucket,
clones this repo using GITHUB_TOKEN env var, then runs:
  terraform init -no-color
  terraform apply -auto-approve -no-color
TF_VAR_* env vars are injected by the Lambda as CodeBuild environment
variable overrides.
The terraform-eks-deployment repo lives in SCT-Engineering, not CSVD.
CSVD would have caused git clone 404 in CodeBuild.
- Add REPO_BRANCH env var (currently fix/eca-copilot-instructions-and-callnotes)
  pointing to the branch with dynamic path fixes and HTTPS module source
- Pass --branch to git clone so CodeBuild checks out the right code
- Update REPO_BRANCH to 'main' once the fix branch is merged
terraform_1.9.0 zip was not in s3://csvd-packer-pipeline-assets/terraform/.
Uploaded terraform_1.9.1_linux_amd64.zip to that path from local tfenv install.
Public releases.hashicorp.com is blocked by Census network proxy.
…access

registry.terraform.io is blocked directly inside CodeBuild (Census network).
Must route through http://proxy.tco.census.gov:3128.
NO_PROXY excludes AWS-internal endpoints (.amazonaws.com) from proxy.
CSVD/terraform-github-repo module requires ~> 6.11; workspace had >= 6.6.0, < 6.7.0
which is incompatible. Lock file was pinned to 6.6.0 — delete so terraform init
regenerates it against the updated constraint.
The Census GHE TLS cert is signed by the Census internal CA which is not
present in the CodeBuild container trust store. insecure=true disables
x509 verification so terraform apply can call the GHE API.
- Download census-ca.pem from S3 assets bucket and add to Amazon Linux 2
  trust store via update-ca-trust during INSTALL phase
- Add github.e.it.census.gov to NO_PROXY so Terraform provider connects
  directly (not through proxy) and trusts Census CA chain
- Keep insecure=true in providers.tf as belt-and-suspenders
…hitecture

- Replace 'Lambda-Only Approach' and 'Do NOT suggest CodeBuild' sections
- Document full buildspec.yml runtime environment (proxy, CA cert, TF binary from S3)
- Add complete Key Resources table with CodeBuild projects and token sources
- Add Important Runtime Notes section with Census-specific networking requirements
- Update What NOT to Do section with correct guidance
The SC Product Deployment Methods section was near-identical to the
canonical version in lambda-template-repo-generator. Replace with a
concise cross-reference to keep a single source of truth.
main.tf Show resolved Hide resolved
main.tf Show resolved Hide resolved
…tion

Add scripts/rename_template_dirs.py (Python, httpx + rich) that calls the
GitHub API to delete environment/region/vpc/cluster/ placeholder paths from
the repo-init PR branch and re-add the eks-*/terragrunt.hcl files at their
correct computed paths:

  environment/region/vpc/cluster/eks-*/terragrunt.hcl
    → ${environment}/${region}/${vpc_name}/${cluster_name}/eks-*/terragrunt.hcl

Files already rendered by managed_extra_files (account.hcl, region.hcl,
vpc.hcl, cluster.hcl) are deleted from the placeholder paths but not
re-added — Terraform already wrote them with real values.

Controlled by var.run_in_codebuild (default false). buildspec.yml sets
TF_VAR_run_in_codebuild=true so the null_resource only runs in CodeBuild.

Also adds the null provider to providers.tf and pip3 install of httpx+rich
to the buildspec install phase.
Copy link
Collaborator

@morga471 morga471 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@morga471 morga471 merged commit c501efa into main Apr 21, 2026
1 check passed
@morga471 morga471 deleted the fix/eca-copilot-instructions-and-callnotes branch April 21, 2026 19:36
Sign in to join this conversation on GitHub.
Labels
None yet
Projects
None yet
2 participants