From a92195401c54d6e9de21ba6d53594dc9848211c8 Mon Sep 17 00:00:00 2001 From: Your Name Date: Tue, 17 Mar 2026 17:18:09 -0400 Subject: [PATCH] chore: vendor codebuild/tf; add make sync-gist to push back to gist - README: replace all gist download references with codebuild/tf path - Automation section: reflect that tf is vendored, not downloaded - install phase table: 'copies codebuild/tf -> /usr/local/bin/tf' - Makefile: add sync-gist target (PATCH GHE gist API via GITHUB_TOKEN) --- README.md | 14 +++++++------- codebuild/Makefile | 25 ++++++++++++++++++++++++- 2 files changed, 31 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 9b34c32..467b395 100644 --- a/README.md +++ b/README.md @@ -111,8 +111,8 @@ See **[AWS_PERMISSIONS.md](./AWS_PERMISSIONS.md)** for detailed permission polic ### Tools Required - Terraform >= 1.0 - - **Local shell**: use the [`tf` script](https://github.e.it.census.gov/gist/arnol377/21b70dd6790d2680a119a9f86369eced). It wraps `terraform` and automatically sets `TF_DATA_DIR`, `TF_CLI_ARGS_*` var-file injection, and loads workspace-specific JSON env files — all things the plain `terraform` binary does not do on its own. - - **CodeBuild**: uses native `terraform` commands. The buildspec replicates the `tf` script's behavior explicitly via environment variables (see [Automation (CodeBuild)](#automation-codebuild)). + - **Local shell**: use the [`tf` script](codebuild/tf) vendored in the repo at `codebuild/tf`. It wraps `terraform` and automatically sets `TF_DATA_DIR`, `TF_CLI_ARGS_*` var-file injection, and loads workspace-specific JSON env files — all things the plain `terraform` binary does not do on its own. + - **CodeBuild**: installs `codebuild/tf` from the repo checkout during the `install` phase (see [Automation (CodeBuild)](#automation-codebuild)). - AWS CLI configured - Python 3 + pip (required for Lambda package build when `enable_lambda_token_refresh = true`) - Access to the organization's Terraform state backend @@ -197,7 +197,7 @@ tf plan tf apply ``` -> **Important:** Always use the [`tf` script](https://github.e.it.census.gov/gist/arnol377/21b70dd6790d2680a119a9f86369eced) instead of `terraform` directly in your local shell. It sets `TF_DATA_DIR`, injects the workspace-specific var-file via `TF_CLI_ARGS_*`, and loads env vars from the JSON file — plain `terraform` will not do this automatically. +> **Important:** Always use the [`tf` script](codebuild/tf) (`codebuild/tf`, also installed to your `PATH` as `tf`) instead of `terraform` directly in your local shell. It sets `TF_DATA_DIR`, injects the workspace-specific var-file via `TF_CLI_ARGS_*`, and loads env vars from the JSON file — plain `terraform` will not do this automatically. ### 7. Confirm SNS Email Subscription @@ -217,15 +217,15 @@ aws ecs list-tasks --cluster ecs-ghe-runners-us-gov-west-1 ## Automation (CodeBuild) -The `codebuild/` directory contains a Terraform module that deploys a scheduled CodeBuild project (`ghe-runner-daily-{workspace}`) that runs `tf apply` daily. The buildspec downloads the [`tf` script](https://github.e.it.census.gov/gist/arnol377/21b70dd6790d2680a119a9f86369eced) from the team gist during the `install` phase and uses it throughout, so all workspace-aware behavior (`TF_DATA_DIR`, var-file injection, JSON env loading) is handled automatically — no manual replication needed. +The `codebuild/` directory contains a Terraform module that deploys a scheduled CodeBuild project (`ghe-runner-daily-{workspace}`) that runs `tf apply` daily. The `tf` script is **vendored in the repo** at [`codebuild/tf`](codebuild/tf) and copied to `/usr/local/bin/tf` during the `install` phase, so all workspace-aware behavior (`TF_DATA_DIR`, var-file injection, JSON env loading) is handled automatically. To publish changes back to the canonical gist, run `make sync-gist` from `codebuild/`. ### How Workspace Selection Works in CodeBuild -The `tf` script is downloaded once at install time and used natively: +The `tf` script is installed from the repo checkout at build time: | Responsibility | How it's handled | |----------------|------------------| -| `TF_DATA_DIR`, `TF_CLI_ARGS_*`, JSON env vars | Downloaded [`tf` script](https://github.e.it.census.gov/gist/arnol377/21b70dd6790d2680a119a9f86369eced) handles everything automatically | +| `TF_DATA_DIR`, `TF_CLI_ARGS_*`, JSON env vars | Vendored [`codebuild/tf`](codebuild/tf) handles everything automatically | | Target workspace | `TF_WORKSPACE` env var injected by CodeBuild at build time | | Backend config | `pre_build` selects `backend-configs/${TF_WORKSPACE}.tf` if present | | GitHub PAT | `GITHUB_TOKEN` (from Secrets Manager) exported as `TF_VAR_github_token` | @@ -236,7 +236,7 @@ The `tf` binary resolver falls back to `terraform` in `PATH`, which is the binar | Phase | What happens | |-------|--------------| -| `install` | Downloads Terraform `${TF_VERSION}`; downloads `tf` script from gist; installs `requests` | +| `install` | Downloads Terraform `${TF_VERSION}`; copies `codebuild/tf` → `/usr/local/bin/tf`; warms provider cache from S3; installs `requests` | | `pre_build` | Exports `TF_VAR_github_token`; runs `tf init` (with optional backend-config); runs `tf workspace select` | | `build` | `tf apply -auto-approve`; invokes Lambda token refresh (so token is fresh before containers start); forces new ECS deployment | | `post_build` | Logs completion timestamp and active workspace | diff --git a/codebuild/Makefile b/codebuild/Makefile index 22cc2e2..332e181 100644 --- a/codebuild/Makefile +++ b/codebuild/Makefile @@ -86,7 +86,7 @@ RED := $(shell tput setaf 1 2>/dev/null) .PHONY: help init plan apply destroy validate fmt \ setup-credentials secret show-outputs trigger logs clean check-env \ - force-unlock break-lock + force-unlock break-lock sync-gist # ── Default target ──────────────────────────────────────────────────────────── @@ -317,6 +317,29 @@ clean: @echo "$(BOLD)Removing .terraform/...$(RESET)" rm -rf .terraform .terraform.lock.hcl +## sync-gist: Push the vendored codebuild/tf script to the canonical GHE gist +GIST_ID ?= 21b70dd6790d2680a119a9f86369eced +GIST_HOST ?= github.e.it.census.gov +sync-gist: check-env + @[ -n "$(GITHUB_TOKEN)" ] || { echo "$(RED)ERROR: GITHUB_TOKEN is not set$(RESET)"; exit 1; } + @echo "$(BOLD)Syncing codebuild/tf -> gist $(GIST_ID) on $(GIST_HOST)...$(RESET)" + @python3 -c " +import json, sys +content = open('tf').read() +print(json.dumps({'files': {'tf': {'content': content}}})) +" | curl -s -k -X PATCH \ + -H "Authorization: token $(GITHUB_TOKEN)" \ + -H "Content-Type: application/json" \ + -d @- \ + "https://$(GIST_HOST)/api/v3/gists/$(GIST_ID)" | python3 -c " +import json, sys +r = json.load(sys.stdin) +if 'id' in r: + print(' OK: gist/' + r['id'] + ' updated') +else: + print('ERROR: ' + json.dumps(r)); sys.exit(1) +" + # ── Internal helpers (not shown in help) ───────────────────────────────────── _require-secret-arn: