Skip to content

Commit

Permalink
chore: vendor codebuild/tf; add make sync-gist to push back to gist
Browse files Browse the repository at this point in the history
- README: replace all gist download references with codebuild/tf path
- Automation section: reflect that tf is vendored, not downloaded
- install phase table: 'copies codebuild/tf -> /usr/local/bin/tf'
- Makefile: add sync-gist target (PATCH GHE gist API via GITHUB_TOKEN)
  • Loading branch information
Your Name committed Mar 17, 2026
1 parent 03aefd5 commit a921954
Show file tree
Hide file tree
Showing 2 changed files with 31 additions and 8 deletions.
14 changes: 7 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,8 +111,8 @@ See **[AWS_PERMISSIONS.md](./AWS_PERMISSIONS.md)** for detailed permission polic

### Tools Required
- Terraform >= 1.0
- **Local shell**: use the [`tf` script](https://github.e.it.census.gov/gist/arnol377/21b70dd6790d2680a119a9f86369eced). It wraps `terraform` and automatically sets `TF_DATA_DIR`, `TF_CLI_ARGS_*` var-file injection, and loads workspace-specific JSON env files — all things the plain `terraform` binary does not do on its own.
- **CodeBuild**: uses native `terraform` commands. The buildspec replicates the `tf` script's behavior explicitly via environment variables (see [Automation (CodeBuild)](#automation-codebuild)).
- **Local shell**: use the [`tf` script](codebuild/tf) vendored in the repo at `codebuild/tf`. It wraps `terraform` and automatically sets `TF_DATA_DIR`, `TF_CLI_ARGS_*` var-file injection, and loads workspace-specific JSON env files — all things the plain `terraform` binary does not do on its own.
- **CodeBuild**: installs `codebuild/tf` from the repo checkout during the `install` phase (see [Automation (CodeBuild)](#automation-codebuild)).
- AWS CLI configured
- Python 3 + pip (required for Lambda package build when `enable_lambda_token_refresh = true`)
- Access to the organization's Terraform state backend
Expand Down Expand Up @@ -197,7 +197,7 @@ tf plan
tf apply
```
> **Important:** Always use the [`tf` script](https://github.e.it.census.gov/gist/arnol377/21b70dd6790d2680a119a9f86369eced) instead of `terraform` directly in your local shell. It sets `TF_DATA_DIR`, injects the workspace-specific var-file via `TF_CLI_ARGS_*`, and loads env vars from the JSON file — plain `terraform` will not do this automatically.
> **Important:** Always use the [`tf` script](codebuild/tf) (`codebuild/tf`, also installed to your `PATH` as `tf`) instead of `terraform` directly in your local shell. It sets `TF_DATA_DIR`, injects the workspace-specific var-file via `TF_CLI_ARGS_*`, and loads env vars from the JSON file — plain `terraform` will not do this automatically.
### 7. Confirm SNS Email Subscription
Expand All @@ -217,15 +217,15 @@ aws ecs list-tasks --cluster ecs-ghe-runners-us-gov-west-1
## Automation (CodeBuild)
The `codebuild/` directory contains a Terraform module that deploys a scheduled CodeBuild project (`ghe-runner-daily-{workspace}`) that runs `tf apply` daily. The buildspec downloads the [`tf` script](https://github.e.it.census.gov/gist/arnol377/21b70dd6790d2680a119a9f86369eced) from the team gist during the `install` phase and uses it throughout, so all workspace-aware behavior (`TF_DATA_DIR`, var-file injection, JSON env loading) is handled automatically — no manual replication needed.
The `codebuild/` directory contains a Terraform module that deploys a scheduled CodeBuild project (`ghe-runner-daily-{workspace}`) that runs `tf apply` daily. The `tf` script is **vendored in the repo** at [`codebuild/tf`](codebuild/tf) and copied to `/usr/local/bin/tf` during the `install` phase, so all workspace-aware behavior (`TF_DATA_DIR`, var-file injection, JSON env loading) is handled automatically. To publish changes back to the canonical gist, run `make sync-gist` from `codebuild/`.
### How Workspace Selection Works in CodeBuild
The `tf` script is downloaded once at install time and used natively:
The `tf` script is installed from the repo checkout at build time:
| Responsibility | How it's handled |
|----------------|------------------|
| `TF_DATA_DIR`, `TF_CLI_ARGS_*`, JSON env vars | Downloaded [`tf` script](https://github.e.it.census.gov/gist/arnol377/21b70dd6790d2680a119a9f86369eced) handles everything automatically |
| `TF_DATA_DIR`, `TF_CLI_ARGS_*`, JSON env vars | Vendored [`codebuild/tf`](codebuild/tf) handles everything automatically |
| Target workspace | `TF_WORKSPACE` env var injected by CodeBuild at build time |
| Backend config | `pre_build` selects `backend-configs/${TF_WORKSPACE}.tf` if present |
| GitHub PAT | `GITHUB_TOKEN` (from Secrets Manager) exported as `TF_VAR_github_token` |
Expand All @@ -236,7 +236,7 @@ The `tf` binary resolver falls back to `terraform` in `PATH`, which is the binar
| Phase | What happens |
|-------|--------------|
| `install` | Downloads Terraform `${TF_VERSION}`; downloads `tf` script from gist; installs `requests` |
| `install` | Downloads Terraform `${TF_VERSION}`; copies `codebuild/tf` → `/usr/local/bin/tf`; warms provider cache from S3; installs `requests` |
| `pre_build` | Exports `TF_VAR_github_token`; runs `tf init` (with optional backend-config); runs `tf workspace select` |
| `build` | `tf apply -auto-approve`; invokes Lambda token refresh (so token is fresh before containers start); forces new ECS deployment |
| `post_build` | Logs completion timestamp and active workspace |
Expand Down
25 changes: 24 additions & 1 deletion codebuild/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ RED := $(shell tput setaf 1 2>/dev/null)

.PHONY: help init plan apply destroy validate fmt \
setup-credentials secret show-outputs trigger logs clean check-env \
force-unlock break-lock
force-unlock break-lock sync-gist

# ── Default target ────────────────────────────────────────────────────────────

Expand Down Expand Up @@ -317,6 +317,29 @@ clean:
@echo "$(BOLD)Removing .terraform/...$(RESET)"
rm -rf .terraform .terraform.lock.hcl

## sync-gist: Push the vendored codebuild/tf script to the canonical GHE gist
GIST_ID ?= 21b70dd6790d2680a119a9f86369eced
GIST_HOST ?= github.e.it.census.gov
sync-gist: check-env
@[ -n "$(GITHUB_TOKEN)" ] || { echo "$(RED)ERROR: GITHUB_TOKEN is not set$(RESET)"; exit 1; }
@echo "$(BOLD)Syncing codebuild/tf -> gist $(GIST_ID) on $(GIST_HOST)...$(RESET)"
@python3 -c "
import json, sys
content = open('tf').read()
print(json.dumps({'files': {'tf': {'content': content}}}))
" | curl -s -k -X PATCH \
-H "Authorization: token $(GITHUB_TOKEN)" \
-H "Content-Type: application/json" \
-d @- \
"https://$(GIST_HOST)/api/v3/gists/$(GIST_ID)" | python3 -c "
import json, sys
r = json.load(sys.stdin)
if 'id' in r:
print(' OK: gist/' + r['id'] + ' updated')
else:
print('ERROR: ' + json.dumps(r)); sys.exit(1)
"

# ── Internal helpers (not shown in help) ─────────────────────────────────────

_require-secret-arn:
Expand Down

0 comments on commit a921954

Please sign in to comment.