sc-lambda-ghactions

Service Catalog → Lambda → CodeBuild → Account Repo (tf-run + PR) automation.

This repo contains every piece of the next-generation SC automation platform: the Lambda handler, the CodeBuild buildspec, the tf-run toolchain scripts, the Service Catalog CFN product template, and all Terraform that deploys it.

Architecture

User (SC Console)
  └─> fills product form → submits
        → CFN creates Custom::TerraformRun resource
              → ServiceToken → Lambda tf-run-executor-trigger
                    │         (centralized in csvd-dev, us-gov-west-1)
                    ├─> Validates inputs (Pydantic v2)
                    ├─> Fetches GHE PAT from Secrets Manager (ghe-runner/github-token)
                    ├─> Starts CodeBuild tf-run-executor with env-var overrides
                    └─> Polls CodeBuild every 20 s → returns PR URL to CFN

CodeBuild tf-run-executor (Amazon Linux 2, 60 min timeout)
  ├─> INSTALL
  │     ├─> Terraform binary from S3 (registry.terraform.io is blocked)
  │     ├─> Census CA cert → update-ca-trust (for GHE TLS)
  │     ├─> tf-run toolchain from scripts/ (tf-run, tf-control.sh, tf-directory-setup.py)
  │     └─> gh CLI from S3
  └─> BUILD
        ├─> git clone https://<token>@github.e.it.census.gov/SCT-Engineering/<ACCOUNT_REPO>
        ├─> git checkout -B <GIT_BRANCH>   (default: repo-init)
        ├─> Write EXTRA_FILES (JSON map path→content) into the working tree
        ├─> git add + commit + push
        ├─> cd <LAYER>/<REGION_DIR>/
        ├─> TFARGS=-auto-approve tf-run apply [tag:<TF_RUN_START_TAG>]
        │     (or tf-plan if DRY_RUN=true)
        └─> gh pr create --base main --head <GIT_BRANCH>
              POST_BUILD: emit PR_URL= line for Lambda to parse

Repo Layout

sc-lambda-ghactions/
├── buildspec.yml                  # CodeBuild build definition (source: this repo)
├── scripts/
│   ├── tf-run                     # Bash tf-run orchestrator (v1.13.13)
│   ├── tf-control.sh              # tf-{action} wrapper script (v1.11.0)
│   ├── tf-run.py                  # Python port of tf-run (v2.0.0)
│   └── tf-directory-setup.py     # Generates remote_state.backend.tf from remote_state.yml
├── data/
│   └── tf-run.data                # Sample tf-run.data for reference
├── lambda/
│   ├── app.py                     # Lambda handler (Python 3.12)
│   ├── requirements.txt           # boto3, pydantic
│   └── Dockerfile                 # Python 3.12 Lambda container image
├── service-catalog/
│   └── product-template.yaml      # CFN template for the SC product
└── deploy/
    ├── provider.tf                # AWS provider, Terraform version constraint
    ├── variables.tf               # All tunable inputs (with sensible defaults)
    ├── codebuild.tf               # aws_codebuild_project.tf_run_executor + GHE credential
    ├── iam.tf                     # Lambda execution role + CodeBuild service role
    ├── lambda.tf                  # ECR repo + aws_lambda_function + cross-account permission
    └── service_catalog.tf         # SC portfolio, product, launch constraint, S3 template upload

Service Catalog Product Parameters

SC Form Field	Variable	Notes
Account Repo Name	`account_repo`	e.g. `229685449397-csvd-dev-platform-dev-gov`
Terraform Layer	`layer`	`common`, `infrastructure`, or `vpc`
Region Directory	`region_dir`	`east` or `west`
Git Branch	`git_branch`	branch to commit/PR from; default `repo-init`
Start Tag (optional)	`tf_run_start_tag`	tf-run.data TAG label; empty = run all steps
Dry Run	`dry_run`	`true` = tf plan only, no apply
Extra Config Files (JSON)	`extra_files`	`{"relative/path": "content"}` written before tf-run

aws_account_id and aws_region are not user-facing — resolved via !Sub in the CFN template.

Deploying

Prerequisites

Terraform ≥ 1.3 (via tf alias)
AWS credentials for csvd-dev (229685449397, us-gov-west-1)
GHE PAT already in Secrets Manager as ghe-runner/github-token
An S3 bucket to hold the SC product template artifact

Required Terraform variables

# deploy/terraform.tfvars
source_repo_url       = "https://github.e.it.census.gov/SCT-Engineering/sc-lambda-ghactions"
artifacts_bucket_name = "csvd-sc-product-templates"   # your SC artifacts bucket
org_id                = "o-abc123def4"                 # your AWS Org ID

Deploy

export AWS_DEFAULT_REGION=us-gov-west-1
source ~/aws-creds

cd sc-lambda-ghactions/deploy
tf init
tf apply

Build and push the Lambda image

After tf apply creates the ECR repo:

aws ecr get-login-password --region us-gov-west-1 \
  | docker login --username AWS \
      --password-stdin 229685449397.dkr.ecr.us-gov-west-1.amazonaws.com

docker build -t tf-run-executor/lambda:latest lambda/
docker tag tf-run-executor/lambda:latest \
  229685449397.dkr.ecr.us-gov-west-1.amazonaws.com/tf-run-executor/lambda:latest
docker push 229685449397.dkr.ecr.us-gov-west-1.amazonaws.com/tf-run-executor/lambda:latest

Then update the function to pick up the new image:

aws lambda update-function-code \
  --function-name tf-run-executor-trigger \
  --image-uri 229685449397.dkr.ecr.us-gov-west-1.amazonaws.com/tf-run-executor/lambda:latest \
  --region us-gov-west-1

Manual CodeBuild test (before SC wiring)

export AWS_DEFAULT_REGION=us-gov-west-1
source ~/aws-creds

aws codebuild start-build \
  --project-name tf-run-executor \
  --environment-variables-override \
    "name=ACCOUNT_REPO,value=229685449397-csvd-dev-platform-dev-gov,type=PLAINTEXT" \
    "name=LAYER,value=infrastructure,type=PLAINTEXT" \
    "name=REGION_DIR,value=west,type=PLAINTEXT" \
    "name=DRY_RUN,value=true,type=PLAINTEXT" \
    "name=GITHUB_TOKEN,value=$(aws secretsmanager get-secret-value \
        --secret-id ghe-runner/github-token --query SecretString --output text),type=PLAINTEXT"

Key AWS Resources

Resource	Name	Purpose
Lambda	`tf-run-executor-trigger`	CFN Custom Resource handler
CodeBuild	`tf-run-executor`	Runs tf-run in the account repo
ECR	`tf-run-executor/lambda`	Lambda container image
Secrets Manager	`ghe-runner/github-token`	GHE PAT used by both Lambda (to start CodeBuild) and CodeBuild (to clone + gh CLI)
SC Portfolio	`sc-automation-tf-run`	Groups the tf-run product
SC Product	`sc-automation-tf-run-executor`	CFN template + launch constraint

Census Network Notes

Terraform registry (registry.terraform.io) is blocked — binary pulled from s3://csvd-packer-pipeline-assets/terraform/
GHE TLS: Census CA cert not in standard bundles → installed from s3://csvd-packer-pipeline-assets/certs/census-ca.pem via update-ca-trust
Proxy: HTTPS_PROXY=http://proxy.tco.census.gov:3128 for provider downloads; NO_PROXY includes github.e.it.census.gov,169.254.169.254
SSH blocked: all git operations use HTTPS with token in URL

Related Repos

Repo	Purpose
`lambda-template-repo-generator`	Current CodeBuild-based EKS Lambda (predecessor)
`terraform-service-catalog-census`	Census-managed SC product templates (production deployment path)
`terraform-eks-deployment`	EKS repo creation Terraform workspace (predecessor CodeBuild payload)

SCT-Engineering/sc-lambda-ghactions

About

Resources

Stars

Watchers

Forks

Releases

Languages