chore: update docs, scripts, and state to reflect current architecture

- copilot-instructions: simplify rebuild steps to use packer-pipeline; add Python/CLI standards section; add AWS_DEFAULT_REGION reminder; add 'do not re-add AwsRegion/AWSAccountId' to What NOT to Do - DEPLOYMENT.md, CLOUDFORMATION_CUSTOM_RESOURCE_MIGRATION.md: fix packer-pipeline invocation to use csvd_config_packer.hcl - csvd_config_packer.hcl: remove duplicate stale comment block at EOF - DEMO_SCRIPT.md: update param table and step 3 walkthrough to match the corrected SC form (no AwsRegion/AWSAccountId, correct field names) - test_service_catalog.py: bump artifact to v2.1; remove AwsRegion and - docs/SC-TEMPLATE-FIX-PLAN.md: add completed fix plan for reference - deploy state: updated after today's tf apply
CSVD · Apr 21, 2026 · 8b268ff · 8b268ff
1 parent 237ab9b
commit 8b268ff
Show file tree

Hide file tree

Showing 10 changed files with 509 additions and 318 deletions.
diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md
@@ -174,26 +174,20 @@ Properties:
 
 ## Rebuilding the Lambda Image
 
-When `template_automation/app.py` or other Lambda source files change:
+When `template_automation/app.py` or other Lambda source files change, use `packer-pipeline`:
 
 ```bash
-# 1. Zip source and upload to S3
 cd lambda-template-repo-generator
-zip -r ~/tmp/lambda-source.zip . -x "*.git*" -x "design-docs/*" -x "__pycache__/*" -x "*.pyc" -x "deploy/.terraform/*" -x "deploy/terraform.tfstate*"
-UUID=$(python3 -c "import uuid; print(uuid.uuid4())")
 source ~/aws-creds
-aws s3 cp ~/tmp/lambda-source.zip \
-  "s3://csvd-packer-pipeline-builds/packer-builds/eks-terragrunt-repo-generator/source/${UUID}/repo.zip" \
-  --region us-gov-west-1
+packer-pipeline --config csvd_config_packer.hcl
+```
+
+This handles zipping the source, uploading to S3, and triggering the
+`eks-terragrunt-repo-generator-builder` CodeBuild project automatically.
 
-# 2. Start the packer CodeBuild build
-aws codebuild start-build \
-  --project-name eks-terragrunt-repo-generator-builder \
-  --region us-gov-west-1 \
-  --source-type-override S3 \
-  --source-location-override "csvd-packer-pipeline-builds/packer-builds/eks-terragrunt-repo-generator/source/${UUID}/repo.zip"
+After the build completes (SUCCEEDED), force the Lambda to pull the new image:
 
-# 3. After build SUCCEEDED, force Lambda to pull the new image
+```bash
 aws lambda update-function-code \
   --function-name eks-terragrunt-repo-gen-template-automation \
   --image-uri "229685449397.dkr.ecr.us-gov-west-1.amazonaws.com/eks-terragrunt-repo-generator/lambda:latest" \
@@ -214,13 +208,49 @@ python scripts/cleanup_test_repos.py
 
 ---
 
+## Python & CLI Automation Standards
+
+All automation scripts in this project are written in **Python 3**. Use the following libraries
+as the standard stack — do not introduce alternatives without good reason:
+
+| Purpose | Library |
+|---------|---------|
+| Data validation / config models | `pydantic` (v2) |
+| Rich terminal output / progress | `rich` |
+| CLI argument parsing | `typer` (preferred) or `argparse` |
+| AWS API calls | `boto3` |
+| YAML config files | `pyyaml` |
+| HTTP calls | `httpx` or `requests` |
+
+### `AWS_DEFAULT_REGION` — always required
+
+The account is in `us-gov-west-1`. Many boto3 calls and the AWS CLI silently fail or
+target the wrong region if `AWS_DEFAULT_REGION` is not set.
+
+**Always export it before any AWS CLI or boto3 script:**
+
+```bash
+export AWS_DEFAULT_REGION=us-gov-west-1
+source ~/aws-creds
+```
+
+### SC Template Parameters
+
+`aws_account_id` and `aws_region` are **not** on the SC product form — the CFN template
+resolves them automatically via `!Sub "${AWS::AccountId}"` and `!Sub "${AWS::Region}"`
+before the Lambda is called. Do not add them back as user-facing parameters.
+
+---
+
 ## What NOT to Do
 
 - ❌ Do not rewrite repo creation logic in Lambda Python — all repo creation runs in CodeBuild via `terraform-eks-deployment`
 - ❌ Do not use `HappyPathway/terraform-github-repo` **public** module — it pins `github ~> 6.0` (conflicts with internal `>= 6.6.0`)
 - ✅ DO use `CSVD/terraform-github-repo` (https://github.e.it.census.gov/CSVD/terraform-github-repo) — internal module, supports `template_repo` + `managed_extra_files`
 - ❌ Do not pass `vpc_id` to the Lambda — use `vpc_name`
 - ❌ Do not re-add `LambdaFunctionArn` as a CFN parameter — use `!Sub "arn:..."` directly
+- ❌ Do not re-add `AWSAccountId` or `AwsRegion` as SC product form parameters — use `!Sub` auto-resolution
 - ❌ Do not use SSH-based module sources (`git::ssh://`) — Census proxy blocks SSH host key exchange; use HTTPS
 - ❌ Do not write temp files or command output to `/tmp` — use `~/tmp` (i.e. `/home/a/arnol377/tmp`) instead
 - ❌ Do not use the `terraform` command directly — always use the `tf` alias (e.g. `tf plan`, `tf apply`, `tf init`)
+- ❌ Do not run AWS CLI or boto3 without first exporting `AWS_DEFAULT_REGION=us-gov-west-1`
diff --git a/CLOUDFORMATION_CUSTOM_RESOURCE_MIGRATION.md b/CLOUDFORMATION_CUSTOM_RESOURCE_MIGRATION.md
@@ -229,7 +229,8 @@ Sees outputs:
 1. **Deploy Lambda** with updated code:
    ```bash
    cd /home/a/arnol377/git/lambda-template-repo-generator
-   packer-pipeline build --config config_packer.hcl
+   source ~/aws-creds
+   packer-pipeline --config csvd_config_packer.hcl
    ```
 
 2. **Update Infrastructure**:

diff --git a/DEPLOYMENT.md b/DEPLOYMENT.md
@@ -64,7 +64,7 @@ terraform apply -var-file=varfiles/default.tfvars
 cd /path/to/lambda-template-repo-generator
 
 # Build container image via CodeBuild (waits for completion, ~4 minutes)
-packer-pipeline --config config_packer.hcl --wait
+packer-pipeline --config csvd_config_packer.hcl
 ```
 
 This will:
@@ -259,7 +259,7 @@ When you change Lambda code in `template_automation/`:
 
 ```bash
 # 1. Build new container
-packer-pipeline --config config_packer.hcl --wait
+packer-pipeline --config csvd_config_packer.hcl
 
 # 2. Update Lambda to new image
 aws lambda update-function-code \

diff --git a/csvd_config_packer.hcl b/csvd_config_packer.hcl
@@ -109,92 +109,3 @@ packer_pipeline {
     }
   ]
 }
-
-  // Required parameters
-  packer_template_file = "packer.pkr.hcl"  // Relative path within the repo to the Packer template
-  s3_bucket            = "csvd-packer-pipeline-builds"     // S3 bucket for artifacts (derived from environment_name)
-  assets_bucket        = "csvd-packer-pipeline-assets"     // S3 bucket containing tool assets (derived from environment_name)
-  codebuild_project_name = "eks-terragrunt-repo-generator-builder"  // Name for the CodeBuild project
-
-  // Tools configuration
-  tools = [
-    {
-      name         = "packer"
-      version      = "1.10.3"
-      zip_path     = "packer_1.10.3_linux_amd64.zip"
-      binary_name  = "packer"
-      install_path = "/usr/local/bin"
-    }
-  ]
-
-  // AWS Account Configuration
-  account_number       = "229685449397"  // AWS account number
-  partition            = "aws-us-gov"    // AWS partition (aws or aws-us-gov)
-
-  // Role management
-  create_role          = false  // Role already exists; provide ARN directly
-  codebuild_role_arn   = "arn:aws-us-gov:iam::229685449397:role/CodeBuildPackerRole-eks-terragrunt-repo-generator-builder"
-
-  // Region and partition configuration
-  aws_region           = "us-gov-west-1"  // AWS region
-  gov_cloud           = true  // Explicitly set GovCloud partition
-
-  // Optional parameters with defaults
-  s3_key_prefix        = "packer-builds/eks-terragrunt-repo-generator"  // Prefix for S3 keys
-  compute_type         = "BUILD_GENERAL1_MEDIUM"  // CodeBuild compute type
-  image                = "aws/codebuild/amazonlinux2-x86_64-standard:3.0"  // CodeBuild image
-  buildspec_template   = "buildspec.yml.j2"  // Buildspec template file
-
-  // Post-build commands to push Docker image to ECR
-  additional_post_build_commands  = "docker push ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/${ECR_REPOSITORY}:${IMAGE_TAG}"
-
-  // Exclude patterns for zip creation
-  exclude_dirs     = [
-    "design-docs",
-    "docs",
-    "dist",
-    "events",
-    "scripts",
-    "tests"
-  ]
-
-  // VPC Configuration with the specified details
-  vpc_config {
-    vpc_id             = "vpc-00576a396ec570b94"  // Specified VPC
-    subnet_ids         = ["subnet-0b1992a84536c581b"]  // Subnet ID
-    security_group_ids = ["sg-0641c697588b9aa6b"]  // Security group ID
-  }
-
-  // Environment variables for the CodeBuild environment
-  environment_variables = {
-    REPOSITORY_NAME = "eks-terragrunt-repo-generator-lambda"
-    ECR_REPOSITORY  = "eks-terragrunt-repo-generator/lambda"
-    AWS_ACCOUNT_ID  = "229685449397"
-    IMAGE_TAG       = "latest"
-    HTTP_PROXY      = "http://proxy.tco.census.gov:3128"
-    HTTPS_PROXY     = "http://proxy.tco.census.gov:3128"
-    // NO_PROXY: things that should bypass the proxy entirely
-    //   - 169.254.*           = EC2 metadata / ECS task credentials
-    //   - .s3.*amazonaws.com  = S3 via VPC gateway endpoint
-    //   - .dkr.ecr / .ecr    = private ECR via internal routing
-    //   - sts / logs          = AWS service endpoints via internal routing
-    //   - github.e.it / nexus = internal census hosts
-    // Everything else (pypi.org, files.pythonhosted.org, public.ecr.aws) goes through proxy
-    NO_PROXY        = "169.254.169.254,169.254.170.2,.s3.us-gov-west-1.amazonaws.com,.s3.amazonaws.com,.s3-fips.us-gov-west-1.amazonaws.com,.dkr.ecr.us-gov-west-1.amazonaws.com,.ecr.us-gov-west-1.amazonaws.com,sts.us-gov-west-1.amazonaws.com,logs.us-gov-west-1.amazonaws.com,github.e.it.census.gov,nexus.it.census.gov"
-    ECR_REGISTRY    = "229685449397.dkr.ecr.us-gov-west-1.amazonaws.com"  // ECR registry URL
-  }
-
-  // ECR Image Cloning Configuration
-  ecr_registry_name = "eks-terragrunt-repo-generator"  // ECR registry prefix for cloned images
-
-  ecr_clone_images = [
-    {
-      name            = "lambda-python"
-      tag             = "3.11"
-      source_registry = "public.ecr.aws"
-      source_image    = "lambda/python"
-      source_tag      = "3.11"
-      enabled         = true
-    }
-  ]
-}
diff --git a/deploy/.terraform_commits b/deploy/.terraform_commits
@@ -160,5 +160,23 @@
     "commit_message": "fix: use PAT (ghe-runner/github-token) for Terraform GitHub provider in CodeBuild\n\nThe standard github_token (/eks-cluster-deployment/github_token) is a GitHub\nApp installation token (ghs_ prefix) which cannot access /api/v3/user. This\nendpoint is always called by the CSVD terraform-github-repo module's\ndata.github_user.current resource.\n\nChanges:\n- app.py: check TF_GITHUB_TOKEN_SECRET_NAME env var first for CodeBuild token;\n  falls back to GITHUB_TOKEN_SECRET_NAME if not set\n- deploy/main.tf: add TF_GITHUB_TOKEN_SECRET_NAME=ghe-runner/github-token env var\n- deploy/main.tf: add IAM policy granting Lambda access to ghe-runner/github-token",
     "author": "Your Name",
     "timestamp": "2026-04-07T13:10:20.067727"
+  },
+  {
+    "commit_hash": "e6547ed0a07eaddd227ba8ab7b278f03e4896a91",
+    "commit_message": "docs: add ECA demo script with talking points and Q&A prep",
+    "author": "Your Name",
+    "timestamp": "2026-04-20T13:33:25.979480"
+  },
+  {
+    "commit_hash": "e6547ed0a07eaddd227ba8ab7b278f03e4896a91",
+    "commit_message": "docs: add ECA demo script with talking points and Q&A prep",
+    "author": "Your Name",
+    "timestamp": "2026-04-21T15:45:48.422507"
+  },
+  {
+    "commit_hash": "e6547ed0a07eaddd227ba8ab7b278f03e4896a91",
+    "commit_message": "docs: add ECA demo script with talking points and Q&A prep",
+    "author": "Your Name",
+    "timestamp": "2026-04-21T16:04:00.320816"
   }
 ]