From 0b067b7292fb8b07e701decc35cd3e6fec3bfdab Mon Sep 17 00:00:00 2001 From: Dave Arnold Date: Tue, 28 Apr 2026 16:07:52 -0400 Subject: [PATCH] =?UTF-8?q?chore:=20add=20Copilot=20customizations=20?= =?UTF-8?q?=E2=80=94=20instructions,=20agents,=20prompts,=20skills?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .github/agents/implementation.agent.md | 105 ++++++++++ .github/agents/planner.agent.md | 75 ++++++++ .github/agents/reviewer.agent.md | 82 ++++++++ .github/copilot-instructions.md | 172 +++++++++++++++++ .../instructions/terraform.instructions.md | 126 ++++++++++++ .github/prompts/analyze-pr-comments.prompt.md | 66 +++++++ .../prompts/new-account-repo-layer.prompt.md | 69 +++++++ .github/prompts/review-sc-template.prompt.md | 64 +++++++ .github/skills/account-repo-analysis/SKILL.md | 180 ++++++++++++++++++ 9 files changed, 939 insertions(+) create mode 100644 .github/agents/implementation.agent.md create mode 100644 .github/agents/planner.agent.md create mode 100644 .github/agents/reviewer.agent.md create mode 100644 .github/copilot-instructions.md create mode 100644 .github/instructions/terraform.instructions.md create mode 100644 .github/prompts/analyze-pr-comments.prompt.md create mode 100644 .github/prompts/new-account-repo-layer.prompt.md create mode 100644 .github/prompts/review-sc-template.prompt.md create mode 100644 .github/skills/account-repo-analysis/SKILL.md diff --git a/.github/agents/implementation.agent.md b/.github/agents/implementation.agent.md new file mode 100644 index 0000000..9b824e3 --- /dev/null +++ b/.github/agents/implementation.agent.md @@ -0,0 +1,105 @@ +--- +name: implementation +description: Full implementation agent. Writes code, creates files, runs tests, and commits changes based on an approved plan. +tools: + - read + - search + - fetch + - edit + - create + - delete + - terminal + - run +model: Claude Sonnet 4.6 (copilot) +handoffs: + - label: "πŸ” Review Changes" + agent: reviewer + prompt: "Implementation is complete. Please review the changes for correctness, security, and adherence to project conventions." + send: false + - label: "πŸ“‹ Back to Planning" + agent: planner + prompt: "Something needs rethinking. Please re-examine the approach and update the plan." + send: false +--- + +# Implementation Agent + +You are a senior Python and Terraform engineer implementing AWS serverless +automation at the US Census Bureau on GovCloud infrastructure. + +## Your Role + +Implement changes based on an approved plan. Write code, create/edit files, +run commands to verify, and commit results. + +## Before Writing Any Code + +1. Read `.github/copilot-instructions.md` for all conventions and constraints +2. Read the plan from the planner agent (or ask for it if none was provided) +3. Read the specific files you will modify β€” never guess at their contents + +## Implementation Standards + +### Python +- Use **Pydantic v2** for all data models and input validation +- Use **`rich`** for terminal output in scripts +- Use **`typer`** for CLI argument parsing +- Use **`httpx`** for HTTP calls (not `requests`) +- Use **`boto3`** for AWS API calls +- Always annotate function signatures with types +- Docstrings for all public functions +- Handle CFN Custom Resource lifecycle: CREATE, UPDATE, DELETE events +- Always send `cfn-response` even on exceptions (use try/finally) + +### AWS / Lambda +```python +import os +AWS_DEFAULT_REGION = "us-gov-west-1" # always set this +VERIFY_SSL = os.environ.get("VERIFY_SSL", "false").lower() != "false" +``` + +- Fetch secrets from Secrets Manager β€” never hardcode tokens +- Lambda environment variables use SCREAMING_SNAKE_CASE +- CFN Properties arrive as `snake_case` β€” use Pydantic to parse them + +### Terraform / HCL +- Use `tf` alias, never `terraform` directly +- Module sources: HTTPS only (`git::https://...`) +- ARNs: `data.aws_partition.current.partition` not hardcoded `aws-us-gov` +- GitHub provider: `insecure = true` + +### Git / GitHub Actions +```bash +# Always specify GHE host +GH_HOST=github.e.it.census.gov gh pr create ... +GH_HOST=github.e.it.census.gov gh api repos/{org}/{repo}/dispatches ... +``` + +- Commit messages: `{type}: {description}` (fix/feat/chore/docs/refactor) +- PR description should include: Summary, Root Cause (if fix), Changes table, Related links + +## Testing Before Committing + +For Lambda Python changes: +```bash +cd /home/a/arnol377/git/lambda-template-repo-generator +python -m pytest tests/ -v 2>&1 +``` + +For SC template changes, verify `${AWS::Partition}` is used everywhere: +```bash +grep -r "aws-us-gov" templates/ && echo "FOUND hardcoded partition!" || echo "OK" +``` + +For Terraform changes: +```bash +tf validate +tf plan +``` + +## After Implementation + +1. Run the relevant tests +2. If tests pass, commit with a clear message +3. Push to the feature branch (create one if working from main) +4. Use the "Review Changes" handoff to hand off to the reviewer agent diff --git a/.github/agents/planner.agent.md b/.github/agents/planner.agent.md new file mode 100644 index 0000000..44e1f00 --- /dev/null +++ b/.github/agents/planner.agent.md @@ -0,0 +1,75 @@ +--- +name: planner +description: Read-only planning agent. Analyzes the codebase, designs interfaces, and produces detailed implementation plans without making any changes. +tools: + - read + - search + - fetch + - web +model: Claude Sonnet 4.6 (copilot) +handoffs: + - label: "β–Ά Start Implementation" + agent: implementation + prompt: "The plan above has been reviewed and approved. Please implement it now, following the plan step by step." + send: false +--- + +# Planning Agent + +You are a senior solutions architect specializing in AWS serverless automation, +GitHub Enterprise integrations, and Terraform infrastructure-as-code. + +## Your Role + +Gather context, analyze existing code, and produce a detailed implementation plan +before any code is written. You MUST NOT create, edit, or delete any files. + +## Context to Always Gather First + +Before planning anything, read the following to orient yourself: + +1. `.github/copilot-instructions.md` in this repo β€” architecture, conventions, constraints +2. `design-docs/README.md` β€” current design decisions and flow +3. For Lambda work: `lambda-template-repo-generator/template_automation/app.py` +4. For SC templates: `terraform-service-catalog-census/templates/products/` and + `lambda-template-repo-generator/service-catalog/product-template.yaml` +5. For account repo structure: review a sample from + `account-repos/229685449397-csvd-dev-gov/` + +## Planning Output Format + +Produce your plan in this structure: + +``` +## Problem Statement +[One paragraph describing what we're solving and why] + +## Current State +[Bullet list of how things work today β€” be specific with file names and function names] + +## Proposed Changes +[Numbered list of changes, each with: what changes, which file, why] + +## Interface Contracts +[Any new API shapes, event schemas, env vars, or IAM permissions needed] + +## Step-by-Step Implementation Order +[Numbered steps in the order a developer should execute them] + +## Testing Plan +[How to verify each change works before moving on] + +## Risk & Rollback +[What could go wrong and how to recover] +``` + +## Key Architecture Points to Consider + +- Lambda runs in `csvd-dev` (229685449397, us-gov-west-1), invoked cross-account +- GitHub Enterprise is at `github.e.it.census.gov` β€” TLS requires `insecure=true` +- SC product parameters must be `snake_case` β€” PascalCase normalizer breaks acronyms +- Always `${AWS::Partition}` not hardcoded `aws-us-gov` in ARNs +- `vpc_name` (string), never `vpc_id` +- GHA `repository_dispatch` requires an installation token (`ghs_`) or PAT (`ghp_`) + with `repo` scope and `workflow` scope +- Lambda timeout is 900s β€” GHA polling must complete within that window diff --git a/.github/agents/reviewer.agent.md b/.github/agents/reviewer.agent.md new file mode 100644 index 0000000..6f81c24 --- /dev/null +++ b/.github/agents/reviewer.agent.md @@ -0,0 +1,82 @@ +--- +name: reviewer +description: Code review agent. Reviews changes for correctness, security, Census conventions, and GovCloud compliance. Read-only β€” suggests changes as comments, does not edit files. +tools: + - read + - search + - fetch +model: Claude Sonnet 4.6 (copilot) +handoffs: + - label: "πŸ”§ Fix Issues" + agent: implementation + prompt: "The review found issues. Please address each comment in the review above." + send: false + - label: "βœ… Approve & Summarize PR" + agent: implementation + prompt: "The review is clean. Please write a PR description summarizing the changes, update the branch if needed, and prepare it for merge." + send: false +--- + +# Code Reviewer Agent + +You are a security-conscious senior engineer reviewing code for a GovCloud +AWS Lambda automation system at the US Census Bureau. + +## Your Role + +Review changes for correctness, security, and convention adherence. +You MUST NOT edit files β€” only report findings. + +## Review Checklist + +### Security +- [ ] No hardcoded credentials, tokens, or secrets +- [ ] No hardcoded account IDs (use `${AWS::AccountId}` or variables) +- [ ] No hardcoded `aws-us-gov` partition (must use `${AWS::Partition}`) +- [ ] IAM policies follow least-privilege +- [ ] Lambda functions send `cfn-response` in all code paths (including exceptions) +- [ ] All GHE API calls use `insecure=True` or equivalent (Census CA cert not in bundle) + +### Python Quality +- [ ] Pydantic v2 used for all input models +- [ ] Type annotations on all function signatures +- [ ] `AWS_DEFAULT_REGION` set before any boto3 calls +- [ ] Exceptions caught and re-raised with context (not swallowed) +- [ ] No `requests` library β€” use `httpx` + +### CloudFormation / Service Catalog +- [ ] SC product parameters passed as `snake_case` Properties +- [ ] `aws_account_id` and `aws_region` auto-resolved via `!Sub` (not user form fields) +- [ ] `vpc_name` used (not `vpc_id`) +- [ ] ServiceToken ARN uses `${AWS::Partition}` + +### Terraform / HCL +- [ ] Module sources use HTTPS (not SSH) +- [ ] GitHub provider has `insecure = true` +- [ ] `tf` alias used (not `terraform`) +- [ ] `remote_state.yml` has all required fields + +### General +- [ ] Commit message follows `{type}: {description}` convention +- [ ] No temp files written to `/tmp` (use `~/tmp`) +- [ ] No `terraform` direct calls (use `tf` alias) + +## Review Output Format + +``` +## Review Summary +Status: βœ… APPROVED | ⚠️ MINOR ISSUES | ❌ REQUIRES CHANGES + +## Issues Found +### πŸ”΄ Blocking +- [file:line] Description of issue and how to fix it + +### 🟑 Warning +- [file:line] Description of concern + +### 🟒 Suggestions (optional) +- [file:line] Optional improvement + +## What Looks Good +- Bullet list of things done well +``` diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md new file mode 100644 index 0000000..d87774f --- /dev/null +++ b/.github/copilot-instructions.md @@ -0,0 +1,172 @@ +# GitHub Copilot Instructions β€” sc-lambda-ghactions + +## Project Purpose + +This repo designs and implements the **next-generation Service Catalog automation** +system at Census. The architecture is: + +``` +SC Console (user fills product form) + └─> CFN Stack (Custom::* resource) + └─> Lambda (centralized in csvd-dev, 229685449397, us-gov-west-1) + β”œβ”€> Validates inputs (Pydantic v2 models) + β”œβ”€> Fetches GHE token from Secrets Manager + β”œβ”€> POSTs repository_dispatch to target repo on GHE + └─> Polls GHA workflow run β†’ returns repo URL + PR URL to CFN + +GitHub Enterprise (github.e.it.census.gov) + └─> GHA workflow (repository_dispatch event) + β”œβ”€> Clones the target account repo + β”œβ”€> Renders HCL/YAML files from templates + └─> Commits + opens PR (repo-init β†’ main) +``` + +This replaces the current CodeBuild + terraform-eks-deployment path with a +GHA-native approach that keeps workflow logic inside the target repos. + +--- + +## Current Production System (for reference / migration context) + +The live system in `lambda-template-repo-generator` works as follows: +1. CFN Custom Resource calls Lambda `eks-terragrunt-repo-gen-template-automation` +2. Lambda triggers CodeBuild project `eks-terragrunt-repo-creator` +3. CodeBuild runs `terraform apply` inside `terraform-eks-deployment` workspace +4. Terraform uses `CSVD/terraform-github-repo` module to create repo, write HCL, open PR +5. Lambda polls CodeBuild β†’ fetches PR URL β†’ signals CFN SUCCESS/FAILED + +--- + +## Account Repo Structure (what GHA operates on) + +Every account repo follows this strict layout. GHA will clone the repo and +write or modify files within this structure. + +``` +{account_id}-{team}-{workload}-{tier}-{partition}/ +β”œβ”€β”€ .tf-control # tf-run toolchain version pin +β”œβ”€β”€ .tf-control.tfrc # Terraform provider cache config +β”œβ”€β”€ .gitsecret/ # GPG-encrypted secrets +β”œβ”€β”€ region.tf # locals { region = var.region } +β”œβ”€β”€ outputs.common.tf # shared output declarations +β”œβ”€β”€ credentials.d/ # per-region AWS credentials tf files +β”‚ β”œβ”€β”€ us-gov-east-1.credentials.tf +β”‚ └── us-gov-west-1.credentials.tf +β”œβ”€β”€ variables.d/ # common variable definitions +β”‚ β”œβ”€β”€ variables.common.tf +β”‚ β”œβ”€β”€ variables.tfstate.tf +β”‚ └── {region}.variables.common.auto.tfvars +β”œβ”€β”€ includes.d/ # tag variable files +β”‚ β”œβ”€β”€ variables.account_tags.tf +β”‚ β”œβ”€β”€ variables.application_tags.tf +β”‚ └── variables.infrastructure_tags.tf +β”œβ”€β”€ provider_configs.d/ # optional extra providers (DNS, LDAP, Infoblox) +β”œβ”€β”€ init/ # bootstrap: git-secret, gpg-setup +β”œβ”€β”€ common/ # IAM, SAML, LDAP β€” split by east/ west/ apps/ +β”œβ”€β”€ infrastructure/ # CloudTrail, Config, KMS, S3 logs β€” INF.*.tf files +└── vpc/ # VPC definitions +``` + +Sub-account repos (`_apps-{team}-{purpose}`) are minimal versions with no +`applications/` dir, no `init/`, and a `SUBMODULE` marker file instead of +`.initialized`. + +Each layer (`common/`, `infrastructure/`, `vpc/`) contains regional +subdirectories (`east/`, `west/`, `apps/`) each with: +- `remote_state.backend.tf` β€” S3 backend +- `remote_state.{layer}.tf` β€” symlink to `.s3`, `.local`, or `.none` variant +- `remote_state.yml` β€” metadata for `tf-directory-setup.py` +- `outputs.common.tf`, `variables.common.tf`, `variables.tfstate.tf` +- `tf-run.data` β€” step list for the `tf-run` orchestrator + +### `remote_state.yml` schema (required by tf-directory-setup.py) +```yaml +directory: "infrastructure/west" +profile: "{account_id}-{account_alias}" +bucket: "inf-tfstate-{account_id}" +bucket_region: "us-gov-east-1" +region: "us-gov-west-1" +regions: ["us-gov-west-1"] +account_id: "{account_id}" +account_alias: "{account_alias}" +aws_environment: "gov" # or "ew" for commercial +``` + +--- + +## The `tf-run` Toolchain + +`tf-run` and `tf-{action}` (plan/apply/destroy/etc.) are all symlinks to +scripts in `/apps/terraform/bin/`. Key behavior: + +- **`tf-{action}`** (`tf-control.sh`) β€” wraps `terraform {action}` with: + - Reads `.tf-control` at git root β†’ `TFCOMMAND=terraform_current` (or pinned version) + - Reads `.tf-control.tfrc` β†’ `TF_CLI_CONFIG_FILE` (provider cache) + - Auto-injects Census proxy for `tf-init` (`https_proxy`) + - Logs all output to `logs/{action}.{timestamp}.log` + - `TFARGS` env var for extra args on apply/destroy only + - `TFNOLOG=1` disables tee (needed for `tf-state pull`) + +- **`tf-run`** (`tf-run.sh`) β€” multi-step orchestrator reading `tf-run.data`: + - Steps through entries sequentially, prompts `continue [y/n]` between steps + - Can start/stop at step number or `tag:name` + - Key `tf-run.data` directives: + - `COMMAND ` β€” run any shell command + - `LINKTOP ` β€” symlink from git root to current dir + - `LINK ` β€” find file in parent dirs, symlink it here + - `REMOTE-STATE` β€” derive `remote_state.yml` from parent + subdirectory + - `ALL` β€” terraform apply with no `-target=` + - `POLICY` β€” auto-target all `aws_iam_policy` resources + - `TAG name` / `STOP` / `PAUSE` / `COMMENT` β€” flow control + +- **`tf-directory-setup.py`** β€” reads `remote_state.yml`, generates: + - `remote_state.backend.tf` + - `remote_state.{dir}.tf.s3` / `.local` / `.none` + - Prints `ln -sf` commands to activate the correct variant + +--- + +## Python & CLI Standards + +| Purpose | Library | +|---------|---------| +| Data validation / config models | `pydantic` (v2) | +| Rich terminal output | `rich` | +| CLI arg parsing | `typer` | +| AWS API | `boto3` | +| HTTP | `httpx` | +| YAML | `pyyaml` | + +- Always `export AWS_DEFAULT_REGION=us-gov-west-1` before AWS CLI / boto3 +- GitHub Enterprise host: `github.e.it.census.gov` +- GHE org: `SCT-Engineering` (production SC products), `arnol377` (personal dev) +- Use `GH_HOST=github.e.it.census.gov gh ...` for all `gh` CLI commands +- TLS verification is disabled for GHE (`VERIFY_SSL=false` / `insecure=true`) + because the Census CA cert is not in standard bundles + +--- + +## CFN / Lambda Conventions + +- Always use `${AWS::Partition}` not hardcoded `aws-us-gov` in ARNs +- SC product form parameters β†’ Lambda in `snake_case` Properties (not PascalCase) + β€” the Lambda normalizer mishandles acronyms in PascalCase +- `aws_account_id` and `aws_region` are auto-resolved via `!Sub` in CFN; + do NOT add them as user-facing SC form parameters +- Lambda ServiceToken: `arn:${AWS::Partition}:lambda:${AWS::Region}:${AWS::AccountId}:function:{name}` +- Lambda timeout must be β‰₯ CodeBuild/GHA poll window (currently 900s) + +--- + +## What NOT to Do + +- ❌ Do not hardcode `aws-us-gov` β€” use `${AWS::Partition}` +- ❌ Do not use SSH git module sources (`git::ssh://`) β€” Census proxy blocks SSH +- ❌ Do not write temp files to `/tmp` β€” use `~/tmp` +- ❌ Do not use `terraform` directly β€” use `tf` alias (`tf plan`, `tf apply`) +- ❌ Do not run AWS CLI/boto3 without `export AWS_DEFAULT_REGION=us-gov-west-1` +- ❌ Do not add `vpc_id` β€” field is `vpc_name` +- ❌ Do not use `HappyPathway/terraform-github-repo` public module +- βœ… DO use `CSVD/terraform-github-repo` (https://github.e.it.census.gov/CSVD/terraform-github-repo) +- βœ… DO use `gh` CLI for PR management +- βœ… DO use `GH_HOST=github.e.it.census.gov` for all GHE commands diff --git a/.github/instructions/terraform.instructions.md b/.github/instructions/terraform.instructions.md new file mode 100644 index 0000000..ac3b3f0 --- /dev/null +++ b/.github/instructions/terraform.instructions.md @@ -0,0 +1,126 @@ +--- +name: 'Terraform & HCL Standards' +description: 'Conventions for all Terraform, HCL, and account repo files' +applyTo: '**/*.tf,**/*.hcl,**/remote_state.yml,**/tf-run.data' +--- + +# Terraform & HCL Standards + +## Account Repo Layer Conventions + +### File naming inside a layer directory (`common/`, `infrastructure/`, `vpc/`) +- `INF.{description}.tf` β€” infrastructure-managed resource files +- `remote_state.backend.tf` β€” S3 backend configuration (generated by tf-directory-setup.py) +- `remote_state.{dir_replaced}.tf` β€” symlink to `.s3`, `.local`, or `.none` variant +- `remote_state.{dir_replaced}.tf.s3` β€” reads state from S3 +- `remote_state.{dir_replaced}.tf.local` β€” reads state from local file (for testing) +- `remote_state.{dir_replaced}.tf.none` β€” empty file (initial bootstrap state) +- `{region}.credentials.tf` β€” provider auth scoped to a region +- `{region}.variables.common.auto.tfvars` β€” region-scoped variable values +- `*.tf.off` β€” disabled resource (excluded from terraform) +- `*.tf.completed` β€” one-time init file, already applied, no longer needed + +### `remote_state.yml` β€” required fields +```yaml +directory: "infrastructure/west" # path within repo; determines symlink name +profile: "{account_id}-{account_alias}" +bucket: "inf-tfstate-{account_id}" +bucket_region: "us-gov-east-1" +region: "us-gov-west-1" +regions: ["us-gov-west-1"] +account_id: "{account_id}" +account_alias: "{account_alias}" +aws_environment: "gov" # "gov" for GovCloud, "ew" for commercial +``` + +### `tf-run.data` step file +``` +VERSION 2.0.2 +REMOTE-STATE +COMMAND tf-directory-setup.py -l none -f +COMMAND setup-new-directory.sh +LINKTOP includes.d/variables.account_tags.tf +LINKTOP credentials.d/us-gov-west-1.credentials.tf +LINKTOP variables.d/us-gov-west-1.variables.common.auto.tfvars +COMMAND tf-init -upgrade +ALL +COMMAND tf-directory-setup.py -l s3 +``` + +## ARN Construction + +Always use `${AWS::Partition}` (CloudFormation) or `data.aws_partition.current.partition` +(Terraform) β€” never hardcode `aws-us-gov` or `aws`. + +```hcl +# βœ… Correct +resource "aws_iam_policy" "example" { + policy = jsonencode({ + Statement = [{ + Resource = "arn:${data.aws_partition.current.partition}:s3:::my-bucket" + }] + }) +} + +data "aws_partition" "current" {} +``` + +```yaml +# βœ… Correct in CloudFormation +ServiceToken: !Sub "arn:${AWS::Partition}:lambda:${AWS::Region}:${AWS::AccountId}:function:my-function" + +# ❌ Wrong +ServiceToken: !Sub "arn:aws-us-gov:lambda:${AWS::Region}:${AWS::AccountId}:function:my-function" +``` + +## Terraform Module Sources + +Always use HTTPS sources β€” Census proxy blocks SSH host key exchange. + +```hcl +# βœ… Correct +module "repo" { + source = "git::https://github.e.it.census.gov/CSVD/terraform-github-repo.git?ref=v1.2.0" +} + +# ❌ Wrong β€” SSH blocked by Census proxy +module "repo" { + source = "git::ssh://git@github.e.it.census.gov/CSVD/terraform-github-repo.git" +} +``` + +## GitHub Provider (GHE) + +```hcl +provider "github" { + base_url = "https://github.e.it.census.gov/" + token = var.github_token + insecure = true # required β€” Census CA cert not in standard bundles +} +``` + +## Terraform Commands + +Always use the `tf` alias, not `terraform` directly: +```bash +tf init +tf plan +tf apply +tf destroy +tf-run plan # multi-step orchestrator +tf-run apply 3 # start at step 3 +``` + +## Variable Naming in SC Product Templates + +Pass all parameters as `snake_case` in CFN Properties to avoid the +Lambda PascalCaseβ†’snake_case normalizer mishandling acronyms: + +```yaml +Properties: + ServiceToken: !Sub "arn:${AWS::Partition}:lambda:..." + cluster_name: !Ref ClusterName # βœ… snake_case + account_name: !Ref AccountName # βœ… snake_case + vpc_name: !Ref VpcName # βœ… vpc_name, NOT vpc_id + aws_account_id: !Sub "${AWS::AccountId}" # βœ… auto-resolved, not user form field +``` diff --git a/.github/prompts/analyze-pr-comments.prompt.md b/.github/prompts/analyze-pr-comments.prompt.md new file mode 100644 index 0000000..9febfdc --- /dev/null +++ b/.github/prompts/analyze-pr-comments.prompt.md @@ -0,0 +1,66 @@ +--- +name: analyze-pr-comments +description: Fetch all review comments on a GHE pull request, summarize the issues, and propose or implement fixes. +argument-hint: "[repo org/name] [PR number]" +agent: implementation +tools: + - terminal + - read + - edit +--- + +# Analyze and Address PR Review Comments + +Fetch and address the review comments on PR `${input:pr_number}` in +`${input:repo}` on GitHub Enterprise. + +## Step 1 β€” Fetch Comments + +```bash +GH_HOST=github.e.it.census.gov gh pr view ${input:pr_number} \ + --repo ${input:repo} --comments 2>&1 + +GH_HOST=github.e.it.census.gov gh api \ + repos/${input:repo}/pulls/${input:pr_number}/comments \ + | python3 -c " +import json, sys +data = json.load(sys.stdin) +for c in data: + print(f'=== {c[\"user\"][\"login\"]} on {c[\"path\"]} (line {c.get(\"line\",\"?\")}) ===') + print(c['body']) + print() +" +``` + +## Step 2 β€” Read the Affected Files + +For each comment, read the relevant file at the flagged line. + +## Step 3 β€” Apply Fixes + +For each comment: +1. Understand what change is requested +2. Apply the fix to the file +3. Note the fix in a summary + +## Step 4 β€” Summarize + +After fixing all comments, output: +``` +## PR ${input:pr_number} β€” Fix Summary + +### Changes Made +| File | Line | Comment | Fix Applied | +|------|------|---------|-------------| +| ... | ... | ... | ... | + +### Remaining (need human decision) +- [any comments that need clarification or design input] +``` + +Then commit all changes with: +```bash +git add -A +git commit -m "fix: address PR review comments from {reviewer}" +GH_HOST=github.e.it.census.gov git push +``` diff --git a/.github/prompts/new-account-repo-layer.prompt.md b/.github/prompts/new-account-repo-layer.prompt.md new file mode 100644 index 0000000..6fa4491 --- /dev/null +++ b/.github/prompts/new-account-repo-layer.prompt.md @@ -0,0 +1,69 @@ +--- +name: new-account-repo-layer +description: Scaffold a new layer directory (common, infrastructure, or vpc) inside an account repo, complete with remote_state.yml, tf-run.data, and boilerplate symlink stubs. +argument-hint: "[layer: common|infrastructure|vpc] [region: us-gov-west-1|us-gov-east-1] [account_id] [account_alias]" +agent: implementation +tools: + - read + - create + - edit + - terminal +--- + +# Scaffold a New Account Repo Layer + +Generate a new `${input:layer:common|infrastructure|vpc}` layer for the +`${input:region:us-gov-west-1}` region in account +`${input:account_id}` (`${input:account_alias}`). + +## What to Create + +### 1. `remote_state.yml` +Use this schema β€” fill in values from the inputs above: +```yaml +directory: "${input:layer}/${input:region_short}" +profile: "${input:account_id}-${input:account_alias}" +bucket: "inf-tfstate-${input:account_id}" +bucket_region: "us-gov-east-1" +region: "${input:region}" +regions: ["${input:region}"] +account_id: "${input:account_id}" +account_alias: "${input:account_alias}" +aws_environment: "gov" +``` +where `region_short` = `west` for `us-gov-west-1`, `east` for `us-gov-east-1`. + +### 2. `tf-run.data` +``` +VERSION 2.0.2 +REMOTE-STATE +COMMAND tf-directory-setup.py -l none -f +COMMAND setup-new-directory.sh +LINKTOP includes.d/variables.account_tags.tf +LINKTOP includes.d/variables.application_tags.tf +LINKTOP includes.d/variables.infrastructure_tags.tf +LINKTOP credentials.d/${input:region}.credentials.tf +LINKTOP variables.d/${input:region}.variables.common.auto.tfvars +COMMAND tf-init -upgrade +ALL +COMMAND tf-directory-setup.py -l s3 +``` + +### 3. Stub files (create as empty placeholders) +- `region.tf` β€” `locals { region = var.region }` +- `outputs.common.tf` β€” empty +- `variables.common.tf` β€” empty +- `variables.tfstate.tf` β€” empty + +### 4. Print instructions +After creating the files, print the commands the user needs to run: +```bash +cd {layer}/{region_short} +tf-run init # fetches .tf-control, base tf-run.data +tf-run apply # steps through tf-run.data interactively +``` + +## References +- Account repo structure: see `.github/copilot-instructions.md` β†’ "Account Repo Structure" +- `tf-run.data` directives: see `.github/skills/account-repo-analysis/SKILL.md` +- Sample: `account-repos/229685449397-csvd-dev-gov/infrastructure/west/` diff --git a/.github/prompts/review-sc-template.prompt.md b/.github/prompts/review-sc-template.prompt.md new file mode 100644 index 0000000..83bb5f2 --- /dev/null +++ b/.github/prompts/review-sc-template.prompt.md @@ -0,0 +1,64 @@ +--- +name: review-sc-template +description: Review a Service Catalog CloudFormation product template for compliance with Census Lambda interface conventions. +argument-hint: "[path to template yaml file]" +agent: reviewer +tools: + - read + - search +--- + +# Review SC Product Template + +Review the Service Catalog CloudFormation template at `${input:template_path}` and +check it against the Census Lambda interface conventions. + +## Checklist to Verify + +### ARN Partition +- All ARNs must use `${AWS::Partition}` not `aws-us-gov` +- Run: search for `aws-us-gov` in the template + +### Parameter Passing to Lambda +- All parameters in the `Properties:` block of the Custom Resource must be `snake_case` +- `aws_account_id` and `aws_region` should be auto-resolved via `!Sub "${AWS::AccountId}"` + and `!Sub "${AWS::Region}"` β€” they should NOT be user-facing form parameters +- The field must be `vpc_name` (String), NOT `vpc_id` (AWS::EC2::VPC::Id) + +### ServiceToken +- Must use `arn:${AWS::Partition}:lambda:...` format +- Must NOT use a hardcoded ARN or a parameter for the Lambda ARN + +### Parameters Section +- No `LambdaFunctionArn` parameter +- No `AWSAccountId` parameter (auto from `${AWS::AccountId}`) +- No `AwsRegion` parameter (auto from `${AWS::Region}`) + +### Required EKS Parameters (for eks-terragrunt-repo product) +All of these must be present as Properties on the Custom Resource: +- `cluster_name` +- `account_name` +- `aws_account_id` +- `vpc_name` +- `vpc_domain_name` +- `organization_path` +- `finops_project_name` +- `finops_project_number` + +## Compare Against Reference +Read `lambda-template-repo-generator/service-catalog/product-template.yaml` as the +canonical reference and note any differences. + +## Output Format +``` +## Template Review: {filename} + +### βœ… Passing +- [list what's correct] + +### ❌ Issues +- Line {N}: [issue description and fix] + +### Diff from Canonical Template +[Any meaningful differences from lambda-template-repo-generator/service-catalog/product-template.yaml] +``` diff --git a/.github/skills/account-repo-analysis/SKILL.md b/.github/skills/account-repo-analysis/SKILL.md new file mode 100644 index 0000000..5bd8b2b --- /dev/null +++ b/.github/skills/account-repo-analysis/SKILL.md @@ -0,0 +1,180 @@ +--- +name: account-repo-analysis +description: > + Analyze or scaffold Census AWS account repos. Understands the 3-layer directory + structure (common/infrastructure/vpc), regional splits (east/west), tf-run DSL + directives, remote_state.yml schema, and the tf-run toolchain scripts. Use this + skill when asked to analyze an account repo, explain tf-run steps, scaffold a new + layer, or debug tf-directory-setup.py failures. +--- + +# Account Repo Analysis Skill + +## Account Repo Overview + +Census AWS account repos live in `~/git/account-repos/` (or `~/git/`) and follow +the naming convention `{account_id}-{team}-{workload}-{tier}-{partition}`: + +- `tier`: `prod`, `dev`, `stage`, `test` +- `partition`: `gov` = us-gov-west/east, `ew` = us-east/west commercial +- Sub-repos use `_apps-{team}-{purpose}` suffix and contain a `SUBMODULE` file + +### 3-Layer Structure + +Every account repo contains three Terraform layers β€” each split into regional subdirs: + +``` +{account_id}-{alias}/ +β”œβ”€β”€ .tf-control # tf-run toolchain version pin +β”œβ”€β”€ .tf-control.tfrc # Terraform registry mirror config (Census proxy) +β”œβ”€β”€ credentials.d/ +β”‚ └── {region}.credentials.tf # AWS provider auth per region +β”œβ”€β”€ variables.d/ +β”‚ β”œβ”€β”€ variables.common.tf # shared input variables +β”‚ β”œβ”€β”€ variables.tfstate.tf # remote state S3 variables +β”‚ └── {region}.auto.tfvars # region-specific values +β”œβ”€β”€ includes.d/ +β”‚ β”œβ”€β”€ variables.account_tags.tf +β”‚ β”œβ”€β”€ variables.application_tags.tf +β”‚ └── variables.infrastructure_tags.tf +β”œβ”€β”€ provider_configs.d/ +β”‚ β”œβ”€β”€ provider.{name}.tf +β”‚ β”œβ”€β”€ provider.{name}.variables.tf +β”‚ └── provider.{name}.auto.tfvars.secret +β”œβ”€β”€ common/ +β”‚ β”œβ”€β”€ east/ # IAM, SAML, LDAP +β”‚ └── west/ +β”œβ”€β”€ infrastructure/ +β”‚ β”œβ”€β”€ east/ # CloudTrail, Config, KMS, S3 logs +β”‚ └── west/ +└── vpc/ + β”œβ”€β”€ east/ # VPC, subnets, routing + └── west/ +``` + +### Per-Regional-Subdir Contents + +Each `{layer}/{region}/` contains: +- `remote_state.yml` β€” backend configuration schema +- `remote_state.backend.tf` β€” S3 backend (generated by tf-directory-setup.py) +- `remote_state.{dir_replaced}.tf` β†’ symlink to `.s3`, `.local`, or `.none` +- `remote_state.{dir_replaced}.tf.s3` β€” reads cross-layer state from S3 +- `remote_state.{dir_replaced}.tf.local` β€” reads state from local file +- `remote_state.{dir_replaced}.tf.none` β€” empty stub (bootstrap only) +- `outputs.common.tf` β€” shared outputs +- `variables.common.tf` β€” shared variables +- `variables.tfstate.tf` β€” remote state variable declarations +- `{region}.credentials.tf` β€” symlink to credentials.d/ +- `{region}.variables.common.auto.tfvars` β€” auto-loaded tfvars +- `tf-run.data` β€” ordered step file for tf-run.sh +- `INF.*.tf` β€” infrastructure managed resource files +- `setup/` β€” one-time init scripts + +--- + +## `remote_state.yml` Schema + +```yaml +directory: "infrastructure/west" # layer/region_short path +profile: "{account_id}-{account_alias}" +bucket: "inf-tfstate-{account_id}" +bucket_region: "us-gov-east-1" # state bucket is always east +region: "us-gov-west-1" # working region +regions: ["us-gov-west-1"] +account_id: "229685449397" +account_alias: "csvd-dev" +aws_environment: "gov" # "gov" = GovCloud, "ew" = commercial +``` + +Derived automatically by `tf-directory-setup.py`: +- `remote_state.backend.tf` β€” S3 backend block +- Symlink `remote_state.{directory_with_slashes_replaced}.tf` β†’ `.s3`/`.local`/`.none` + +--- + +## tf-run Toolchain + +### tf-control.sh (aka `tf-{action}` symlinks) + +Wraps `terraform {action}` with: +- Version resolution from `.tf-control` file in the dir or parents +- Census proxy injection during `init` only +- Log output to `logs/{action}.{timestamp}.log` +- Color and pager control via env vars + +Key env vars: +| Var | Effect | +|-----|--------| +| `TFCOMMAND` | Override terraform binary path | +| `TFARGS` | Extra args appended to terraform | +| `TFNOLOG` | Skip log file writing | +| `TFNOPROXY` | Skip proxy injection | +| `TFNOCOLOR` | Disable ANSI colors | +| `TFLESS` | Pipe output through less | + +### tf-run.sh (tf-run command) + +Reads `tf-run.data` and executes steps in order with an interactive `y/n` prompt +between each. Start at a specific step: `tf-run apply 3` or `tf-run apply tag:init`. + +### tf-run.data DSL Directives + +| Directive | Description | +|-----------|-------------| +| `VERSION x.y.z` | Version comment β€” no action | +| `TAG name` | Named label β€” used as start-point | +| `COMMENT text` | Inline comment β€” no action | +| `COMMAND ` | Run arbitrary shell command | +| `LINKTOP ` | Symlink git-root-relative path to cwd | +| `LINK ` | Walk parents to find file, symlink here | +| `REMOTE-STATE` | Parse `remote_state.yml`, derive vars for setup | +| `ALL` | `terraform apply` with no `-target` | +| `POLICY` | Auto-build `-target=aws_iam_policy.*` list and apply | +| `BACKUP-STATE` | Copy `terraform.tfstate` to backup file | +| `PAUSE` | Ask user to confirm before continuing | +| `STOP` | Halt execution (used for disabled steps) | +| `CHECK` | Dry-run check β€” no apply | +| `module.name` | `terraform apply -target=module.name` | +| `resource.type.name` | `terraform apply -target=resource.type.name` | + +### tf-directory-setup.py + +Reads `remote_state.yml` and generates: +1. `remote_state.backend.tf` β€” S3 backend config +2. `remote_state.{dir}.tf.s3` β€” data source reading state from S3 +3. `remote_state.{dir}.tf.local` β€” data source reading local state file +4. `remote_state.{dir}.tf.none` β€” empty (initial bootstrap) +5. Symlink `remote_state.{dir}.tf` β†’ active variant + +Usage: +```bash +tf-directory-setup.py -l none -f # initialize with empty state (bootstrap) +tf-directory-setup.py -l s3 # switch to S3 remote state (post-apply) +tf-directory-setup.py -l local # switch to local state file (testing) +``` + +--- + +## Analysis Workflow + +When asked to analyze an account repo, follow these steps: + +1. **Identify the repo type**: full account repo vs. `_apps-*` sub-repo (look for `SUBMODULE` file) +2. **Read `.tf-control`**: determines which terraform binary version is used +3. **Read each layer's `remote_state.yml`**: reveals account_id, region, partition +4. **Inspect `tf-run.data`**: explains the provisioning order and any custom steps +5. **Check symlinks**: verify `remote_state.{dir}.tf` points to `.s3` (should be s3 post-bootstrap) +6. **Review `INF.*.tf` files**: list the resources under management +7. **Check `.tf-control.tfrc`**: shows the Terraform registry mirror URL (Census proxy) + +--- + +## Common Issues + +| Symptom | Likely Cause | Fix | +|---------|--------------|-----| +| `tf-directory-setup.py: no such file or directory` | Not in a layer subdir | `cd infrastructure/west` first | +| State backend shows wrong region | `bucket_region` in `remote_state.yml` is wrong | Fix to `us-gov-east-1` | +| `Error: SSH authentication required` | Module source uses `git::ssh://` | Change to `git::https://` | +| Provider download timeout | Proxy not set, or `NO_PROXY` missing GHE host | Check `.tf-control.tfrc` | +| `registry.terraform.io connection refused` | Missing proxy config | Set `HTTPS_PROXY=http://proxy.tco.census.gov:3128` and `NO_PROXY` to include GHE host |