diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 46bc2db..898c348 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -19,23 +19,41 @@ All actual repo creation runs inside **CodeBuild** via the `terraform-eks-deploy --- -## Architecture: Lambda as Thin Orchestrator over CodeBuild + Terraform +## Architecture: Centralized Lambda in csvd-dev, Cross-Account Invocation + +The SC product is shared to **multiple accounts** via the portfolio, but all compute runs +**centrally in csvd-dev** (`229685449397`). When a user in any account provisions the product, +CloudFormation invokes the Lambda cross-account using the hardcoded `ServiceToken` ARN. +This works because the Lambda has an `aws_lambda_permission` with `principal_org_id`, +allowing CloudFormation in any org account to invoke it. + +The provisioning account only sees a CloudFormation stack with outputs (repo URL, PR URL). +It never needs the Lambda, CodeBuild, ECR image, or Secrets Manager secrets locally. ``` -SC Console (user fills form) - → CFN Stack creates Custom::GitHubRepository resource - → CFN calls Lambda (eks-terragrunt-repo-gen-template-automation) via ServiceToken - → Lambda fetches PAT from Secrets Manager (ghe-runner/github-token) - → Lambda starts CodeBuild project (eks-terragrunt-repo-creator) with TF_VAR_* env overrides - → CodeBuild clones terraform-eks-deployment repo from GHE - → CodeBuild runs: terraform init + terraform apply -auto-approve - → Terraform (CSVD/terraform-github-repo module) creates GHE repo + writes HCL files + opens PR - → Lambda polls CodeBuild, then fetches PR URL from GitHub API - → Lambda sends cfn-response SUCCESS with repository_url + pull_request_url - → CFN stack transitions to CREATE_COMPLETE - → SC provisioned product shows as AVAILABLE +Any Account (SC portfolio shared via OU): + SC Console (user fills form) + → CFN Stack creates Custom::GitHubRepository resource + +csvd-dev (229685449397) — all compute here: + → CFN calls Lambda (eks-terragrunt-repo-gen-template-automation) cross-account via ServiceToken + → Lambda fetches PAT from Secrets Manager (ghe-runner/github-token) + → Lambda starts CodeBuild project (eks-terragrunt-repo-creator) with TF_VAR_* env overrides + → CodeBuild clones terraform-eks-deployment repo from GHE + → CodeBuild runs: terraform init + terraform apply -auto-approve + → Terraform (CSVD/terraform-github-repo module) creates GHE repo + writes HCL files + opens PR + → Lambda polls CodeBuild, then fetches PR URL from GitHub API + → Lambda sends cfn-response SUCCESS with repository_url + pull_request_url + +Any Account: + → CFN stack transitions to CREATE_COMPLETE + → SC provisioned product shows as AVAILABLE ``` +**Why centralized?** The Lambda only interacts with GitHub Enterprise and CodeBuild — it +makes no AWS API calls in the provisioner's account. Deploying per-account would add +complexity (ECR replication, per-account CodeBuild, per-account Secrets Manager) with no benefit. + ### CodeBuild Projects There are **two** CodeBuild projects — do not confuse them: @@ -54,7 +72,6 @@ The Lambda triggers **`eks-terragrunt-repo-creator`** at runtime. The **`eks-ter | File | Purpose | |------|--------| | `template_automation/app.py` | Lambda entry point; CFN Custom Resource handler; `start_codebuild_build()` + `poll_codebuild_build()` | -| `template_automation/eks_config.py` | Pydantic models + `is_eks_deployment` check | | `service-catalog/product-template.yaml` | CFN template for the SC product (canonical source) | | `deploy/main.tf` | Terraform: Lambda, CodeBuild project, SC portfolio/product, IAM | | `deploy/variables.tf` | Input variables including `codebuild_project_name`, `codebuild_role_arn` | @@ -126,15 +143,15 @@ Both `service-catalog/product-template.yaml` here and `2-0-0.yaml` in census mus - `TF_GITHUB_TOKEN_SECRET_NAME` holds a **personal access token** (`ghp_` prefix, user `arnol377`). This is passed to CodeBuild and used by the Terraform GitHub provider. -### EKS mode is triggered when all these fields are present in the event: +### Required EKS fields in the CFN event: - `cluster_name` - `account_name` - `aws_account_id` - `vpc_name` - `vpc_domain_name` -If any of these are missing, the Lambda falls back to **generic mode** (writes only `config.json`). -**Do not pass `vpc_id`** — the Lambda model field is `vpc_name` (a string). +The Lambda is EKS-only — there is no generic fallback mode. +**Do not pass `vpc_id`** — the field is `vpc_name` (a string). --- diff --git a/docs/EKS_SC_RESOURCE_INVENTORY.md b/docs/EKS_SC_RESOURCE_INVENTORY.md new file mode 100644 index 0000000..dd7d59f --- /dev/null +++ b/docs/EKS_SC_RESOURCE_INVENTORY.md @@ -0,0 +1,157 @@ +# EKS Service Catalog — Resource Inventory + +Compiled: 2026-04-14 + +This document catalogs all AWS resources involved in the EKS Terragrunt Repository Creator +Service Catalog product — both the resources the census Terragrunt pipeline provisions and +the prerequisite infrastructure deployed via direct Terraform. + +## Cross-Account Architecture + +The SC product is shared to **multiple accounts** via the portfolio (OU-level sharing), but +all compute runs **centrally in csvd-dev** (`229685449397`). + +``` +Provisioner's account csvd-dev (229685449397) +────────────────────── ──────────────────────── +SC Console → CFN Stack ──→ Lambda (cross-account invoke via principal_org_id) + ├─ Secrets Manager (GitHub PAT) + ├─ CodeBuild (terraform-eks-deployment) + │ └─ Terraform → GitHub Enterprise API + └─ cfn-response SUCCESS +CFN outputs (repo + PR) ←── +``` + +The provisioner's account needs only: +- The **SC launch role** (`r-ent-servicecatalog-eksterragrunt-launch-role`) — deployed via StackSet +- Network path to the Lambda ARN (handled by CloudFormation natively) + +It does **not** need the Lambda, CodeBuild, ECR image, or Secrets Manager secrets locally. + +--- + +## 1. `terraform-service-catalog-census` — EKS-specific resources + +Deployed by `non-prod/csvd-dev/west/service-catalog` (Terragrunt), driven by the configuration +files in `non-prod/csvd-dev/west/configurations/portfolios/eks-terragrunt.yaml.tftpl` and +`non-prod/csvd-dev/west/configurations/products/eks-terragrunt-repo/EKS_REPO.yaml.tftpl`. + +### Service Catalog + +| Resource | Detail | +|----------|--------| +| SC Portfolio | `EKS Terragrunt Repository Creator Portfolio` | +| SC Product | `EKS Terragrunt Repository Creator - dev` (v2.0.0, CFN template `eks-terragrunt-repo/2-0-0.yaml`) | +| S3 Object | Product template uploaded to `servicecatalog-product-artifacts-*` bucket | +| SC Product → Portfolio association | Links the product to the portfolio | +| SC Principal → Portfolio association ×3 | `AWSReservedSSO_inf-admin-t2_*`, `AWSReservedSSO_AdministratorAccess_*`, `AWSReservedSSO_AWSAdministratorAccess_*` | +| SC Launch Constraint | Binds `r-ent-servicecatalog-eksterragrunt-launch-role` to the product | + +### IAM (deployed via CFN StackSet to all shared accounts) + +| Resource | Detail | +|----------|--------| +| IAM Role | `r-ent-servicecatalog-eksterragrunt-launch-role` — assumed by Service Catalog when launching the product. Deployed via `CensusServiceCatalog-RoleAndAction-Roles-global` StackSet to every account in the shared OUs. | +| IAM Role Policy | Inline policy granting `lambda:InvokeFunction` (cross-account to csvd-dev Lambda), CloudFormation stack operations, `ec2:DescribeVpcs`, and `s3:GetObject` (artifacts bucket) | + +### Lambda bundle (`west/lambda-bundle` — SC dependency, not EKS-specific) + +| Resource | Detail | +|----------|--------| +| CFN Stack | Deploys the `FnGetResourcesInfo` helper Lambda used by Service Catalog for resource lookups | + +### Global (`global/service-catalog` — SC-wide dependency) + +| Resource | Detail | +|----------|--------| +| DynamoDB Table | `service-catalog-projects` — owned by account `036716915524`; the census module reads the ARN from SSM and does not create this table | + +--- + +## 2. Prerequisites — deployed via `lambda-template-repo-generator/deploy/` + +These resources must exist before a user can successfully provision the SC product. +Deployed by running `tf apply` in `lambda-template-repo-generator/deploy/`. + +### Lambda function (via `terraform-aws-template-automation` module) + +| Resource | Name / Detail | +|----------|---------------| +| `aws_lambda_function` | `eks-terragrunt-repo-gen-template-automation` (container image from ECR, 512 MB, 900 s timeout) | +| `aws_cloudwatch_log_group` | `/aws/lambda/eks-terragrunt-repo-gen-template-automation` | +| `aws_iam_role` | `eks-terragrunt-repo-gen-lambda-role` | +| `aws_iam_role_policy` | Parameter Store read scoped to `/eks-terragrunt-repo-gen/*` | +| `aws_iam_role_policy` | Secrets Manager read scoped to `/eks-cluster-deployment/github_token` | +| `aws_iam_role_policy` | KMS `Decrypt` + `DescribeKey` for the GitHub token secret | +| `aws_iam_role_policy_attachment` | `AWSLambdaBasicExecutionRole` (managed policy) | +| `aws_iam_role_policy_attachment` | `AWSLambdaVPCAccessExecutionRole` (managed policy) | +| `aws_lambda_permission` | Allow CloudFormation org-wide invocation (`principal_org_id`) | +| `aws_lambda_permission` | Allow API Gateway invocation | +| `aws_apigatewayv2_api` | `eks-terragrunt-repo-gen-api` (HTTP API — not used by SC but deployed by the module) | +| `aws_apigatewayv2_stage` | `$default` auto-deploy stage | +| `aws_apigatewayv2_integration` | `AWS_PROXY` → Lambda | +| `aws_apigatewayv2_route` | `POST /template` | +| `aws_ssm_parameter` ×7 | `GITHUB_API`, `GITHUB_ORG_NAME`, `TEMPLATE_REPO_NAME`, `TEMPLATE_CONFIG_FILE`, `GITHUB_COMMIT_AUTHOR_NAME`, `GITHUB_COMMIT_AUTHOR_EMAIL`, `TEMPLATE_TOPICS` | + +### Lambda IAM extras (in `deploy/main.tf` directly) + +| Resource | Detail | +|----------|--------| +| `aws_iam_role_policy` `eks-repo-creator-codebuild-access` | `codebuild:StartBuild` + `codebuild:BatchGetBuilds` on the creator project | +| `aws_iam_role_policy` `eks-repo-creator-tf-github-token-access` | `secretsmanager:GetSecretValue` on `ghe-runner/github-token` | + +### CodeBuild project + +| Resource | Detail | +|----------|--------| +| `aws_codebuild_project` | `eks-terragrunt-repo-creator` — `NO_SOURCE`; buildspec inlined from `terraform-eks-deployment/buildspec.yml`; VPC-enabled; 15-minute timeout | + +### CodeBuild IAM role (`create_codebuild_role` toggle) + +In **csvd-dev** `create_codebuild_role = false` — looks up pre-existing +`CodeBuildPackerRole-eks-terragrunt-repo-generator-builder` by name (no resources created). + +In a **fresh account** (`create_codebuild_role = true`) the following are created: + +| Resource | Detail | +|----------|--------| +| `aws_iam_role` | `eks-terragrunt-repo-creator-role` (assumes `codebuild.amazonaws.com`) | +| `aws_iam_role_policy` `codebuild-logs` | `logs:CreateLogGroup/CreateLogStream/PutLogEvents` scoped to `/aws/codebuild/eks-terragrunt-repo-creator` | +| `aws_iam_role_policy` `codebuild-s3-assets` | `s3:GetObject` on `csvd-packer-pipeline-assets/*` | +| `aws_iam_role_policy` `codebuild-vpc` | EC2 VPC ENI lifecycle: `ec2:DescribeDhcpOptions/DescribeNetworkInterfaces/DescribeSubnets/DescribeSecurityGroups/DescribeVpcs`, `ec2:CreateNetworkInterface`, `ec2:DeleteNetworkInterface`, `ec2:CreateNetworkInterfacePermission` (with `ec2:AuthorizedService = "codebuild.amazonaws.com"` condition) | + +### Pre-existing resources (not created by this Terraform) + +| Resource | Detail | +|----------|--------| +| ECR Repository | `229685449397.dkr.ecr.us-gov-west-1.amazonaws.com/eks-terragrunt-repo-generator/lambda:latest` — built by `eks-terragrunt-repo-generator-builder` CodeBuild via packer pipeline | +| Secrets Manager Secret | `/eks-cluster-deployment/github_token` — GitHub App installation token (`ghs_`) for Lambda Python API calls | +| Secrets Manager Secret | `ghe-runner/github-token` — PAT (`ghp_`) passed to CodeBuild for the Terraform GitHub provider | +| VPC Subnet | `vpc2-csvd-dev-endpoints-us-gov-west-1a` — looked up by name tag; used by Lambda + CodeBuild VPC config | +| Security Group | `it-linux-base` — looked up by group name; used by Lambda + CodeBuild VPC config | + +--- + +## 3. Per-provisioning resources (created on each SC product launch) + +These are created when a user provisions the product and torn down when the provisioned product +is terminated. They are not permanent Terraform-managed resources. + +| Resource | Detail | +|----------|--------| +| CloudFormation Stack | Wraps the `Custom::GitHubRepository` resource | +| GitHub Repository | `github.e.it.census.gov/SCT-Engineering/` — created by CodeBuild + Terraform via `CSVD/terraform-github-repo` module | +| GitHub Branch | `repo-init` — contains 8 rendered Terragrunt HCL files pushed by Terraform | +| GitHub Pull Request | `repo-init` → `main` — opened by the module; URL returned as a CFN output | + +--- + +## 4. Deployment ownership summary + +| Deployed by | Resources | +|-------------|-----------| +| `lambda-template-repo-generator/deploy/` (direct `tf apply`) | Lambda, API GW, CodeBuild project, Lambda IAM extras, CodeBuild role (optional), SSM parameters | +| `terraform-service-catalog-census` (Terragrunt) | SC Portfolio, SC Product, S3 template object, SC constraints, SC launch role, lambda-bundle CFN stack | +| Packer pipeline (`eks-terragrunt-repo-generator-builder`) | ECR image | +| Pre-existing / manual | Secrets Manager secrets, VPC subnet, security group, DynamoDB table | +| SC product launch (ephemeral) | CFN stack, GitHub repo, `repo-init` branch, pull request |