Skip to content

fix: EKS-only Lambda cleanup + SC template AwsRegion/AWSAccountId removal #1

Merged
merged 20 commits into from
Apr 21, 2026

Commits on Apr 2, 2026

  1. fix: use Lambda-only approach for EKS repo creation; add Copilot inst…

    …ructions
    
    - service-catalog/product-template.yaml: drop LambdaFunctionArn parameter,
    - .github/copilot-instructions.md: document Lambda-first approach, explicitly
      record why CodeBuild+Terraform was abandoned (SSH host keys, proxy, provider
      version conflict), clarify CodeBuild is still used for container image CI/CD
    Your Name committed Apr 2, 2026
    Configuration menu
    Copy the full SHA
    803168a View commit details
    Browse the repository at this point in the history
  2. fix: public visibility by default; add collaborator support for repo …

    …creator
    
    - repository_provider.py: change REPO_VISIBILITY default from 'internal' to
      'public' so created repos are visible to all org members and external
      visitors on GHE
    - github_provider.py: change fallback visibility from 'internal' to 'public';
      add add_collaborator() method (PUT /repos/{org}/{repo}/collaborators/{user})
    - app.py: add optional creator_username field to CloudFormationResourceInput;
      after team permission grant, also add individual creator as admin collaborator
      when creator_username is provided (non-fatal on failure)
    - service-catalog/product-template.yaml: add optional CreatorUsername parameter
      (defaults to empty string, backward-compatible); wire to creator_username
      Lambda property
    Your Name committed Apr 2, 2026
    Configuration menu
    Copy the full SHA
    0a74dd7 View commit details
    Browse the repository at this point in the history
  3. fix: VERIFY_SSL=false; public repo visibility; add ec2:DescribeVpcs t…

    …o SC launch role
    
    - VERIFY_SSL was incorrectly set to 'true' (Census CA cert not in certifi)
    - repo_visibility changed from 'internal' to 'public' per ECA requirements
    - Added EC2DescribeVpcs permission to SC launch role IAM policy
    Your Name committed Apr 2, 2026
    Configuration menu
    Copy the full SHA
    528f4b3 View commit details
    Browse the repository at this point in the history

Commits on Apr 6, 2026

  1. feat: path_mapper for dynamic EKS repo structure (safe revert baseline)

    - build_eks_path_mapper remaps environment/region/vpc/cluster/ placeholders
      to concrete values from CFN params
    - github_provider: clone_repository_contents accepts optional path_mapper
    - csvd_config_packer.hcl: create_role=false, packer 1.10.3, role ARN wired
    - deploy/service_catalog.tf: SC product and portfolio synced
    - Updated copilot instructions with Lambda-only architecture decision
    Your Name committed Apr 6, 2026
    Configuration menu
    Copy the full SHA
    a79cee4 View commit details
    Browse the repository at this point in the history
  2. feat: Lambda delegates EKS repos to CodeBuild + terraform-eks-deployment

    - app.py: add start_codebuild_build() and poll_codebuild_build() helpers
    - app.py: EKS deployment path (is_eks_deployment=True) now starts CodeBuild
      project 'eks-terragrunt-repo-creator', polls until SUCCEEDED/FAILED,
      and sends cfn-response accordingly; non-EKS path unchanged
    - deploy/main.tf: add aws_codebuild_project.eks_repo_creator resource
      (NO_SOURCE, uses buildspec.yml from terraform-eks-deployment)
      CODEBUILD_PROJECT_NAME injected into Lambda environment
    - deploy/variables.tf: codebuild_project_name, codebuild_role_arn, codebuild_vpc_id
    - deploy/terraform.tfvars: set CodeBuild project name, role ARN, VPC ID
    Your Name committed Apr 6, 2026
    Configuration menu
    Copy the full SHA
    ec54b54 View commit details
    Browse the repository at this point in the history
  3. chore: tf apply — add eks-terragrunt-repo-creator CodeBuild project +…

    … Lambda CODEBUILD_PROJECT_NAME env var
    Your Name committed Apr 6, 2026
    Configuration menu
    Copy the full SHA
    52ebef0 View commit details
    Browse the repository at this point in the history
  4. fix: add CodeBuild VPC endpoint + IAM policy for Lambda→CodeBuild con…

    …nectivity
    
    Root cause of 'Connect timeout on codebuild.us-gov-west-1.amazonaws.com':
    - Lambda is in a VPC with no NAT gateway path to CodeBuild's public endpoint
    - Lambda role was missing codebuild:StartBuild / codebuild:BatchGetBuilds perms
    
    Fix:
    - aws_vpc_endpoint.codebuild[0]: interface endpoint for CodeBuild in the Lambda
      VPC with private DNS enabled (Lambda API calls resolve to private IPs)
    - aws_iam_role_policy.codebuild_access: StartBuild + BatchGetBuilds on the
      eks-terragrunt-repo-creator project ARN only
    Your Name committed Apr 6, 2026
    Configuration menu
    Copy the full SHA
    aee6987 View commit details
    Browse the repository at this point in the history
  5. fix: increase Lambda timeout to 900s to cover CodeBuild poll window

    Lambda was set to 300s but poll_codebuild_build loops for up to 12 min (720s).
    Lambda would be killed by AWS before it could report back to CloudFormation.
    900s gives a ~180s buffer beyond the poll window.
    Your Name committed Apr 6, 2026
    Configuration menu
    Copy the full SHA
    8310ee1 View commit details
    Browse the repository at this point in the history

Commits on Apr 7, 2026

  1. fix: remove spurious '- ' prefix from additional_post_build_commands

    The packer-pipeline internal buildspec template already wraps the value
    in '- {{ additional_post_build_commands }}', so prefixing the value with
    '- ' caused YAML_FILE_ERROR (nested list) in CodeBuild build #8.
    Your Name committed Apr 7, 2026
    Configuration menu
    Copy the full SHA
    eb18463 View commit details
    Browse the repository at this point in the history
  2. fix: use PAT (ghe-runner/github-token) for Terraform GitHub provider …

    …in CodeBuild
    
    The standard github_token (/eks-cluster-deployment/github_token) is a GitHub
    App installation token (ghs_ prefix) which cannot access /api/v3/user. This
    endpoint is always called by the CSVD terraform-github-repo module's
    data.github_user.current resource.
    
    Changes:
    - app.py: check TF_GITHUB_TOKEN_SECRET_NAME env var first for CodeBuild token;
      falls back to GITHUB_TOKEN_SECRET_NAME if not set
    - deploy/main.tf: add TF_GITHUB_TOKEN_SECRET_NAME=ghe-runner/github-token env var
    - deploy/main.tf: add IAM policy granting Lambda access to ghe-runner/github-token
    Your Name committed Apr 7, 2026
    Configuration menu
    Copy the full SHA
    5d3ff19 View commit details
    Browse the repository at this point in the history
  3. fix: add pull_request_url and branch_name to CodeBuild success response

    returning repository_url/repository_name, causing CFN to fail with:
      'Vendor response doesn't contain pull_request_url attribute'
    
    After CodeBuild SUCCEEDED, query GitHub API /pulls?state=open on the
    created repo to get the real PR URL and branch name.
    Your Name committed Apr 7, 2026
    Configuration menu
    Copy the full SHA
    26c6fe9 View commit details
    Browse the repository at this point in the history
  4. docs: rewrite copilot-instructions to reflect CodeBuild+Terraform arc…

    …hitecture
    
    - Replace 'Lambda, NOT CodeBuild' section with the actual working architecture
    - Document Lambda as thin orchestrator triggering eks-terragrunt-repo-creator
    - Add two-token split explanation (ghs_ App token vs ghp_ PAT for Terraform)
    - Add TF_GITHUB_TOKEN_SECRET_NAME and CODEBUILD_PROJECT_NAME env vars
    - Add correct rebuild/test commands
    - Remove outdated CodeBuild-was-abandoned rationale
    Your Name committed Apr 7, 2026
    Configuration menu
    Copy the full SHA
    12a742a View commit details
    Browse the repository at this point in the history
  5. chore: update deploy Terraform state after tf apply

    Your Name committed Apr 7, 2026
    Configuration menu
    Copy the full SHA
    065d2f2 View commit details
    Browse the repository at this point in the history

Commits on Apr 14, 2026

  1. fix: address PR1 review comments — EKS-only Lambda + Terraform cleanup

    - Rewrite app.py to ~380 lines (EKS + CodeBuild only); remove all generic
      GitHub/GitLab/template-manager code paths
    - Delete 7 dead modules: repository_provider, github_provider, gitlab_provider,
      github_client, gitlab_client, template_manager, models
    - Delete eks_config.py (Jinja2 rendering now handled entirely by terraform-eks-
      deployment in CodeBuild)
    - Rewrite tests/test_app.py for EKS-only handler; drop stale test files
    - Simplify requirements.txt (remove requests/requests-mock; now using urllib)
    
    deploy/ Terraform:
    - Add conditional CodeBuild IAM role (create_codebuild_role var); default=true
      creates a minimal role with scoped logs/S3/VPC policies; false looks up a
      pre-existing role by name
    - Add data sources for subnet + security group lookup by name (no hardcoded IDs)
    - Replace hardcoded 'ghe-runner/github-token' with var.tf_github_token_secret_name
    - Fix IAM policy partition: arn:aws-us-gov: -> arn:${partition}:
    - Remove aws_vpc_endpoint.codebuild (VPC endpoints already exist in the account)
    - Remove codebuild_role_arn / codebuild_vpc_id / subnet_ids / security_group_ids
      variables; add codebuild_role_name, subnet_name, security_group_name,
      tf_github_token_secret_name, codebuild_assets_bucket
    
    service-catalog/product-template.yaml:
    - Fix ServiceToken partition: arn:aws-us-gov:lambda: -> arn:${AWS::Partition}:lambda:
    
    csvd_config_packer.hcl:
    - Annotate account-specific fields with (*) comments explaining they cannot be
      dynamically resolved by packer-pipeline HCL
    Your Name committed Apr 14, 2026
    Configuration menu
    Copy the full SHA
    560a5ec View commit details
    Browse the repository at this point in the history
  2. docs: clarify cross-account architecture + fix stale refs

    - Rewrite architecture section to explicitly document cross-account flow:
      the SC product is shared to multiple accounts but all compute (Lambda,
      CodeBuild, Secrets Manager) runs centrally in csvd-dev (229685449397)
    - Add cross-account ASCII diagram showing provisioner account vs csvd-dev
    - Remove stale reference to eks_config.py (deleted in prior commit)
    - Remove stale generic-mode fallback description (Lambda is EKS-only)
    - Add EKS_SC_RESOURCE_INVENTORY.md: full catalog of resources by owner
      (census Terragrunt, direct Terraform, StackSet, pre-existing, ephemeral)
      with cross-account architecture section and StackSet note on launch role
    Your Name committed Apr 14, 2026
    Configuration menu
    Copy the full SHA
    dff9bfa View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    e6547ed View commit details
    Browse the repository at this point in the history

Commits on Apr 21, 2026

  1. fix(lambda): make EKS fields required; remove is_eks_deployment dead …

    …branch
    
    The Lambda is EKS-only by design — the is_eks_deployment false-branch only
    ever raised a ValueError with no alternative path. Remove it in favour of
    Pydantic required fields, which surface missing fields with precise error
    messages at validation time.
    
    Required fields (was Optional): cluster_name, environment, aws_region,
    account_name, aws_account_id, environment_abbr, vpc_name, vpc_domain_name.
    
    Also simplifies start_codebuild_build() — 'or ""' fallbacks on required
    fields are no longer needed.
    Dave Arnold committed Apr 21, 2026
    Configuration menu
    Copy the full SHA
    ff2a6b5 View commit details
    Browse the repository at this point in the history
  2. fix(sc-template): remove AwsRegion/AWSAccountId as user-facing parame…

    …ters
    
    Both values are now resolved automatically by CloudFormation:
    
    This ensures the provisioner's own account ID and region are used,
    not hardcoded values or user-supplied inputs that could mismatch.
    
    Removes two parameters from the form users see when provisioning.
    Dave Arnold committed Apr 21, 2026
    Configuration menu
    Copy the full SHA
    f37b6c6 View commit details
    Browse the repository at this point in the history
  3. fix(deploy): add eks-repo-creator buildspec; fix partition refs in IA…

    …M policies
    
    Adds deploy/buildspec-eks-repo-creator.yml — the CodeBuild buildspec for
    the eks-terragrunt-repo-creator project, versioned alongside the Terraform
    that manages it rather than referencing terraform-eks-deployment.
    
    Currently points at REPO_BRANCH=test_cluster (PR #16 under review) — must
    be updated to 'main' once PR #16 merges.
    
    Also fixes all IAM policy ARN constructions to use data.aws_partition.current
    instead of data.aws_caller_identity.current (caller_identity has no
    .partition attribute in this provider version).
    Dave Arnold committed Apr 21, 2026
    Configuration menu
    Copy the full SHA
    237ab9b View commit details
    Browse the repository at this point in the history
  4. chore: update docs, scripts, and state to reflect current architecture

    - copilot-instructions: simplify rebuild steps to use packer-pipeline;
      add Python/CLI standards section; add AWS_DEFAULT_REGION reminder;
      add 'do not re-add AwsRegion/AWSAccountId' to What NOT to Do
    - DEPLOYMENT.md, CLOUDFORMATION_CUSTOM_RESOURCE_MIGRATION.md: fix
      packer-pipeline invocation to use csvd_config_packer.hcl
    - csvd_config_packer.hcl: remove duplicate stale comment block at EOF
    - DEMO_SCRIPT.md: update param table and step 3 walkthrough to match
      the corrected SC form (no AwsRegion/AWSAccountId, correct field names)
    - test_service_catalog.py: bump artifact to v2.1; remove AwsRegion and
    - docs/SC-TEMPLATE-FIX-PLAN.md: add completed fix plan for reference
    - deploy state: updated after today's tf apply
    Dave Arnold committed Apr 21, 2026
    Configuration menu
    Copy the full SHA
    8b268ff View commit details
    Browse the repository at this point in the history