diff --git a/docs/generalized-terraform-product-architecture.md b/docs/generalized-terraform-product-architecture.md index de17343..94610e6 100644 --- a/docs/generalized-terraform-product-architecture.md +++ b/docs/generalized-terraform-product-architecture.md @@ -225,19 +225,21 @@ lambda/ #### How it works (design intent for `app.py`) ```python -# 1. Read template_repo from CFN props (before TfRunRequest is constructed) -template_repo = normalized.get("template_repo") # e.g. "template-s3-bucket" +# 1. Read template_repo + optional ref from CFN props +template_repo = normalized.get("template_repo") # e.g. "template-s3-bucket" +template_repo_ref = normalized.get("template_repo_ref", "") # e.g. "v2.0.0"; empty = default branch if not template_repo: raise ValueError("template_repo is required") -# 2. Fetch handler.py from GHE via the raw contents API +# 2. Fetch handler.py from GHE via the contents API, at the pinned ref github_org = os.environ.get("GITHUB_ORG_NAME", "SCT-Engineering") github_api = os.environ.get("GITHUB_API", "https://github.e.it.census.gov/api/v3") -handler_url = f"{github_api}/repos/{github_org}/{template_repo}/contents/handler.py" +ref_param = f"?ref={template_repo_ref}" if template_repo_ref else "" +handler_url = f"{github_api}/repos/{github_org}/{template_repo}/contents/handler.py{ref_param}" # ...fetch with Authorization header, base64-decode the content... # 3. Load the module dynamically -import types, importlib +import types mod = types.ModuleType("_handler") exec(compile(handler_source, "handler.py", "exec"), mod.__dict__) @@ -245,6 +247,7 @@ exec(compile(handler_source, "handler.py", "exec"), mod.__dict__) if not (callable(getattr(mod, "handle", None)) and getattr(mod, "PRODUCT_TYPE", None)): raise ValueError(f"{template_repo}/handler.py must define PRODUCT_TYPE and handle()") normalized = mod.handle(normalized) +# template_repo_ref is preserved in normalized; CodeBuild receives it as TEMPLATE_REPO_REF tf_req = TfRunRequest(**normalized) ``` @@ -259,6 +262,32 @@ auditable (every template repo change is a PR in the SCT-Engineering org). **Adding a new product type requires only creating a new template repo with a `handler.py`. No Lambda code changes, no Lambda redeployment, no registry entries.** +#### Version pinning + +The `template_repo_ref` property pins the exact git ref (tag, branch, or SHA) that both +the Lambda and the Proposer CodeBuild use. This is how template repo changes are gated +from production use. + +| Value | Behaviour | +|-------|-----------| +| `v2.0.0` (SemVer tag) | Lambda fetches `handler.py` at that tag; Proposer clones and checks out that tag. **Recommended for production.** | +| `main` (branch) | Always latest; appropriate for development and testing. | +| `abc1234` (SHA) | Exact commit; maximally stable but requires manual update. | +| *(absent / empty)* | GHE API returns the default branch; same as `main`. | + +**The ref is set as a static string in the CFN product template — it is not a +user-facing form parameter.** Bumping to a new version is an operator action: + +1. Tag the template repo: `git tag v2.0.0 && git push origin v2.0.0` +2. Update `template_repo_ref` in `service-catalog/{product_type}-product-template.yaml` +3. Run `tf apply` in `deploy_products/` with a bumped `version` key — this creates a new + SC provisioning artifact. Existing provisioned products are unaffected until they are + updated or re-provisioned. + +Because `template_repo_ref` flows through the Lambda to the CodeBuild `TEMPLATE_REPO_REF` +env var, the Lambda and the Proposer always run the **exact same version** of `handler.py` +and the Jinja2 templates — there is no split-brain risk between the two. + ### 5. CloudFormation product template The CFN template for a product type lives in `service-catalog/{product_type}-product-template.yaml` @@ -274,6 +303,8 @@ inside the `sc-lambda-ghactions` repo. It follows the same pattern as the existi Properties: ServiceToken: !Sub "arn:${AWS::Partition}:lambda:${AWS::Region}:${AWS::AccountId}:function:sc-template-automation" product_type: s3_bucket + template_repo: template-s3-bucket + template_repo_ref: v2.0.0 # pinned; bump here + new SC artifact version to release bucket_name: !Ref BucketName account_name: !Ref AccountName aws_account_id: !Sub "${AWS::AccountId}" @@ -283,6 +314,10 @@ Properties: tier: !Ref Tier ``` +`template_repo` and `template_repo_ref` are static strings — they are not user-facing +parameters and do not appear on the SC product form. Changing them requires creating a +new SC provisioning artifact version (managed by `deploy_products/`). + ### 6. `deploy_products/` — dedicated Terraform workspace for SC product management SC portfolio and product registration lives in a **dedicated `deploy_products/` workspace**, @@ -324,6 +359,11 @@ share_ous = [ ] ``` +> **Version alignment:** the `version` key in `sc_products` is the **SC provisioning artifact +> label** displayed in the SC console (e.g. `"2.0.0"`). It is independent of, but should match, +> the `template_repo_ref` property baked into the CFN template file. Convention: +> bump `version` in tfvars and update `template_repo_ref` in the CFN YAML at the same time. + Terraform iterates `var.sc_products` with `for_each` to create the S3 object, SC product, provisioning artifact, and launch constraint for each entry. A single shared portfolio (`aws_servicecatalog_portfolio`) is created once and shared to the OUs listed in @@ -351,8 +391,11 @@ onboard any new Terraform workload without Lambda or CodeBuild changes: - flat `.tf.j2` file(s) rendered by the Proposer - `tf-run.data` - `.sc-automation.yml.j2` (optional; Proposer writes a default if absent) -- [ ] Add `service-catalog/{product_type}-product-template.yaml` to `sc-lambda-ghactions` +- [ ] Tag the initial release: `git tag v1.0.0 && git push origin v1.0.0` +- [ ] Add `service-catalog/{product_type}-product-template.yaml` to `sc-lambda-ghactions`, + setting `template_repo_ref: v1.0.0` as a static property - [ ] Add one entry to `var.sc_products` in `deploy_products/terraform.tfvars` + with `version = "1.0.0"` matching the tag - [ ] Run `tf apply` in `deploy_products/` — creates S3 artifact, SC product, provisioning artifact, launch constraint; all OU-member accounts see the new product immediately - [ ] Validate end-to-end via `scripts/test_service_catalog.py` diff --git a/docs/template-management.md b/docs/template-management.md index 0507459..64584a3 100644 --- a/docs/template-management.md +++ b/docs/template-management.md @@ -254,7 +254,7 @@ variables: # Extra key/value pairs injected as CodeBuild | `product_type` | ✅ | Routes to the correct Pydantic model and template directory | | `executor_project` | ✅ | CodeBuild project started by the webhook on PR merge | | `dry_run` | ✅ | `true` → `tf plan` only; `false` → `tf apply` | -| `template_repo` | ✅ | GHE repo used as the Jinja2 template source | +| `template_repo` | ✅ | GHE repo used as the Jinja2 template source; the Lambda fetches it at the ref specified by `template_repo_ref` in the CFN template | | `template_source_path` | ❌ | Subdirectory within `template_repo`; omit for whole-repo templates | | `fleet_entry` | ❌ | Relative path of this workload's entry in `terraform-sc-fleet` | | `variables` | ❌ | Product-type-specific overrides; merged with SSM global defaults | @@ -308,7 +308,10 @@ Checklist when onboarding a new product type: - `tf-run.data` - `.sc-automation.yml.j2` (optional) **No files in the Lambda repository need to be created or modified.** -- [ ] Add `service-catalog/{product_type}-product-template.yaml` to `sc-lambda-ghactions` +- [ ] Tag the initial release: `git tag v1.0.0 && git push origin v1.0.0` +- [ ] Add `service-catalog/{product_type}-product-template.yaml` to `sc-lambda-ghactions`, + setting `template_repo_ref: v1.0.0` as a static property in the CFN `Properties` block - [ ] Add one entry to `var.sc_products` in `deploy_products/terraform.tfvars` + with `version = "1.0.0"` matching the tag - [ ] Run `tf apply` in `deploy_products/` — portfolio, product, artifact, launch roles, and OU sharing are updated automatically; all OU-member accounts see the change immediately