Skip to content

Callnotes 09152025 #11

Merged
merged 4 commits into from
Nov 19, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions defaults.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# defaults.tf - Default values for module configuration
# These values should not be exposed to users but are used internally by templates

locals {
# Dynamic AWS profile generation
aws_profile = "${var.cluster_config.account_name}-${var.cluster_config.environment_abbr}"

# Static template values (hidden from users)
repository_defaults = {
template = "template-eks-cluster"
template_owner = "SCT-Engineering"
}

# Static EKS configuration for Karpenter bootstrap node group
eks_defaults = {
instance_disk_size = 200
ng_desired_size = 3
ng_max_size = 10
ng_min_size = 3
enable_cluster_creator_admin_permissions = true
}

# Organization settings - now configurable through finops variable
organization_defaults = {
finops_project_name = var.finops.project_name
finops_project_number = var.finops.project_number
finops_project_role = var.finops.project_role
}
}
96 changes: 86 additions & 10 deletions docs/callnote-09152025-implementation.md
Original file line number Diff line number Diff line change
Expand Up @@ -602,24 +602,100 @@ module "github_repo" {

## Implementation Timeline

### September 15, 2025 (Today)
### September 15, 2025 (Completed)
- [x] Complete action items analysis
- [x] Create implementation plan
- [ ] Begin Phase 1: Variable consolidation

### September 16, 2025
- [ ] Complete Phase 1: Module interface cleanup
- [ ] Create defaults.tf file with all module defaults
- [ ] Begin Phase 2: Workspace creation with correct naming (eks-clusters-workspace)
- [ ] Update templates to use passed variables instead of hardcoded values
- [ ] Conduct dry run demo
- [ ] Final testing and validation
- [x] Begin Phase 1: Variable consolidation

### September 16, 2025 (Today - COMPLETED)
- [x] **COMPLETED:** Create defaults.tf file with all module defaults
- [x] **COMPLETED:** Update variables.tf to remove redundant variables
- [x] **COMPLETED:** Update main.tf to pass local values to templates
- [x] **COMPLETED:** Fix locals.tf to use new simplified interface
- [x] **COMPLETED:** Fix validation errors and template variable references
- [x] **COMPLETED:** Update examples to use simplified interface
- [x] **COMPLETED:** Terraform validation and initialization successful
- [ ] **READY FOR NEXT:** Begin Phase 2: Workspace creation (eks-clusters-workspace)
- [ ] **SCHEDULED:** Conduct dry run demo
- [ ] **PENDING:** Final testing and validation

### September 17, 2025
- [ ] Final documentation updates
- [ ] Live demonstration to team
- [ ] Gather feedback and plan next steps

## Current Status - September 16, 2025

### βœ… Completed Implementations

#### Phase 1: Module Interface Cleanup - **COMPLETED**

1. **Created defaults.tf file** - All module defaults are now centralized:
- Dynamic AWS profile generation from account_name + environment_abbr
- Static repository template values (hidden from users)
- EKS bootstrap node group defaults for Karpenter
- Organization default settings (FinOps configurations)

2. **Updated variables.tf** - Simplified user interface:
- Removed redundant variables: `repository_template`, `repository_template_owner`
- Cleaned up `cluster_config` object to only include user-configurable values
- Moved internal variables to separate sections for backward compatibility
- Consolidated naming to single `name` variable for both repository and cluster

3. **Fixed main.tf template calls** - Templates now receive computed values:
- All templates now receive values from locals instead of hardcoded defaults
- Repository template values sourced from `local.repository_defaults`
- EKS sizing values sourced from `local.eks_defaults`
- Organization settings sourced from `local.organization_defaults`

4. **Updated locals.tf** - Fixed to use new variable structure:
- Removed references to non-existent `var.cluster_config.cluster_name`
- Updated `config_json` to use `var.name` for cluster name
- Fixed all template variable references

5. **Updated examples** - Simplified user interface demonstrated:
- Basic example now uses consolidated variable structure
- Removed all redundant configuration options
- Clear demonstration of minimal required inputs

### πŸ”„ Next Steps

#### Phase 2: Workspace Structure Implementation - **READY TO START**

1. **Create eks-clusters-workspace Repository**
- Repository name corrected from "terraform-eks-workspace" to "eks-clusters-workspace"
- Will demonstrate multi-cluster management without conflicts
- Includes examples from both David and Matthew's configurations

2. **Template Variable Handling - ARCHITECTURE DECISION IMPLEMENTED**
- Templates now properly receive variables instead of using hardcoded values
- All defaults defined in defaults.tf and passed via templatefile() calls
- This ensures generated configurations reflect computed values

### πŸ“Š Validation Results

- βœ… **Terraform Validation**: PASSED
- βœ… **Terraform Init**: PASSED
- βœ… **Template Rendering**: WORKING
- βœ… **Variable References**: RESOLVED
- βœ… **Module Loading**: SUCCESSFUL

### πŸ—οΈ Implementation Architecture

The implemented solution follows the call notes discussion:

1. **Single Name Variable**: Both repository and cluster use the same `name` variable
2. **Hidden Complexity**: Static values and implementation details are in defaults.tf
3. **Clean Interface**: Users only see variables they actually need to configure
4. **Template Architecture**: Values are computed in locals and passed to templates

### πŸ”§ Technical Debt Resolved

- **Variable Duplication**: Eliminated separate repository and cluster naming
- **Hardcoded Values**: Moved all defaults to centralized location
- **Template Issues**: Fixed template variable passing instead of hardcoding
- **Interface Complexity**: Reduced user-facing variables by ~60%

## Risk Mitigation

### Potential Issues and Solutions
Expand Down
Submodule eks_deployment.github_repo updated from b6b6cb to 550560
2 changes: 1 addition & 1 deletion examples/basic/.terraform/modules/modules.json
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{"Modules":[{"Key":"","Source":"","Dir":"."},{"Key":"eks_deployment","Source":"../..","Dir":"../.."},{"Key":"eks_deployment.github_repo","Source":"registry.terraform.io/HappyPathway/repo/github","Version":"1.0.83","Dir":".terraform/modules/eks_deployment.github_repo"}]}
{"Modules":[{"Key":"","Source":"","Dir":"."},{"Key":"eks_deployment","Source":"../..","Dir":"../.."},{"Key":"eks_deployment.github_repo","Source":"git::ssh://git@github.e.it.census.gov/CSVD/terraform-github-repo.git","Dir":".terraform/modules/eks_deployment.github_repo"}]}
64 changes: 24 additions & 40 deletions examples/basic/main.tf
Original file line number Diff line number Diff line change
@@ -1,52 +1,36 @@
data "aws_secretsmanager_secret_version" "github_token" {
secret_id = "/eks-cluster-deployment/github_token"
}

provider "github" {
token = data.aws_secretsmanager_secret_version.github_token.secret_string
}

module "eks_deployment" {
source = "../../"

# Repository and cluster configuration - single name for both
name = "eks-test-cluster"
template_repo_org = "SCT-Engineering"
repository_template = "template-eks-cluster"
is_private = false # Set to false to make the repository public
repository_teams = {
"platform-team" = "admin",
"devops-team" = "maintain",
"developers" = "push"
}

# Basic settings
organization = "SCT-Engineering"
environment = "dev"
region = "us-gov-east-1"
github_server_url = "https://github.e.it.census.gov"
name = "mojo-cubestore-dev"
environment = "dev"
region = "us-gov-east-1"

# Cluster configuration
# Cluster configuration - simplified interface
cluster_config = {
account_name = "csvd-dev-ew"
aws_account_id = "229685449397"
environment_abbr = "dev"
aws_profile = "default"
vpc_name = "vpc3-csvd-dev"
vpc_domain_name = "dev.inf.csp1.census.gov"
cluster_mailing_list = "david.j.arnold.jr@census.gov"
eks_instance_disk_size = 100
eks_ng_desired_size = 3
eks_ng_max_size = 6
eks_ng_min_size = 2
organization = "census:ocio:csvd"
finops_project_name = "csvd_platformbaseline"
finops_project_number = "fs0000000078"
finops_project_role = "csvd_platformbaseline_app"
account_name = "ma6-gov"
aws_account_id = "252960665057"
cluster_mailing_list = "adep.mojo.development.list@census.gov"
environment_abbr = "dev"
finops_project_name = "PPSI_DICE"
finops_project_number = "fs0000000015"
finops_project_role = "dice:dev:mojo"
vpc_domain_name = "dev.dice.census.gov"
vpc_name = "vpc2-dice-dev"
tags = {
Owner = "Platform Team",
Environment = "Development",
CostCenter = "123-456"
}
module_enablement_overrides = {
cert_manager = true,
prometheus = true,
grafana = true,
istio = false
Owner = "PETeam"
Environment = "Development"
CostCenter = "fs0000000015"
}
organization = "census:ocio:dice"
}
}

Expand Down
62 changes: 62 additions & 0 deletions examples/mojo-cubestr-dev/.terraform_commits
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
[
{
"commit_hash": "2d1a9976c65f53b370b4a3ff0829394449127d5a",
"commit_message": "Fix formatting of runs-on value in GitHub Actions workflow",
"author": "Your Name",
"timestamp": "2025-09-04T13:49:32.940158"
},
{
"commit_hash": "2d1a9976c65f53b370b4a3ff0829394449127d5a",
"commit_message": "Fix formatting of runs-on value in GitHub Actions workflow",
"author": "Your Name",
"timestamp": "2025-09-04T13:52:48.384810"
},
{
"commit_hash": "2d1a9976c65f53b370b4a3ff0829394449127d5a",
"commit_message": "Fix formatting of runs-on value in GitHub Actions workflow",
"author": "Your Name",
"timestamp": "2025-09-04T14:36:08.594715"
},
{
"commit_hash": "2d1a9976c65f53b370b4a3ff0829394449127d5a",
"commit_message": "Fix formatting of runs-on value in GitHub Actions workflow",
"author": "Your Name",
"timestamp": "2025-09-04T15:59:08.094373"
},
{
"commit_hash": "94a518393002dd67ec0a178aee46323921889767",
"commit_message": "Refactor README and ROADMAP for clarity; update Terraform configurations and remove deprecated variables",
"author": "Your Name",
"timestamp": "2025-09-05T13:52:50.300053"
},
{
"commit_hash": "94a518393002dd67ec0a178aee46323921889767",
"commit_message": "Refactor README and ROADMAP for clarity; update Terraform configurations and remove deprecated variables",
"author": "Your Name",
"timestamp": "2025-09-05T13:57:43.873358"
},
{
"commit_hash": "94a518393002dd67ec0a178aee46323921889767",
"commit_message": "Refactor README and ROADMAP for clarity; update Terraform configurations and remove deprecated variables",
"author": "Your Name",
"timestamp": "2025-09-08T13:08:10.570707"
},
{
"commit_hash": "94a518393002dd67ec0a178aee46323921889767",
"commit_message": "Refactor README and ROADMAP for clarity; update Terraform configurations and remove deprecated variables",
"author": "Your Name",
"timestamp": "2025-09-10T15:25:49.548237"
},
{
"commit_hash": "1694ff2abd5271962c65bf1a8798c15cfdcd0af9",
"commit_message": "Update main.tf",
"author": "David John Arnold Jr",
"timestamp": "2025-09-10T17:11:02.705468"
},
{
"commit_hash": "1694ff2abd5271962c65bf1a8798c15cfdcd0af9",
"commit_message": "Update main.tf",
"author": "David John Arnold Jr",
"timestamp": "2025-09-10T17:17:55.685167"
}
]
45 changes: 45 additions & 0 deletions examples/mojo-cubestr-dev/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
data "aws_secretsmanager_secret_version" "github_token" {
secret_id = "/eks-cluster-deployment/github_token"
}

provider "github" {
token = data.aws_secretsmanager_secret_version.github_token.secret_string
}

module "eks_deployment" {
source = "../../"

# Repository and cluster configuration - single name for both
name = "mojo-cubestr-dev"
environment = "dev"
region = "us-gov-east-1"

# Cluster configuration - simplified interface
cluster_config = {
account_name = "ma6-gov"
aws_account_id = "252960665057"
cluster_mailing_list = "adep.mojo.development.list@census.gov"
environment_abbr = "dev"
finops_project_name = "PPSI_DICE"
finops_project_number = "fs0000000015"
finops_project_role = "dice:dev:mojo"
vpc_domain_name = "dev.dice.census.gov"
vpc_name = "vpc2-dice-dev"
tags = {
Owner = "PETeam"
Environment = "Development"
CostCenter = "fs0000000015"
}
organization = "census:ocio:dice"
}
}

output "repository_url" {
description = "URL of the created GitHub repository"
value = module.eks_deployment.repository_url
}

output "ssh_clone_url" {
description = "SSH clone URL of the repository"
value = module.eks_deployment.ssh_clone_url
}
2 changes: 2 additions & 0 deletions examples/mojo-cubestr-dev/providers.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
provider "aws" {
}
1 change: 1 addition & 0 deletions examples/mojo-cubestr-dev/varfiles/default.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{}
45 changes: 45 additions & 0 deletions examples/ois-misp-prod/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
data "aws_secretsmanager_secret_version" "github_token" {
secret_id = "/eks-cluster-deployment/github_token"
}

provider "github" {
token = data.aws_secretsmanager_secret_version.github_token.secret_string
}

module "eks_deployment" {
source = "../../"

# Repository and cluster configuration - single name for both
name = "ois-misp-prod"
environment = "prod"
region = "us-gov-east-1"

# Cluster configuration - simplified interface
cluster_config = {
account_name = "ois-misp-prod-gov"
aws_account_id = "399570725053"
cluster_mailing_list = "ois.misp.list@census.gov"
environment_abbr = "prod"
finops_project_name = "OIS_MISP"
finops_project_number = "fs0000000102"
finops_project_role = "OIS_MISP_APP"
vpc_domain_name = "shared.inf.csp1.census.gov"
vpc_name = "vpc2-inf-shared"
tags = {
Owner = "ois.misp.list@census.gov"
Environment = "production"
CostCenter = "ocio:ois-misp"
}
organization = "census:ocio:ois"
}
}

output "repository_url" {
description = "URL of the created GitHub repository"
value = module.eks_deployment.repository_url
}

output "ssh_clone_url" {
description = "SSH clone URL of the repository"
value = module.eks_deployment.ssh_clone_url
}
2 changes: 2 additions & 0 deletions examples/ois-misp-prod/providers.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
provider "aws" {
}
1 change: 1 addition & 0 deletions examples/ois-misp-prod/varfiles/default.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{}
Loading
Loading