Cleanup (#25)

* remove old clsuters

* cleanup test clusters

* update structures

.github/platform-tg-infra.code-workspace

@@ -4,10 +4,6 @@
 			"name": "platform-tg-infra",
 			"path": ".."
 		},
-		{
-			"name": "tfmod-cert-mgr",
-			"path": "../../tfmod-cert-mgr"
-		},
 		{
 			"name": "tfmod-config-job",
 			"path": "../../tfmod-config-job"
@@ -16,6 +12,10 @@
 			"name": "tfmod-cribl",
 			"path": "../../tfmod-cribl"
 		},
+		{
+			"name": "tfmod-cubestore",
+			"path": "../../tfmod-cubestore"
+		},
 		{
 			"name": "tfmod-custom-iam-role-for-service-account-eks",
 			"path": "../../tfmod-custom-iam-role-for-service-account-eks"
@@ -77,8 +77,8 @@
 			"path": "../../tfmod-loki"
 		},
 		{
-			"name": "tfmod-metrics-server",
-			"path": "../../tfmod-metrics-server"
+			"name": "tfmod-misp",
+			"path": "../../tfmod-misp"
 		},
 		{
 			"name": "tfmod-open-telemetry",
@@ -101,16 +101,24 @@
 			"path": "../../tfmod-tempo"
 		},
 		{
+			"name": "repo-setup",
 			"path": "../../repo-setup"
 		},
 		{
+			"name": "template-eks-cluster",
 			"path": "../../template-eks-cluster"
 		},
 		{
-			"path": "../../module-decomposition"
+			"name": "terraform-eks-deployment",
+			"path": "../../terraform-eks-deployment"
+		},
+		{
+			"name": "terraform-github-repo",
+			"path": "../../terraform-github-repo"
 		},
 		{
-			"path": "../../csvd-platform-lab-srn"
+			"name": "csvd-mcm2-lab",
+			"path": "../../csvd-mcm2-lab"
 		}
 	]
 }

.github/prompts/socratic_planning.prompt.md

@@ -0,0 +1,105 @@
+---
+agent: agent
+---
+# Copilot Instructions: Iterative Socratic Planning Assistant
+
+## Role
+You are a Socratic planning facilitator. Your purpose is to help users develop well-reasoned plans through guided questioning and iterative refinement, not by providing direct solutions.
+
+## Core Behavior
+
+### Always Begin With Questions
+- Start by understanding the user's goal through open-ended questions
+- Never jump to solutions or implementation details immediately
+- Build context before making suggestions
+
+### Question Framework
+
+**Phase 1: Discovery**
+- "What problem are you trying to solve?"
+- "What does success look like for this plan?"
+- "Who are the stakeholders or users?"
+- "What's the broader context or motivation?"
+
+**Phase 2: Challenge Assumptions**
+- "What assumptions are you making about [X]?"
+- "Why do you believe [approach] will work?"
+- "What alternatives have you considered?"
+- "What constraints are real vs. perceived?"
+
+**Phase 3: Explore Context**
+- "What resources or tools do you have available?"
+- "What similar efforts have been attempted before?"
+- "What dependencies or prerequisites exist?"
+- "What timeline are you working with?"
+
+**Phase 4: Break Down Complexity**
+- "What's the smallest first step?"
+- "Which components are most critical?"
+- "What would happen if we removed [element]?"
+- "How would you prioritize these aspects?"
+
+**Phase 5: Test Logic**
+- "How would you verify that assumption?"
+- "What could go wrong with this approach?"
+- "What evidence supports this direction?"
+- "What would cause you to change course?"
+
+**Phase 6: Iterate**
+- "Based on what we've discussed, what changes to your initial thinking?"
+- "What have we learned that affects the plan?"
+- "What questions remain unanswered?"
+
+## Response Guidelines
+
+### DO:
+- Ask one or two focused questions per response
+- Acknowledge user insights and build on them
+- Point out contradictions gently: "Earlier you mentioned X, but now you're suggesting Y. How do these fit together?"
+- Summarize understanding periodically: "So if I understand correctly..."
+- Encourage the user to document their own conclusions
+
+### DON'T:
+- Provide complete solutions unprompted
+- Make decisions for the user
+- Move to implementation before the plan is well-reasoned
+- Ignore contradictions or logical gaps
+- Rush through the discovery process
+
+## Iteration Pattern
+1. User provides initial idea/plan
+2. You ask clarifying questions
+3. User responds
+4. You probe deeper or challenge assumptions
+5. User refines thinking
+6. Repeat until plan is well-defined
+7. Summarize key decisions and rationale
+
+## When to Shift from Questions to Assistance
+Only after:
+- Core goals are clearly defined
+- Major assumptions have been validated or acknowledged
+- User has explored alternatives
+- Critical constraints are understood
+- User explicitly requests implementation help
+
+Then you may say: "It sounds like you have a solid plan. Would you like help implementing [specific aspect]?"
+
+## Example Interaction Flow
+
+**User:** "I need to build a monitoring system."
+
+**You:** "Let's explore this together. What specific problems are you experiencing that led you to need monitoring? What are you hoping to observe or track?"
+
+**User:** [Explains issues]
+
+**You:** "I see. You mentioned [X]. What have you tried so far to address this? What monitoring tools or systems already exist in your environment?"
+
+**User:** [Provides context]
+
+**You:** "Interesting. What would success look like? How would you know the monitoring system is working effectively?"
+
+[Continue iterating...]
+
+## Remember
+Your goal is to help users think through their plans thoroughly, not to think for them. The best outcome is when the user arrives at their own well-reasoned conclusions.

.gitignore

@@ -44,3 +44,4 @@ terraform.rc
 backend.tf
 provider*.tf
 remote_state.backend.tf
+terragrunt-debug.tfvars.json

.gitmodules

@@ -1,12 +1,15 @@
-[submodule "lab/clusters/csvd-platform-lab-mcm"]
-	path = lab/clusters/csvd-platform-lab-mcm
-	url = git@github.e.it.census.gov:sct-engineering/csvd-platform-lab-mcm
-[submodule "lab/clusters/csvd-platform-lab-srn"]
-	path = lab/clusters/csvd-platform-lab-srn
-	url = git@github.e.it.census.gov:sct-engineering/csvd-platform-lab-srn
-[submodule "lab/clusters/csvd-platform-lab-dja"]
-	path = lab/clusters/csvd-platform-lab-dja
-	url = git@github.e.it.census.gov:sct-engineering/csvd-platform-lab-dja
-[submodule "lab/clusters/eks-ois-eem-cribl-prod"]
-	path = lab/clusters/eks-ois-eem-cribl-prod
+[submodule "ent/prod/eks-ois-eem-cribl-prod"]
+	path = ent/prod/eks-ois-eem-cribl-prod
 	url = git@github.e.it.census.gov:sct-engineering/eks-ois-eem-cribl-prod
+[submodule "ent/dev/csvd-dev-mcm"]
+	path = ent/dev/csvd-dev-mcm
+	url = git@github.e.it.census.gov:sct-engineering/csvd-dev-mcm
+[submodule "ent/dev/adsd-tools-dev"]
+	path = ent/dev/adsd-tools-dev
+	url = git@github.e.it.census.gov:sct-engineering/adsd-tools-dev
+[submodule "ent/dev/csvd-mcm-common"]
+	path = ent/dev/csvd-mcm-common
+	url = git@github.e.it.census.gov:sct-engineering/csvd-mcm-common
+[submodule "lab/dev/csvd-lab-mcm"]
+	path = lab/dev/csvd-lab-mcm
+	url = git@github.e.it.census.gov:sct-engineering/csvd-lab-mcm

Makefile

@@ -2,12 +2,12 @@
 # Default values
 ENV ?= development
 AWS_REGION ?= us-gov-east-1
-AWS_ACCOUNT_ID ?= 066921446319
-ACCOUNT_NAME ?= ent-ew-logging-prod
-AWS_PROFILE ?= 066921446319-ent-gov-logging-prod
-CLUSTER_NAME ?= eks-ois-eem-cribl-prod
+AWS_ACCOUNT_ID ?= 224384469011
+ACCOUNT_NAME ?= lab-dev-gov
+AWS_PROFILE ?= 224384469011-lab-dev-gov
+CLUSTER_NAME ?= csvd-platform-lab-mcm
 REGION_DIR ?= us-gov-east-1
-CLUSTER_DIR ?= ois-eem-cribl-prod
+CLUSTER_DIR ?= "lab/clusters/csvd-platform-lab-mcm"
 
 help:
 	@echo "Available targets:"
@@ -25,22 +25,6 @@ help:
 config:
 	@echo "Loading configuration..."
 
-# # Detect configuration files
-# 	$(eval ACCOUNT_HCL=lab/$(ENV)/account.hcl)
-# 	$(eval REGION_HCL=lab/$(ENV)/$(REGION_DIR)/region.hcl)
-# 	$(eval CLUSTER_HCL=lab/$(ENV)/$(REGION_DIR)/vpc/$(CLUSTER_DIR)/cluster.hcl)
-
-# 	@if [ ! -f "$(ACCOUNT_HCL)" ]; then echo "Error: $(ACCOUNT_HCL) not found"; exit 1; fi
-# 	@if [ ! -f "$(REGION_HCL)" ]; then echo "Error: $(REGION_HCL) not found"; exit 1; fi
-# 	@if [ ! -f "$(CLUSTER_HCL)" ]; then echo "Error: $(CLUSTER_HCL) not found"; exit 1; fi
-
-# # Extract values from HCL files
-# 	$(eval AWS_ACCOUNT_ID=$(shell grep -oP 'aws_account_id\s*=\s*"\K[^"]+' $(ACCOUNT_HCL)))
-# 	$(eval ACCOUNT_NAME=$(shell grep -oP 'account_name\s*=\s*"\K[^"]+' $(ACCOUNT_HCL)))
-# 	$(eval AWS_PROFILE=$(shell echo $(AWS_ACCOUNT_ID)-$(shell echo $(ACCOUNT_NAME) | sed 's/-ew/-gov/')))
-# 	$(eval AWS_REGION=$(shell grep -oP 'aws_region\s*=\s*"\K[^"]+' $(REGION_HCL)))
-# 	$(eval CLUSTER_NAME=$(shell grep -oP 'cluster_name\s*=\s*"\K[^"]+' $(CLUSTER_HCL)))
-
 # Calculate derived values
 	$(eval REGION_SHORT=$(shell echo $(AWS_REGION) | sed 's/\([a-z]\)[a-z]*-/\1/g'))
 	$(eval S3_BUCKET=v-s3-eks-$(CLUSTER_NAME)-artifacts-$(AWS_ACCOUNT_ID)-$(REGION_SHORT))
@@ -105,10 +89,10 @@ deploy-to-pipeline: config
 
 	@echo "Copy buildspecs from tfmod-pipeline"
 	mkdir -p ./buildspecs
-	cp -r ../tfmod-pipeline/buildspecs/* ./buildspecs
+	cp -r /apps/terraform/workspaces/morga471/terraform/tfmod-pipeline/buildspecs/* ./buildspecs
 
 	@echo "Creating zip file..."
-	zip -r platform-tg-infra.zip . -x "*.git*" "*.github*" "*.terragrunt-cache*" "*.terraform*"
+	zip -r platform-tg-infra.zip buildspecs $(CLUSTER_DIR) -x "*.git*" "*.github*" "*.terragrunt-cache*" "*.terraform*"
 
 	rm -rf ./buildspecs
 

NOTES.md

@@ -0,0 +1,118 @@
+# NOTES.md
+
+```json
+{
+  "body": {
+    "project_name": "eks-ois-cribl-prod",
+    "owning_team": "tf-module-admins",
+    "template_settings": {
+      "environment": "production",
+      "region": "us-gov-east-1",
+      "cluster_dir": "eks-ois-cribl-prod",
+      "enable_all_modules": true,
+      "account": {
+        "account_name": "ent-ew-logging-prod",
+        "aws_account_id": "066921446319",
+        "aws_profile": "066921446319-ent-gov-logging-prod",
+        "environment_abbr": "prod"
+      },
+      "vpc": {
+        "vpc_name": "vpc1-inf-services",
+        "vpc_domain_name": "services.inf.csp1.census.gov"
+      },
+      "cluster": {
+        "cluster_name": "eks-ois-cribl-prod",
+        "cluster_mailing_list": "ois.acmt.list@census.gov",
+        "eks_instance_disk_size": 200,
+        "eks_ng_desired_size": 2,
+        "eks_ng_max_size": 2,
+        "eks_ng_min_size": 2,
+        "organization": "CHANGEME",
+        "finops_project_name": "CHANGEME",
+        "finops_project_number": "CHANGEME",
+        "finops_project_role": "CHANGEME",
+        "tags": {
+          "slim:schedule": "CHANGEME",
+          "environment": "production",
+          "owner": "OIS Security Engineering",
+          "managed_by": "terragrunt"
+        }
+      }
+    }
+  }
+}
+```
+
+## EKS Cluster Deployment Instructions
+
+The above config.json contains placeholders that need to be replaced with actual values before deployment.
+
+### FYI
+- csvd-platform-lab-mcm and csvd-platform-lab-srn clusters are provisioned and ready
+- The dja cluster has been destroyed but files/repo remain
+
+### Step 1: Merge Required PRs
+Ensure these PRs are merged before proceeding:
+- **Platform Infrastructure PR**: https://github.e.it.census.gov/SCT-Engineering/platform-tg-infra/pull/24
+  - This adds submodules for provisioned clusters
+- **Cluster Repository PR**: https://github.e.it.census.gov/SCT-Engineering/eks-ois-eem-cribl-prod/pull/1
+  - This updates the main branch for the cluster
+
+### Step 2: Update Submodules and Navigate to Cluster
+```bash
+git submodule update
+cd lab/clusters/eks-ois-eem-cribl-prod/
+```
+*Note: If the cluster folder is empty, verify that the cluster repository PR above has been merged.*
+
+### Step 3: Update Configuration with Actual Values
+Replace all "CHANGEME" placeholders in config.json with the actual values:
+- `organization`
+- `finops_project_name`
+- `finops_project_number`
+- `finops_project_role`
+- `slim:schedule` (in tags)
+
+### Step 4: Regenerate Files with Ansible
+```bash
+# Install ansible if not already installed
+pip install ansible
+
+# Generate HCL files from updated config
+ansible-playbook ansible/generate_hcl_files.yml -e config_file=../config.json
+```
+
+### Step 5: Commit and Push Changes
+```bash
+# Review the changes
+git diff
+
+# Stage all changes
+git add .
+
+# Create a new branch
+git checkout -b change
+
+# Commit changes
+git commit -m "updated finops codes and regenerated files"
+
+# Push to remote
+git push
+```
+
+### Step 6: Create and Merge PR
+1. Open a Pull Request with the changes
+2. Review the changes in the PR
+3. Get approval and merge the PR
+
+### Step 7: Deploy Infrastructure
+```bash
+# Navigate back to cluster root
+cd lab/clusters/eks-ois-eem-cribl-prod/
+
+# Apply Terragrunt configuration
+terragrunt run-all apply
+```
+
+### Questions?
+Contact me via cell phone for any questions or issues.