Skip to content

Commit

Permalink
Add EKS addons to deprecate tg modules (#46)
Browse files Browse the repository at this point in the history
- now uses aws provider 6.x
- eks module updated to 21.6.1
- k8s version 1.34 support
- added ingress route for metrics-server
- added ingress route for cert-manager
- tags on resources removed as they should be handled at provider (also
provider throws on dupe tags)
- added cert-manager as eks-addon
- added metrics-server as eks-addon
- added eks-node-monitoring-agent as eks-addon
- added before_compute = true to vpc-cni and ebs-pod-identity addons
- updated node-pool label for karpenter.sh/controller = true
  • Loading branch information
morga471 committed Jan 13, 2026
1 parent 2dc2eda commit 54692ae
Show file tree
Hide file tree
Showing 13 changed files with 144 additions and 163 deletions.
26 changes: 4 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,14 +97,14 @@ efs-csi-controller 0 5m
| Name | Version |
|------|---------|
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.13 |
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | ~> 5.0 |
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | ~> 6.0 |
| <a name="requirement_null"></a> [null](#requirement\_null) | ~> 3.2 |

## Providers

| Name | Version |
|------|---------|
| <a name="provider_aws"></a> [aws](#provider\_aws) | 5.100.0 |
| <a name="provider_aws"></a> [aws](#provider\_aws) | 6.27.0 |
| <a name="provider_null"></a> [null](#provider\_null) | 3.2.4 |
| <a name="provider_terraform"></a> [terraform](#provider\_terraform) | n/a |

Expand All @@ -113,7 +113,7 @@ efs-csi-controller 0 5m
| Name | Source | Version |
|------|--------|---------|
| <a name="module_cloudwatch_observability_irsa_role"></a> [cloudwatch\_observability\_irsa\_role](#module\_cloudwatch\_observability\_irsa\_role) | git::https://github.e.it.census.gov/SCT-Engineering/terraform-aws-iam//modules/iam-role-for-service-accounts-eks | n/a |
| <a name="module_cluster"></a> [cluster](#module\_cluster) | git::https://github.e.it.census.gov/SCT-Engineering/terraform-aws-eks/ | v20.37.2 |
| <a name="module_cluster"></a> [cluster](#module\_cluster) | git::https://github.e.it.census.gov/SCT-Engineering/terraform-aws-eks/ | v21.11.0 |
| <a name="module_ebs_csi_irsa_role"></a> [ebs\_csi\_irsa\_role](#module\_ebs\_csi\_irsa\_role) | git::https://github.e.it.census.gov/SCT-Engineering/terraform-aws-iam//modules/iam-role-for-service-accounts-eks | n/a |
| <a name="module_efs_csi_irsa_role"></a> [efs\_csi\_irsa\_role](#module\_efs\_csi\_irsa\_role) | git::https://github.e.it.census.gov/SCT-Engineering/terraform-aws-iam//modules/iam-role-for-service-accounts-eks | n/a |
| <a name="module_vpc_cni_irsa_role"></a> [vpc\_cni\_irsa\_role](#module\_vpc\_cni\_irsa\_role) | git::https://github.e.it.census.gov/SCT-Engineering/terraform-aws-iam//modules/iam-role-for-service-accounts-eks | n/a |
Expand All @@ -123,39 +123,23 @@ efs-csi-controller 0 5m
| Name | Type |
|------|------|
| [aws_ec2_tag.container_subnets](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_tag) | resource |
| [aws_iam_policy.cloudwatch-policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy.cluster-admin-policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy.nlb-policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy_attachment.cluster-admin-attach](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy_attachment) | resource |
| [aws_iam_role.role_cluster-admin](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
| [aws_iam_role.role_eks-cluster](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
| [aws_iam_role_policy_attachment.eks-cluster-cloudwatch](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_role_policy_attachment.eks-cluster-managed](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_role_policy_attachment.eks-cluster-nlb](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_security_group.additional_eks_cluster_sg](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource |
| [aws_security_group.all_worker_mgmt](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource |
| [aws_security_group.extra_cluster_sg](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource |
| [aws_security_group_rule.allow_sidecar_injection](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource |
| [aws_vpc_security_group_egress_rule.additional](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_security_group_egress_rule) | resource |
| [aws_vpc_security_group_ingress_rule.additional](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_security_group_ingress_rule) | resource |
| [aws_vpc_security_group_ingress_rule.additional_ingress_rules_2](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_security_group_ingress_rule) | resource |
| [null_resource.git_version](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) | resource |
| [terraform_data.subnet_validation](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource |
| [aws_arn.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/arn) | data source |
| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
| [aws_ebs_default_kms_key.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ebs_default_kms_key) | data source |
| [aws_iam_policy.cluster_managed_policies](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy) | data source |
| [aws_iam_policy_document.allow_sts](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.cloudwatch-policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.cluster-admin-policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.eks_assume](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.nlb-policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_roles.roles](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_roles) | data source |
| [aws_iam_roles.sso_admins](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_roles) | data source |
| [aws_iam_roles.sso_devs](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_roles) | data source |
| [aws_iam_roles.sso_read](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_roles) | data source |
| [aws_iam_session_context.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_session_context) | data source |
| [aws_kms_key.ebs_key](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/kms_key) | data source |
| [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |
| [aws_subnet.subnets](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnet) | data source |
| [aws_subnets.subnets](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnets) | data source |
| [aws_vpc.eks_vpc](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc) | data source |
Expand All @@ -167,8 +151,6 @@ efs-csi-controller 0 5m
| <a name="input_access_entries"></a> [access\_entries](#input\_access\_entries) | Map of access entries to add to the cluster | `any` | `{}` | no |
| <a name="input_census_private_cidr"></a> [census\_private\_cidr](#input\_census\_private\_cidr) | Census Private CIR Blocks | `list(string)` | <pre>[<br/> "148.129.0.0/16",<br/> "172.16.0.0/12",<br/> "192.168.0.0/16",<br/> "10.0.0.0/16"<br/>]</pre> | no |
| <a name="input_cloudwatch_retention_days"></a> [cloudwatch\_retention\_days](#input\_cloudwatch\_retention\_days) | number of days to retain logs in cloudwatch | `string` | `"14"` | no |
| <a name="input_cluster_endpoint_private_access"></a> [cluster\_endpoint\_private\_access](#input\_cluster\_endpoint\_private\_access) | Whether the EKS cluster API server endpoint is privately accessible | `bool` | `true` | no |
| <a name="input_cluster_endpoint_public_access"></a> [cluster\_endpoint\_public\_access](#input\_cluster\_endpoint\_public\_access) | Whether the EKS cluster API server endpoint is publicly accessible | `bool` | `true` | no |
| <a name="input_cluster_name"></a> [cluster\_name](#input\_cluster\_name) | EKS cluster name name component used through out the EKS cluster describing its purpose (ex: dice-dev) | `string` | n/a | yes |
| <a name="input_cluster_version"></a> [cluster\_version](#input\_cluster\_version) | Kubernetes version to use for the EKS cluster | `string` | n/a | yes |
| <a name="input_eks_instance_disk_size"></a> [eks\_instance\_disk\_size](#input\_eks\_instance\_disk\_size) | Size of the EKS node disk in GB | `number` | `80` | no |
Expand Down
7 changes: 6 additions & 1 deletion access_entries.tf → access-entries.tf
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,11 @@ data "aws_iam_roles" "sso_admins" {
path_prefix = "/aws-reserved/sso.amazonaws.com/"
}

data "aws_iam_roles" "sso_devs" {
name_regex = "AWSReservedSSO_sc-developer"
path_prefix = "/aws-reserved/sso.amazonaws.com/"
}

data "aws_iam_roles" "roles" {
name_regex = "r-inf-terraform(-eks)"
}
Expand Down Expand Up @@ -38,7 +43,7 @@ locals {
}
}
viewers = {
for arn in tolist(data.aws_iam_roles.sso_read.arns) :
for arn in concat(tolist(data.aws_iam_roles.sso_read.arns), tolist(data.aws_iam_roles.sso_devs.arns)) :
arn => {
principal_arn = arn
kubernetes_groups = ["eks-console-dashboard-restricted-access-group"]
Expand Down
68 changes: 68 additions & 0 deletions additional-sg-rules.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
locals {
node_security_group_additional_rules = {
ingress_nodes_ephemeral = {
description = "Node to node ingress on ephemeral ports"
from_port = 80
protocol = "tcp"
self = true
to_port = 65535
type = "ingress"
}
# ALB controller, NGINX
ingress_cluster_9443_webhook = {
description = "Cluster API to node 9443/tcp webhook"
from_port = 9443
protocol = "tcp"
source_cluster_security_group = true
to_port = 9443
type = "ingress"
}
# no longer required as of k8s v1.34+
# ingress_metrics_server = {
# description = "Metrics server"
# from_port = 10251
# protocol = "tcp"
# source_cluster_security_group = true
# to_port = 10251
# type = "ingress"
# }
ingress_cert_manager_webhook = {
description = "cert-manager webhook"
from_port = 10260
protocol = "tcp"
source_cluster_security_group = true
to_port = 10260
type = "ingress"
}
}
cluster_security_group_additional_rules = {
api_internal_148_129 = {
cidr_blocks = ["148.129.0.0/16"]
from_port = 443
to_port = 443
ip_protocol = "tcp"
description = "Census Internal 148.129/16"
},
api_internal_192_168 = {
cidr_blocks = ["192.168.0.0/16"]
from_port = 443
to_port = 443
ip_protocol = "tcp"
description = "Census Internal 192.168/16"
},
api_internal_172_16 = {
cidr_blocks = ["172.16.0.0/12"]
from_port = 443
to_port = 443
ip_protocol = "tcp"
description = "Census Internal 172.16/12"
},
api_internal_10_0 = {
cidr_blocks = ["10.0.0.0/8"]
from_port = 443
to_port = 443
ip_protocol = "tcp"
description = "Census Internal 10/8"
}
}
}
21 changes: 0 additions & 21 deletions additional_sg_rules.tf

This file was deleted.

2 changes: 0 additions & 2 deletions aws_data.tf → aws-data.tf
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
data "aws_caller_identity" "current" {}

data "aws_region" "current" {}

data "aws_arn" "current" {
arn = data.aws_caller_identity.current.arn
}
Expand Down
13 changes: 5 additions & 8 deletions cluster-admin.tf → cluster-admin.tf.off
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
#---
# cluster-admin
# This is deprecated by
# enable_cluster_creator_admin_permissions = var.enable_cluster_creator_admin_permissions
# in main.tf
#---
locals {
iam_arn = format("arn:%v:iam::%v:%%v", data.aws_arn.current.partition, data.aws_caller_identity.current.account_id)
common_arn = format("arn:%v:%%v:%v:%v:%%v", data.aws_arn.current.partition, data.aws_region.current.name, data.aws_caller_identity.current.account_id)
common_arn = format("arn:%v:%%v:%v:%v:%%v", data.aws_arn.current.partition, data.aws_region.current.id, data.aws_caller_identity.current.account_id)
eks_resources = ["cluster", "addon", "nodegroup", "identityproviderconfig"]

admin_policy_statements = {
Expand Down Expand Up @@ -58,7 +61,7 @@ locals {
"ssm:GetParameter",
]
resources = [
format("arn:%v:%v:%v:%v:%v", data.aws_arn.current.partition, "ssm", data.aws_region.current.name, "", "parameter/aws/service/eks/*")
format("arn:%v:%v:%v:%v:%v", data.aws_arn.current.partition, "ssm", data.aws_region.current.id, "", "parameter/aws/service/eks/*")
]
}
EKSReadMyClusters = {
Expand All @@ -83,7 +86,6 @@ resource "aws_iam_role" "role_cluster-admin" {

assume_role_policy = data.aws_iam_policy_document.allow_sts.json
force_detach_policies = true
tags = var.tags
}

resource "aws_iam_policy_attachment" "cluster-admin-attach" {
Expand All @@ -100,11 +102,6 @@ resource "aws_iam_policy" "cluster-admin-policy" {
path = "/"
description = "Allow for administration of the cluster ${var.cluster_name} using AWS resources"
policy = data.aws_iam_policy_document.cluster-admin-policy.json

tags = merge(
local.base_tags,
var.tags
)
}

data "aws_iam_policy_document" "cluster-admin-policy" {
Expand Down
5 changes: 4 additions & 1 deletion cluster-role.tf → cluster-role.tf.off
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
#---
# cluster
# cluster role
# This is deprecated by
# enable_cluster_creator_admin_permissions = var.enable_cluster_creator_admin_permissions
# in main.tf
#---
locals {
cluster_managed_policy_list = [
Expand Down
4 changes: 0 additions & 4 deletions irsa_roles.tf → irsa-roles.tf
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ module "vpc_cni_irsa_role" {
namespace_service_accounts = ["kube-system:aws-node"]
}
}
tags = local.tags
}

module "ebs_csi_irsa_role" {
Expand All @@ -31,7 +30,6 @@ module "ebs_csi_irsa_role" {
namespace_service_accounts = ["kube-system:ebs-csi-controller-sa"]
}
}
tags = local.tags
}

module "efs_csi_irsa_role" {
Expand All @@ -48,7 +46,6 @@ module "efs_csi_irsa_role" {
namespace_service_accounts = ["kube-system:efs-csi-controller-sa"]
}
}
tags = local.tags
}

module "cloudwatch_observability_irsa_role" {
Expand All @@ -67,5 +64,4 @@ module "cloudwatch_observability_irsa_role" {
]
}
}
tags = local.tags
}
Loading

0 comments on commit 54692ae

Please sign in to comment.