Skip to content

Commit

Permalink
Cribl-based findings/remediations (#45)
Browse files Browse the repository at this point in the history
- add security groups and ports file for additional sg
- add roles/policies.tf
- refactor roles/policies.tf into cluster-admin and cluster-role
-
  • Loading branch information
morga471 committed Sep 5, 2025
1 parent 24dba52 commit 2dc2eda
Show file tree
Hide file tree
Showing 9 changed files with 433 additions and 19 deletions.
31 changes: 25 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,14 +97,14 @@ efs-csi-controller 0 5m
| Name | Version |
|------|---------|
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.13 |
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | ~> 5.14 |
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | ~> 5.0 |
| <a name="requirement_null"></a> [null](#requirement\_null) | ~> 3.2 |

## Providers

| Name | Version |
|------|---------|
| <a name="provider_aws"></a> [aws](#provider\_aws) | 5.96.0 |
| <a name="provider_aws"></a> [aws](#provider\_aws) | 5.100.0 |
| <a name="provider_null"></a> [null](#provider\_null) | 3.2.4 |
| <a name="provider_terraform"></a> [terraform](#provider\_terraform) | n/a |

Expand All @@ -113,7 +113,7 @@ efs-csi-controller 0 5m
| Name | Source | Version |
|------|--------|---------|
| <a name="module_cloudwatch_observability_irsa_role"></a> [cloudwatch\_observability\_irsa\_role](#module\_cloudwatch\_observability\_irsa\_role) | git::https://github.e.it.census.gov/SCT-Engineering/terraform-aws-iam//modules/iam-role-for-service-accounts-eks | n/a |
| <a name="module_cluster"></a> [cluster](#module\_cluster) | git::https://github.e.it.census.gov/SCT-Engineering/terraform-aws-eks/ | v20.36.0 |
| <a name="module_cluster"></a> [cluster](#module\_cluster) | git::https://github.e.it.census.gov/SCT-Engineering/terraform-aws-eks/ | v20.37.2 |
| <a name="module_ebs_csi_irsa_role"></a> [ebs\_csi\_irsa\_role](#module\_ebs\_csi\_irsa\_role) | git::https://github.e.it.census.gov/SCT-Engineering/terraform-aws-iam//modules/iam-role-for-service-accounts-eks | n/a |
| <a name="module_efs_csi_irsa_role"></a> [efs\_csi\_irsa\_role](#module\_efs\_csi\_irsa\_role) | git::https://github.e.it.census.gov/SCT-Engineering/terraform-aws-iam//modules/iam-role-for-service-accounts-eks | n/a |
| <a name="module_vpc_cni_irsa_role"></a> [vpc\_cni\_irsa\_role](#module\_vpc\_cni\_irsa\_role) | git::https://github.e.it.census.gov/SCT-Engineering/terraform-aws-iam//modules/iam-role-for-service-accounts-eks | n/a |
Expand All @@ -123,20 +123,39 @@ efs-csi-controller 0 5m
| Name | Type |
|------|------|
| [aws_ec2_tag.container_subnets](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_tag) | resource |
| [aws_iam_policy.cloudwatch-policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy.cluster-admin-policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy.nlb-policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy_attachment.cluster-admin-attach](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy_attachment) | resource |
| [aws_iam_role.role_cluster-admin](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
| [aws_iam_role.role_eks-cluster](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
| [aws_iam_role_policy_attachment.eks-cluster-cloudwatch](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_role_policy_attachment.eks-cluster-managed](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_role_policy_attachment.eks-cluster-nlb](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_security_group.additional_eks_cluster_sg](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource |
| [aws_security_group.all_worker_mgmt](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource |
| [aws_security_group.extra_cluster_sg](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource |
| [aws_security_group_rule.allow_sidecar_injection](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource |
| [aws_vpc_security_group_egress_rule.additional](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_security_group_egress_rule) | resource |
| [aws_vpc_security_group_ingress_rule.additional](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_security_group_ingress_rule) | resource |
| [aws_vpc_security_group_ingress_rule.additional_ingress_rules_2](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_security_group_ingress_rule) | resource |
| [null_resource.git_version](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) | resource |
| [terraform_data.subnet_validation](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource |
| [aws_arn.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/arn) | data source |
| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
| [aws_ebs_default_kms_key.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ebs_default_kms_key) | data source |
| [aws_iam_policy.cluster_managed_policies](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy) | data source |
| [aws_iam_policy_document.allow_sts](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.cloudwatch-policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.cluster-admin-policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.eks_assume](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.nlb-policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_roles.roles](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_roles) | data source |
| [aws_iam_roles.sso_admins](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_roles) | data source |
| [aws_iam_roles.sso_read](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_roles) | data source |
| [aws_iam_session_context.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_session_context) | data source |
| [aws_kms_key.ebs_key](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/kms_key) | data source |
| [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |
| [aws_subnet.subnets](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnet) | data source |
| [aws_subnets.subnets](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnets) | data source |
| [aws_vpc.eks_vpc](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc) | data source |
Expand All @@ -154,9 +173,9 @@ efs-csi-controller 0 5m
| <a name="input_cluster_version"></a> [cluster\_version](#input\_cluster\_version) | Kubernetes version to use for the EKS cluster | `string` | n/a | yes |
| <a name="input_eks_instance_disk_size"></a> [eks\_instance\_disk\_size](#input\_eks\_instance\_disk\_size) | Size of the EKS node disk in GB | `number` | `80` | no |
| <a name="input_eks_instance_types"></a> [eks\_instance\_types](#input\_eks\_instance\_types) | List of EC2 instance types for the EKS node group | `list(string)` | <pre>[<br/> "t3a.medium"<br/>]</pre> | no |
| <a name="input_eks_ng_desired_size"></a> [eks\_ng\_desired\_size](#input\_eks\_ng\_desired\_size) | Desired size of the EKS node group | `number` | `4` | no |
| <a name="input_eks_ng_max_size"></a> [eks\_ng\_max\_size](#input\_eks\_ng\_max\_size) | Maximum size of the EKS node group | `number` | `15` | no |
| <a name="input_eks_ng_min_size"></a> [eks\_ng\_min\_size](#input\_eks\_ng\_min\_size) | Minimum size of the EKS node group | `number` | `4` | no |
| <a name="input_eks_ng_desired_size"></a> [eks\_ng\_desired\_size](#input\_eks\_ng\_desired\_size) | Desired size of the EKS node group | `number` | `2` | no |
| <a name="input_eks_ng_max_size"></a> [eks\_ng\_max\_size](#input\_eks\_ng\_max\_size) | Maximum size of the EKS node group | `number` | `2` | no |
| <a name="input_eks_ng_min_size"></a> [eks\_ng\_min\_size](#input\_eks\_ng\_min\_size) | Minimum size of the EKS node group | `number` | `2` | no |
| <a name="input_enable_cluster_creator_admin_permissions"></a> [enable\_cluster\_creator\_admin\_permissions](#input\_enable\_cluster\_creator\_admin\_permissions) | Grant admin permissions to the cluster creator | `bool` | `true` | no |
| <a name="input_subnets_name"></a> [subnets\_name](#input\_subnets\_name) | Name pattern for subnets to be used by EKS cluster | `string` | `"*-container-*"` | no |
| <a name="input_tags"></a> [tags](#input\_tags) | Additional tags to apply to all resources | `map(string)` | `{}` | no |
Expand Down
2 changes: 2 additions & 0 deletions aws_data.tf
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
data "aws_caller_identity" "current" {}

data "aws_region" "current" {}

data "aws_arn" "current" {
arn = data.aws_caller_identity.current.arn
}
Expand Down
138 changes: 138 additions & 0 deletions cluster-admin.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
#---
# cluster-admin
#---
locals {
iam_arn = format("arn:%v:iam::%v:%%v", data.aws_arn.current.partition, data.aws_caller_identity.current.account_id)
common_arn = format("arn:%v:%%v:%v:%v:%%v", data.aws_arn.current.partition, data.aws_region.current.name, data.aws_caller_identity.current.account_id)
eks_resources = ["cluster", "addon", "nodegroup", "identityproviderconfig"]

admin_policy_statements = {
ECRRead = {
actions = [
"ecr:Describe*",
"ecr:Get*",
"ecr:ListImages",
"ecr:BatchGetImage",
"ecr:BatchCheckLayerAvailability",
"ecr:GetDownloadUrlForLayer",
]
resources = ["*"]
}
ECRWrite = {
actions = [
"ecr:BatchDeleteImage",
"ecr:CompleteLayerUpload",
"ecr:CreateRepository",
"ecr:DeleteRepository",
"ecr:InitiateLayerUpload",
"ecr:PutImage",
"ecr:UploadLayerPart"
]
resources = [format(local.common_arn, "ecr", format("repository/eks/%v/*", var.cluster_name))]
}
EKSRead = {
actions = [
"eks:ListClusters",
"eks:ListAddons",
"eks:ListNodegroups",
"eks:DescribeCluster",
"eks:DescribeAddon*",
"eks:DescribeNodegroup",
]
resources = [
format(local.common_arn, "eks", "cluster/*"),
format(local.common_arn, "eks", "addon/*"),
format(local.common_arn, "eks", "addons/*"),
format(local.common_arn, "eks", "/addons/*"),
format(local.common_arn, "eks", "nodegroup/*"),
]
}
IAMRead = {
actions = [
"iam:ListRoles",
]
resources = ["*"]
}
SSMGet = {
actions = [
"ssm:GetParameter",
]
resources = [
format("arn:%v:%v:%v:%v:%v", data.aws_arn.current.partition, "ssm", data.aws_region.current.name, "", "parameter/aws/service/eks/*")
]
}
EKSReadMyClusters = {
actions = [
"eks:List*",
"eks:Read*",
"eks:Describe*",
"eks:AccessKubernetesApi",
]
resources = flatten(concat(
[format(local.common_arn, "eks", format("/clusters/%v/addons", var.cluster_name))],
[for r in local.eks_resources : [format(local.common_arn, "eks", format("%v/%v", r, var.cluster_name)),
format(local.common_arn, "eks", format("%v/%v/*", r, var.cluster_name))]]
))
}
}
}

resource "aws_iam_role" "role_cluster-admin" {
name = format("%v%v-cluster-admin", local.prefixes["eks"], var.cluster_name)
description = "SAML EKS Cluster Admin Role for ${var.cluster_name}"

assume_role_policy = data.aws_iam_policy_document.allow_sts.json
force_detach_policies = true
tags = var.tags
}

resource "aws_iam_policy_attachment" "cluster-admin-attach" {
name = format("%v%v-cluster-admin-attach", local.prefixes["eks"], var.cluster_name)
policy_arn = aws_iam_policy.cluster-admin-policy.arn
roles = [aws_iam_role.role_cluster-admin.name]
}

#---
# cluster admin policy
#---
resource "aws_iam_policy" "cluster-admin-policy" {
name = format("%v%v-cluster-admin", local.prefixes["eks-policy"], var.cluster_name)
path = "/"
description = "Allow for administration of the cluster ${var.cluster_name} using AWS resources"
policy = data.aws_iam_policy_document.cluster-admin-policy.json

tags = merge(
local.base_tags,
var.tags
)
}

data "aws_iam_policy_document" "cluster-admin-policy" {
dynamic "statement" {
for_each = local.admin_policy_statements
iterator = s
content {
sid = format("%v%vAccess", lookup(s.value, "effect", "Allow"), s.key)
effect = lookup(s.value, "effect", "Allow")
actions = lookup(s.value, "actions", [])
resources = lookup(s.value, "resources", [])
}
}
}

#---
# cluster admin assume policy
#---
data "aws_iam_policy_document" "allow_sts" {
statement {
sid = "AllowSTSAssume"
effect = "Allow"
actions = ["sts:AssumeRole"]
principals {
type = "AWS"
identifiers = [
format(local.iam_arn, "root"),
]
}
}
}
92 changes: 92 additions & 0 deletions cluster-role.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
#---
# cluster
#---
locals {
cluster_managed_policy_list = [
"AmazonEKSClusterPolicy",
"AmazonEC2FullAccess",
"CloudWatchLogsFullAccess",
]
cluster_managed_policies = [for p in data.aws_iam_policy.cluster_managed_policies : p.arn]
}

data "aws_iam_policy" "cluster_managed_policies" {
for_each = toset(local.cluster_managed_policy_list)
name = each.key
}

resource "aws_iam_policy" "nlb-policy" {
name = format("%v%v-nlb", local.prefixes["eks-policy"], var.cluster_name)
path = "/"
description = "Allow configuration of the ELB"
policy = data.aws_iam_policy_document.nlb-policy.json

}

# Q: why CreateSecurityGroup
# TBD: refine resources to limit only to eks configurations
data "aws_iam_policy_document" "nlb-policy" {
statement {
sid = "EKSNLBConfiguration"
effect = "Allow"
actions = [
"elasticloadbalancing:*",
"ec2:CreateSecurityGroup",
"ec2:Describe*",
]
resources = ["*"]
}
}

resource "aws_iam_policy" "cloudwatch-policy" {
name = format("%v%v-cloudwatch", local.prefixes["eks-policy"], var.cluster_name)
path = "/"
description = "Allow sending metric data to cloudwatch"
policy = data.aws_iam_policy_document.cloudwatch-policy.json

}

# TBD: refine resources to limit only to eks configurations
data "aws_iam_policy_document" "cloudwatch-policy" {
statement {
sid = "EKSCloudwatchMetrics"
effect = "Allow"
actions = [
"cloudwatch:PutMetricData",
]
resources = ["*"]
}
}

resource "aws_iam_role" "role_eks-cluster" {
name = format("%v%v-cluster", local.prefixes["eks"], var.cluster_name)
description = "EKS Cluster Role for ${var.cluster_name}"
assume_role_policy = data.aws_iam_policy_document.eks_assume.json
}

resource "aws_iam_role_policy_attachment" "eks-cluster-nlb" {
role = aws_iam_role.role_eks-cluster.name
policy_arn = aws_iam_policy.nlb-policy.arn
}
resource "aws_iam_role_policy_attachment" "eks-cluster-cloudwatch" {
role = aws_iam_role.role_eks-cluster.name
policy_arn = aws_iam_policy.cloudwatch-policy.arn
}
resource "aws_iam_role_policy_attachment" "eks-cluster-managed" {
for_each = toset(local.cluster_managed_policies)
role = aws_iam_role.role_eks-cluster.name
policy_arn = each.key
}

data "aws_iam_policy_document" "eks_assume" {
statement {
sid = "EKSAssumeRole"
effect = "Allow"
actions = ["sts:AssumeRole"]

principals {
type = "Service"
identifiers = ["eks.amazonaws.com"]
}
}
}
2 changes: 1 addition & 1 deletion main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ resource "terraform_data" "subnet_validation" {
}

module "cluster" {
source = "git::https://github.e.it.census.gov/SCT-Engineering/terraform-aws-eks/?ref=v20.36.0"
source = "git::https://github.e.it.census.gov/SCT-Engineering/terraform-aws-eks/?ref=v20.37.2"

access_entries = local.access_entries
cloudwatch_log_group_retention_in_days = var.cloudwatch_retention_days
Expand Down
2 changes: 1 addition & 1 deletion requirements.tf
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.14"
version = "~> 5.0"
}
null = {
source = "hashicorp/null"
Expand Down
3 changes: 3 additions & 0 deletions security_groups.tf
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ resource "aws_security_group" "additional_eks_cluster_sg" {
}
}

# once setup, you cannot change any ports here
resource "aws_security_group" "all_worker_mgmt" {
name = local.all_worker_mgmt_name

Expand Down Expand Up @@ -73,6 +74,8 @@ resource "aws_security_group" "all_worker_mgmt" {
}
}

# once setup, you cannot change any ports here
# attach to cluster create, nodegroups
resource "aws_security_group" "extra_cluster_sg" {
name = format("%v%v-extra", local.prefixes["eks-security-group"], var.cluster_name)
description = format("Security group for additional access for EKS cluster %v", var.cluster_name)
Expand Down
Loading

0 comments on commit 2dc2eda

Please sign in to comment.