Skip to content

Commit

Permalink
Merge pull request #19 from SCT-Engineering/no-kubectl
Browse files Browse the repository at this point in the history
No kubectl
  • Loading branch information
morga471 committed Jan 27, 2025
2 parents 5ae71ea + bf4e9c3 commit ff29aae
Show file tree
Hide file tree
Showing 23 changed files with 515 additions and 153 deletions.
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ repos:
- id: trailing-whitespace
args: [--markdown-linebreak-ext=md]
exclude: CHANGELOG.md
- id: check-yaml
# - id: check-yaml
- id: check-merge-conflict
- id: check-executables-have-shebangs

Expand Down
37 changes: 7 additions & 30 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,23 +8,12 @@ The module configures an eks cluster with additional components after the cluste
- A gp2-encrypted storage class is created
- An efs storage class is created
- This storage class is returned as the `rwx_storage_class`
- cluster-autoscaler
- Monitors cpu/memory usage by components being deployed to the clsuter.
- When the cluster is out of capacity, the autoscaler will add nodes to the cluster up to the maximum size of the node group.
- When the autoscaler detects excess capacity, the autoscaler will cause EKS to migrate pods running on the node to a different node and then remove the node from the cluster.

When selecting which version of the cluster-autoscaler to install, use the one that is closest to the same version as the `cluster_version` or slightly higher.
The cluster-autoscaler uses the kuberentes scheduling algorithm to appropriate scale the cluster, and if the autoscaler version sufficiently divirges from the kubernetes version, the autoscaler will scale the node groups incorrectly.

A successful apply will yield:
Apply complete! Resources: 14 added, 0 changed, 0 destroyed.
Outputs:
kubectl_image_full_path = "224384469011.dkr.ecr.us-gov-east-1.amazonaws.com/platform-eng-eks-mcm/kubectl:1.27.14"
kubectl_image_registry = "224384469011.dkr.ecr.us-gov-east-1.amazonaws.com"
kubectl_image_repository = "platform-eng-eks-mcm/kubectl"
kubectl_image_tag = "1.27.14"
module_name = "tfmod-eks-configuration"
module_version = "0.0.2"
module_version = "0.1.1"
rwo_storage_class = "gp3-encrypted"
rwx_storage_class = "efs"
real 2m4.048s
Expand All @@ -44,51 +33,43 @@ sys 0m2.015s
|------|---------|
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.13 |
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 5.14.0 |
| <a name="requirement_http"></a> [http](#requirement\_http) | >= 2.1.0 |
| <a name="requirement_kubectl"></a> [kubectl](#requirement\_kubectl) | >= 1.14.0 |
| <a name="requirement_helm"></a> [helm](#requirement\_helm) | >= 2.11.0 |
| <a name="requirement_kubernetes"></a> [kubernetes](#requirement\_kubernetes) | >= 2.23.0 |

## Providers

| Name | Version |
|------|---------|
| <a name="provider_aws"></a> [aws](#provider\_aws) | 5.73.0 |
| <a name="provider_http"></a> [http](#provider\_http) | 3.4.5 |
| <a name="provider_kubectl"></a> [kubectl](#provider\_kubectl) | 1.14.0 |
| <a name="provider_kubernetes"></a> [kubernetes](#provider\_kubernetes) | 2.33.0 |
| <a name="provider_aws"></a> [aws](#provider\_aws) | 5.84.0 |
| <a name="provider_helm"></a> [helm](#provider\_helm) | 2.17.0 |
| <a name="provider_kubernetes"></a> [kubernetes](#provider\_kubernetes) | 2.35.1 |

## Modules

| Name | Source | Version |
|------|--------|---------|
| <a name="module_efs"></a> [efs](#module\_efs) | git@github.e.it.census.gov:terraform-modules/aws-efs.git | master |
| <a name="module_images"></a> [images](#module\_images) | git@github.e.it.census.gov:terraform-modules/aws-ecr-copy-images.git/ | tf-upgrade |

## Resources

| Name | Type |
|------|------|
| [kubectl_manifest.deploy_cluster_roles](https://registry.terraform.io/providers/gavinbunney/kubectl/latest/docs/resources/manifest) | resource |
| [helm_release.console_access](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) | resource |
| [kubernetes_namespace.operators](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/namespace) | resource |
| [kubernetes_storage_class.ebs_encrypted](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/storage_class) | resource |
| [kubernetes_storage_class.efs_sc](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/storage_class) | resource |
| [kubernetes_storage_class.gp3_encrypted](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/storage_class) | resource |
| [aws_ebs_default_kms_key.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ebs_default_kms_key) | data source |
| [aws_kms_key.ebs_key](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/kms_key) | data source |
| [http_http.cluster_roles](https://registry.terraform.io/providers/hashicorp/http/latest/docs/data-sources/http) | data source |
| [kubectl_file_documents.access_documents](https://registry.terraform.io/providers/gavinbunney/kubectl/latest/docs/data-sources/file_documents) | data source |

## Inputs

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| <a name="input_cluster_name"></a> [cluster\_name](#input\_cluster\_name) | EKS cluster name name component used through out the EKS cluster describing its purpose (ex: dice-dev) | `string` | n/a | yes |
| <a name="input_eks_managed_node_groups_autoscaling_group_names"></a> [eks\_managed\_node\_groups\_autoscaling\_group\_names](#input\_eks\_managed\_node\_groups\_autoscaling\_group\_names) | List of the autoscaling group names created by EKS managed node groups | `list(string)` | n/a | yes |
| <a name="input_kubectl_image_tag"></a> [kubectl\_image\_tag](#input\_kubectl\_image\_tag) | The version of bitnami/kubectl image to use. | `string` | `"1.30.4"` | no |
| <a name="input_oidc_provider_arn"></a> [oidc\_provider\_arn](#input\_oidc\_provider\_arn) | The ARN of the OIDC Provider if `enable_irsa = true` | `string` | n/a | yes |
| <a name="input_operators_ns"></a> [operators\_ns](#input\_operators\_ns) | Namespace to create where operators will be installed. | `string` | `"operators"` | no |
| <a name="input_profile"></a> [profile](#input\_profile) | AWS config profile | `string` | `""` | no |
| <a name="input_region"></a> [region](#input\_region) | AWS region | `string` | n/a | yes |
| <a name="input_release_version"></a> [release\_version](#input\_release\_version) | The version of helm charts to use | `string` | n/a | yes |
| <a name="input_security_group_all_worker_mgmt_id"></a> [security\_group\_all\_worker\_mgmt\_id](#input\_security\_group\_all\_worker\_mgmt\_id) | The security group representing all of the worker nodes in the cluster. | `string` | n/a | yes |
| <a name="input_subnets"></a> [subnets](#input\_subnets) | Specify the subnets used by this cluster | `list(string)` | n/a | yes |
| <a name="input_tag_costallocation"></a> [tag\_costallocation](#input\_tag\_costallocation) | Tag CostAllocation (default) | `string` | `"csvd:infrastructure"` | no |
Expand All @@ -99,10 +80,6 @@ sys 0m2.015s

| Name | Description |
|------|-------------|
| <a name="output_kubectl_image_full_path"></a> [kubectl\_image\_full\_path](#output\_kubectl\_image\_full\_path) | The full URI to access the kubectl image including the registry/repository:tag |
| <a name="output_kubectl_image_registry"></a> [kubectl\_image\_registry](#output\_kubectl\_image\_registry) | The registry portion of the URI to access the kubectl image |
| <a name="output_kubectl_image_repository"></a> [kubectl\_image\_repository](#output\_kubectl\_image\_repository) | The repository portion of the URI to access the kubectl image |
| <a name="output_kubectl_image_tag"></a> [kubectl\_image\_tag](#output\_kubectl\_image\_tag) | The tag portion of the URI to access the kubectl image |
| <a name="output_module_name"></a> [module\_name](#output\_module\_name) | The name of this module. |
| <a name="output_module_version"></a> [module\_version](#output\_module\_version) | The version of this module. |
| <a name="output_rwo_storage_class"></a> [rwo\_storage\_class](#output\_rwo\_storage\_class) | Kubernetes storage class that supports read/write once. |
Expand Down
5 changes: 5 additions & 0 deletions aws_data.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
data "aws_ebs_default_kms_key" "current" {}

data "aws_kms_key" "ebs_key" {
key_id = data.aws_ebs_default_kms_key.current.key_arn
}
24 changes: 2 additions & 22 deletions copy_images.tf → copy_images.tf.off
Original file line number Diff line number Diff line change
@@ -1,17 +1,7 @@
locals {
# autoscaler_key = format("%v#%v", "cluster-autoscaler", var.cluster_autoscaler_tag)
kubectl_key = format("%v#%v", "kubectl", var.kubectl_image_tag)

image_config = [
# {
# enabled = true
# dest_path = null
# name = "cluster-autoscaler"
# source_image = "autoscaling/cluster-autoscaler"
# source_registry = "registry.k8s.io"
# source_tag = null
# tag = var.cluster_autoscaler_tag
# },
{
enabled = true
dest_path = null
Expand All @@ -20,7 +10,7 @@ locals {
source_registry = "docker.io"
source_tag = var.kubectl_image_tag
tag = var.kubectl_image_tag
},
}
]
}

Expand All @@ -32,18 +22,8 @@ module "images" {
image_config = local.image_config
tags = {}

### optional
## account_alias = ""
## account_id = ""
## destination_password = ""
## destination_username = ""
## override_prefixes = {}
## region = ""
## source_password = ""
## source_username = ""

enable_lifecycle_policy = true
lifecycle_policy_all = true
force_delete = true

lifecycle_policy_keep_count = 3
}
23 changes: 23 additions & 0 deletions eks-console-full-access/.helmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*.orig
*~
# Various IDEs
.project
.idea/
*.tmproj
.vscode/
21 changes: 21 additions & 0 deletions eks-console-full-access/Chart.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
apiVersion: v2
name: eks-console-full-access
description: A Helm chart for Kubernetes to install aws console access for eks
# A chart can be either an 'application' or a 'library' chart.
#
# Application charts are a collection of templates that can be packaged into versioned archives
# to be deployed.
#
# Library charts provide useful utilities or functions for the chart developer. They're included as
# a dependency of application charts to inject those utilities and functions into the rendering
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
type: application
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.1.0
# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
# It is recommended to use it with quotes.
appVersion: "0.1.0"
62 changes: 62 additions & 0 deletions eks-console-full-access/templates/_helpers.tpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
{{/*
Expand the name of the chart.
*/}}
{{- define "eks-console-full-access.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end }}

{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "eks-console-full-access.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.nameOverride }}
{{- if contains $name .Release.Name }}
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{- end }}

{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "eks-console-full-access.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}

{{/*
Common labels
*/}}
{{- define "eks-console-full-access.labels" -}}
helm.sh/chart: {{ include "eks-console-full-access.chart" . }}
{{ include "eks-console-full-access.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}

{{/*
Selector labels
*/}}
{{- define "eks-console-full-access.selectorLabels" -}}
app.kubernetes.io/name: {{ include "eks-console-full-access.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}

{{/*
Create the name of the service account to use
*/}}
{{- define "eks-console-full-access.serviceAccountName" -}}
{{- if .Values.serviceAccount.create }}
{{- default (include "eks-console-full-access.fullname" .) .Values.serviceAccount.name }}
{{- else }}
{{- default "default" .Values.serviceAccount.name }}
{{- end }}
{{- end }}
15 changes: 15 additions & 0 deletions eks-console-full-access/templates/binding-rbac.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: {{ include "eks-console-full-access.fullname" . }}-binding
labels:
{{- include "eks-console-full-access.labels" . | nindent 4 }}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: '{{ include "eks-console-full-access.fullname" . }}-clusterrole'
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: eks-console-dashboard-full-access-group
namespace: '{{ .Release.Namespace }}'
107 changes: 107 additions & 0 deletions eks-console-full-access/templates/clusterrole-rbac.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: {{ include "eks-console-full-access.fullname" . }}-clusterrole
labels:
{{- include "eks-console-full-access.labels" . | nindent 4 }}
rules:
- apiGroups:
- ""
resources:
- nodes
- namespaces
- pods
- configmaps
- endpoints
- events
- limitranges
- persistentvolumeclaims
- podtemplates
- replicationcontrollers
- resourcequotas
- secrets
- serviceaccounts
- services
verbs:
- get
- list
- apiGroups:
- apps
resources:
- deployments
- daemonsets
- statefulsets
- replicasets
verbs:
- get
- list
- apiGroups:
- batch
resources:
- jobs
- cronjobs
verbs:
- get
- list
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- get
- list
- apiGroups:
- discovery.k8s.io
resources:
- endpointslices
verbs:
- get
- list
- apiGroups:
- events.k8s.io
resources:
- events
verbs:
- get
- list
- apiGroups:
- extensions
resources:
- daemonsets
- deployments
- ingresses
- networkpolicies
- replicasets
verbs:
- get
- list
- apiGroups:
- networking.k8s.io
resources:
- ingresses
- networkpolicies
verbs:
- get
- list
- apiGroups:
- policy
resources:
- poddisruptionbudgets
verbs:
- get
- list
- apiGroups:
- rbac.authorization.k8s.io
resources:
- rolebindings
- roles
verbs:
- get
- list
- apiGroups:
- storage.k8s.io
resources:
- csistoragecapacities
verbs:
- get
- list
1 change: 1 addition & 0 deletions eks-console-full-access/values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
kubernetesClusterDomain: cluster.local
Loading

0 comments on commit ff29aae

Please sign in to comment.