Skip to content

Commit

Permalink
- update example code 1.24 and 1.25 to rmeove AmazonS3FullAccess
Browse files Browse the repository at this point in the history
  • Loading branch information
badra001 committed Oct 5, 2023
1 parent 644f35c commit b8d1bf3
Show file tree
Hide file tree
Showing 6 changed files with 136 additions and 8 deletions.
7 changes: 4 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,13 @@

## Version 1.x

* v1.0.0 -- 2021-10-14
* 1.0.0 -- 2021-10-14
- patch-aws-auth module creation

## Version 2.x

* v2.0.0 -- 20211223
* 2.0.0 -- 2021-12-23
- add providers for tf 0.13+


* 2.0.1 -- 2023-10-05
- update example code 1.24 and 1.25 to rmeove AmazonS3FullAccess
2 changes: 1 addition & 1 deletion common/version.tf
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
locals {
_module_version = "2.0.0"
_module_version = "2.0.1"
}
2 changes: 1 addition & 1 deletion examples/full-cluster-tf-upgrade/1.24/role.tf
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ locals {
"AmazonEC2ContainerRegistryPowerUser",
"AmazonEC2ContainerRegistryReadOnly",
"CloudWatchLogsFullAccess",
"AmazonS3FullAccess",
# "AmazonS3FullAccess",
"AmazonSSMManagedInstanceCore",
"AmazonEC2RoleforSSM",
]
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
# Extras :: cloudwatch-agent

The configuration in this dierectory will deploy cloudwatch-agent and fluentbit, to be used for EKS Container Insights.

# Links

* AWS Docs
* https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-prerequisites.html
* https://aws.amazon.com/blogs/opensource/centralized-container-logging-fluent-bit/
* https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/ContainerInsights-use-kubelet.html
* https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-setup-logs-FluentBit.html
* https://aws.github.io/eks-charts"
* Cloudwatch Agnet
* https://github.com/aws/eks-charts/tree/master/stable/aws-cloudwatch-metrics
* Fluent Bit
* https://github.com/aws/aws-for-fluent-bit
* https://github.com/aws/eks-charts/tree/master/stable/aws-for-fluent-bit

# Supported Versions

This configuration has been tested and validated on EKS versions

* 1.24
* 1.25

# Configuration

This uses a helm chart, an IRSA role, and pulls the latest images at the time of creating this module.
Look in the `variables.*.auto.tfvars` files for the version numbers.

# Installation

You will need the latest copy of the `aws-eks` module, using the `tf-upgrade` branch. This requires the use of
Terraform 1.x, and as it is deployed in a subdirectory, it should work without issue.

## Step 1: Get aws-eks repo

If you do not have the `aws-eks` repo, clone it in the branch `tf-upgrade`.

```script
# go to your TF repository directory
cd $PATH_TO_TERRAFORM
git clone git@github.e.it.census.gov:terraform-modules/aws-eks.git -b tf-upgrade
cd aws-eks
export EKS_SOURCE=$(pwd)
```

If you already have the repo, go into the directory, checkout the branch and refresh it.

```script
# go to your TF repository directory
cd $PATH_TO_TERRAFORM
cd aws-eks
git checkout tf-upgrade
git pull origin tf-upgrade
export EKS_SOURCE=$(pwd)
```

## Step 2: Copy code

Go into the `common-services` directory of the EKS cluster where you wish to deploy this. Make a directory, `cloudwatch-agent`, and then
rsync the code. Please use rsync, not copy. There is a directory, and there may be softlinks. You'll work in a new branch. An example is below:

```script
cd $PATH_TO_TERRAFORM
cd 107742151971-do2-govcloud/vpc/east/vpc5/apps/eks-ditd-gups-stage/common-services
mkdir cloudwatch-agent
cd cloudwatch-agent
git checkout -b add-cloudwatch-agent
rsync -avRWH $EKS_SOURCE/examples/extra/cloudwatch-agent/./ ./
```

## Step 3: Plan

There is no configuration needed. All relevant details are pulled from the parent directories. You do need EKS cluster access,
so be sure you are running with a user who has K8S RBAC access.

```script
tf-run plan
tf-plan summary
# add to git
git add .
git commit -m 'add cloudwatch, fluentbit' .
git push
# submit PR with plan summary and plan log
```

## Step 4: Apply

Once the PR is merged, apply, and finalize the directory.

```script
tf-run apply
```

Make sure it started up:

```console
% kubectl --kubeconfig setup/kube.config get pods -n aws-cloudwatch
NAME READY STATUS RESTARTS AGE
aws-cloudwatch-metrics-8jlwh 1/1 Running 0 24h
aws-cloudwatch-metrics-8jxqs 1/1 Running 0 24h
aws-cloudwatch-metrics-k668c 1/1 Running 0 24h
fluent-bit-aws-for-fluent-bit-6bvgk 1/1 Running 0 24h
fluent-bit-aws-for-fluent-bit-b4hk5 1/1 Running 0 24h
fluent-bit-aws-for-fluent-bit-chx46 1/1 Running 0 24h
```

All should be running. If any errors, or not running, look at `events` and `logs`.

Then, check AWS CloudWatch Logs. There will be four log as follows: /aws/containerinsights/{clustername}/{name}
where {name} is

* performance
* host
* applications
* dataplane

The Container Insight dashboard should also show performance data for the cluster, though it may take some
time to appear.


# CHANGELOG

* 1.0.0 -- 2023-08-24
- initial
2 changes: 1 addition & 1 deletion examples/full-cluster-tf-upgrade/1.25/role.tf
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ locals {
"AmazonEC2ContainerRegistryPowerUser",
"AmazonEC2ContainerRegistryReadOnly",
"CloudWatchLogsFullAccess",
"AmazonS3FullAccess",
# "AmazonS3FullAccess",
"AmazonSSMManagedInstanceCore",
"AmazonEC2RoleforSSM",
]
Expand Down
4 changes: 2 additions & 2 deletions examples/full-cluster-tf-upgrade/1.25/securitygroup.tf
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ resource "aws_security_group" "extra_cluster_sg" {
from_port = 0
to_port = 0
protocol = -1
self = true
self = true
}

ingress {
Expand All @@ -153,7 +153,7 @@ resource "aws_security_group" "extra_cluster_sg" {
cidr_blocks = concat(var.census_private_cidr, ["10.0.0.0/8"])
}

# kubectl logs
# kubectl logs
ingress {
from_port = 10250
to_port = 10250
Expand Down

0 comments on commit b8d1bf3

Please sign in to comment.