Skip to content

Helm Refactor + autoscaling #11

Merged
merged 35 commits into from
Mar 17, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
8ef9312
autoscaling
morga471 Feb 28, 2025
0a89848
use bitnami image:
morga471 Mar 4, 2025
14b68eb
lower requests
morga471 Mar 7, 2025
98921ea
refactor chart values
morga471 Mar 7, 2025
fecabc9
cleanup
morga471 Mar 7, 2025
a5e645a
kill the canary
morga471 Mar 7, 2025
46c2792
remove canary
morga471 Mar 7, 2025
54f31cf
unified config
morga471 Mar 7, 2025
c1d791b
fix values
morga471 Mar 7, 2025
6cbf1ac
template error
morga471 Mar 7, 2025
67dde52
remove extras
morga471 Mar 7, 2025
1a7899d
add both storage configs
morga471 Mar 8, 2025
3cca6ca
add some back
morga471 Mar 8, 2025
de5e041
add back required images
morga471 Mar 8, 2025
8f12610
add cleanup_on_fail
morga471 Mar 8, 2025
ec09af8
add replace true
morga471 Mar 8, 2025
895316f
add path_prefix
morga471 Mar 8, 2025
ae2e1b3
give more mem to backend and write
morga471 Mar 8, 2025
3a683f1
sort
morga471 Mar 8, 2025
db2b7a4
update requests resources
morga471 Mar 10, 2025
addd092
set timeout to 10m
morga471 Mar 10, 2025
8a9b621
more values
morga471 Mar 10, 2025
c53d619
more values
morga471 Mar 10, 2025
86dbea9
shorter timeout again
morga471 Mar 10, 2025
944d180
fix config parsing
morga471 Mar 10, 2025
54063d9
fix schema
morga471 Mar 10, 2025
0cfce2a
add s3 prefix
morga471 Mar 10, 2025
9acc3cb
no thanos
morga471 Mar 10, 2025
3aeb21f
more testing
morga471 Mar 10, 2025
21006b4
delete store fix
morga471 Mar 10, 2025
913cba6
fix volumes
morga471 Mar 10, 2025
ba8a190
update s3
morga471 Mar 10, 2025
afd94ea
guess
morga471 Mar 11, 2025
4a00ac9
less is more
morga471 Mar 11, 2025
08ba8d5
increase requests
morga471 Mar 12, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 101 additions & 0 deletions .github/workflows/terragrunt-cicd.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
name: 'Terraform Module CI'

on:
push:
branches:
- main
paths:
- '**/*.hcl'
- '**/*.tf'
pull_request:
branches:
- main
paths:
- '**/*.hcl'
- '**/*.tf'

permissions:
contents: read
pull-requests: write

jobs:
validate:
name: 'Validate Module'
runs-on: self-hosted

steps:
- name: Checkout
uses: actions/checkout@v3

- name: Setup Terraform
uses: hashicorp/setup-terraform@v2
with:
terraform_version: 1.5.0

- name: Terraform Init
run: |
terraform init -backend=false
- name: Terraform Format
run: |
terraform fmt -check
- name: Terraform Validate
run: |
terraform validate
- name: Run tflint
uses: terraform-linters/setup-tflint@v3
if: github.event_name == 'pull_request'

- name: Lint Terraform
if: github.event_name == 'pull_request'
run: |
tflint --format compact
release:
name: 'Create Release'
needs: validate
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
runs-on: self-hosted
permissions:
contents: write

steps:
- name: Checkout
uses: actions/checkout@v3
with:
fetch-depth: 0
token: ${{ secrets.GITHUB_TOKEN }}

- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: '3.9'

- name: Install Commitizen
run: |
pip install commitizen
- name: Configure Git
run: |
git config --local user.email "action@github.com"
git config --local user.name "GitHub Action"
- name: Bump Version and Generate Changelog
id: cz
run: |
cz bump --yes
echo "new_version=$(cz version --project)" >> $GITHUB_OUTPUT
echo "changelog=$(cz changelog --dry-run)" >> $GITHUB_OUTPUT
- name: Create Release
uses: actions/create-release@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: v${{ steps.cz.outputs.new_version }}
release_name: Release v${{ steps.cz.outputs.new_version }}
draft: false
prerelease: false
body: ${{ steps.cz.outputs.changelog }}
8 changes: 2 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,32 +25,29 @@ to loki.

| Name | Version |
|------|---------|
| <a name="provider_aws"></a> [aws](#provider\_aws) | 5.87.0 |
| <a name="provider_aws"></a> [aws](#provider\_aws) | 5.89.0 |
| <a name="provider_helm"></a> [helm](#provider\_helm) | 2.17.0 |
| <a name="provider_kubernetes"></a> [kubernetes](#provider\_kubernetes) | 2.35.1 |

## Modules

| Name | Source | Version |
|------|--------|---------|
| <a name="module_images"></a> [images](#module\_images) | git@github.e.it.census.gov:terraform-modules/aws-ecr-copy-images.git/ | tf-upgrade |
| <a name="module_loki_irsa_role"></a> [loki\_irsa\_role](#module\_loki\_irsa\_role) | git@github.e.it.census.gov:SCT-Engineering/tfmod-custom-iam-role-for-service-account-eks.git | n/a |
| <a name="module_loki_irsa_role"></a> [loki\_irsa\_role](#module\_loki\_irsa\_role) | git@github.e.it.census.gov:SCT-Engineering/tfmod-custom-iam-role-for-service-account-eks.git// | main |
| <a name="module_loki_s3"></a> [loki\_s3](#module\_loki\_s3) | git@github.e.it.census.gov:terraform-modules/aws-s3.git//standard | tf-upgrade |

## Resources

| Name | Type |
|------|------|
| [helm_release.loki](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) | resource |
| [kubernetes_namespace.ns](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/namespace) | resource |
| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
| [aws_s3_bucket.s3_server_access_logs](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/s3_bucket) | data source |

## Inputs

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| <a name="input_canary_tag"></a> [canary\_tag](#input\_canary\_tag) | The tag of the grafana/loki-canary image to use. | `string` | `"3.0.0"` | no |
| <a name="input_cluster_name"></a> [cluster\_name](#input\_cluster\_name) | EKS cluster name name component used through out the EKS cluster describing its purpose (ex: dice-dev) | `string` | n/a | yes |
| <a name="input_enterprise_logs_provisioner_tag"></a> [enterprise\_logs\_provisioner\_tag](#input\_enterprise\_logs\_provisioner\_tag) | The version of the grafana/enterprise-logs-provisioner image to use. | `string` | `"v1.7.0"` | no |
| <a name="input_exporter_tag"></a> [exporter\_tag](#input\_exporter\_tag) | The version of prom/memcached-exporter to use for the gateway. | `string` | `"v0.14.4"` | no |
Expand All @@ -64,7 +61,6 @@ to loki.
| <a name="input_region"></a> [region](#input\_region) | The region holding these resources (for the s3 bucket.) | `string` | n/a | yes |
| <a name="input_rwo_storage_class"></a> [rwo\_storage\_class](#input\_rwo\_storage\_class) | Specify the storage class for read/write/once persistent volumes. | `string` | `"gp3-encrypted"` | no |
| <a name="input_sidecar_tag"></a> [sidecar\_tag](#input\_sidecar\_tag) | The version of kiwigrid/k8s-sidecar to use for the gateway. | `string` | `"1.27.4"` | no |
| <a name="input_tag_costallocation"></a> [tag\_costallocation](#input\_tag\_costallocation) | Tag CostAllocation (default) | `string` | `"csvd:infrastructure"` | no |
| <a name="input_tags"></a> [tags](#input\_tags) | Additional tags to add to resources created in AWS (s3 bucket, ...) | `map(string)` | `{}` | no |

## Outputs
Expand Down
28 changes: 9 additions & 19 deletions copy_images.tf
Original file line number Diff line number Diff line change
@@ -1,37 +1,27 @@
locals {
exporter_key = format("%v#%v", "prom/memcached-exporter", var.exporter_tag)
gateway_key = format("%v#%v", "grafana/nginx-unprivileged", var.gateway_tag)
loki_key = format("%v#%v", "grafana/loki", var.loki_tag)
canary_key = format("%v#%v", "grafana/loki-canary", var.canary_tag)
memcached_key = format("%v#%v", "memcached", var.memcached_tag)
provisioner_key = format("%v#%v", "grafana/enterprise-logs-provisioner", var.enterprise_logs_provisioner_tag)
gateway_key = format("%v#%v", "grafana/nginx-unprivileged", var.gateway_tag)
sidecar_key = format("%v#%v", "kiwigrid/k8s-sidecar", var.sidecar_tag)
memcached_key = format("%v#%v", "memcached", var.memcached_tag)
exporter_key = format("%v#%v", "prom/memcached-exporter", var.exporter_tag)

image_config = [
{
enabled = true
dest_path = null
name = "grafana/loki"
source_image = "grafana/loki"
source_registry = "docker.io"
source_image = "bitnami/grafana-loki"
source_registry = "public.ecr.aws"
source_tag = var.loki_tag
tag = var.loki_tag
},
{
enabled = true
dest_path = null
name = "grafana/loki-canary"
source_image = "grafana/loki-canary"
source_registry = "docker.io"
source_tag = var.canary_tag
tag = var.canary_tag
},
{
enabled = true
dest_path = null
name = "memcached"
source_image = "memcached"
source_registry = "docker.io"
source_image = "bitnami/memcached"
source_registry = "public.ecr.aws"
source_tag = var.memcached_tag
tag = var.memcached_tag
},
Expand Down Expand Up @@ -66,8 +56,8 @@ locals {
enabled = true
dest_path = null
name = "grafana/nginx-unprivileged"
source_image = "nginxinc/nginx-unprivileged"
source_registry = "docker.io"
source_image = "nginx/nginx-unprivileged"
source_registry = "public.ecr.aws"
source_tag = var.gateway_tag
tag = var.gateway_tag
},
Expand Down
Loading
Loading