tfmod-loki

This module installs Grafana Loki as a log aggregation and storage solution in an EKS cluster, with the following components:

Deploys Loki using the official Grafana Helm chart
Creates an S3 bucket for persistent log storage
Configures IAM roles for service accounts (IRSA) to securely access S3
Sets up internal gateway for log queries and ingestion

Architecture

The module sets up:

A Loki deployment via Helm with configurable image versions
An S3 bucket with KMS encryption for log persistence
An IRSA role for Loki to access the S3 bucket securely
Internal gateway service (loki-gateway.{namespace}.svc.cluster.local) for accessing Loki within the cluster

Prerequisites

An existing EKS cluster with OIDC provider configured
Sufficient node storage - nodes should have more than 40GB disk space
AWS S3 access for log storage
Appropriate Kubernetes storage classes configured

Usage

module "loki" {
  source = "git@github.e.it.census.gov:path/to/tfmod-loki.git"

  cluster_name      = "my-eks-cluster"
  oidc_provider_arn = module.eks.oidc_provider_arn
  region            = "us-east-1"
  namespace         = "monitoring"

  # Optional - override default image versions
  loki_tag         = "3.1.1"
  gateway_tag      = "1.25.2-alpine"

  tags = {
    Environment = "production"
    Team        = "platform"
  }
}

# Access Loki internal endpoint
resource "kubernetes_manifest" "example_grafana_datasource" {
  manifest = {
    # Configure Grafana datasource to point to:
    # ${module.loki.gateway_internal_endpoint}
  }
}

Requirements

Name	Version
terraform	>= 0.13
aws	>= 5.14.0
helm	>= 2.11.0
kubernetes	>= 2.23.0
null	>= 3.2.1

Providers

Name	Version
aws	6.0.0
aws.eecr	6.0.0
helm	3.0.2
null	3.2.4
terraform	n/a

Modules

Name	Source	Version
images	git::https://github.e.it.census.gov/terraform-modules/aws-ecr-copy-images.git	tf-upgrade
loki_irsa_role	git::https://github.e.it.census.gov/SCT-Engineering/tfmod-custom-iam-role-for-service-account-eks.git	main
loki_s3	git::https://github.e.it.census.gov/terraform-modules/aws-s3.git//standard	tf-upgrade

Resources

Name	Type
helm_release.loki	resource
null_resource.git_version	resource
terraform_data.bucket_name_validator	resource
aws_caller_identity.current	data source
aws_ecr_authorization_token.ecr_token	data source
aws_ecr_authorization_token.token	data source
aws_s3_bucket.s3_server_access_logs	data source

Inputs

Name	Description	Type	Default	Required
account_id	aws account number	`string`	`""`	no
cluster_name	EKS cluster name name component used through out the EKS cluster describing its purpose (ex: dice-dev)	`string`	n/a	yes
eecr_info	Enterprise ECR source information	object({ account_id = string alias = string profile = string region = string })	{ "account_id": "269222635945", "alias": "lab-gov-shared-nonprod", "profile": "269222635945-lab-gov-shared-nonprod", "region": "us-gov-east-1" }	no
enterprise_logs_provisioner_tag	The version of the grafana/enterprise-logs-provisioner image to use.	`string`	`"v1.7.0"`	no
exporter_tag	The version of prom/memcached-exporter to use for the gateway.	`string`	`"v0.14.4"`	no
gateway_tag	The version of nginxinc/nginx-unprivileged to use for the gateway.	`string`	`"1.25.2-alpine"`	no
loki_chart_version	Which version of the grafana/loki helm chart to use.	`string`	`"6.10.2"`	no
loki_tag	The tag of the loki image to use.	`string`	`"3.1.1"`	no
memcached_tag	The version of memcached to use for the gateway.	`string`	`"1.6.23-alpine"`	no
namespace	The namespace into which grafana will be deployed	`string`	`"loki"`	no
oidc_provider_arn	The ARN in the EKS cluster for the OpenID Connect identity provider.	`string`	n/a	yes
profile	AWS config profile used to upload images into ECR	`string`	`""`	no
region	The region holding these resources (for the s3 bucket.)	`string`	n/a	yes
rwo_storage_class	Specify the storage class for read/write/once persistent volumes.	`string`	`"gp3-encrypted"`	no
sidecar_tag	The version of kiwigrid/k8s-sidecar to use for the gateway.	`string`	`"1.27.4"`	no
tags	Additional tags to add to resources created in AWS (s3 bucket, ...)	`map(string)`	`{}`	no

Outputs

Name	Description
gateway_internal_endpoint	The endpoint which can be used inside of the cluster to access loki-gateway.
image_config	Map of image references to their full source paths
module_name	The name of this module.
module_version	The version of this module.
rwo_storage_class	Specify the storage class for read/write/once persistent volumes.

SCT-Engineering/tfmod-loki

About

Resources

Stars

Watchers

Forks

Releases 6

Contributors 4

Languages