* 2.9.15 -- 2024-05-07

- flowlogs-transit-gateway - remove splunk - add outputs (for generating subscription external to module)
terraform-modules · May 7, 2024 · 4185231 · 4185231
1 parent 62e0336
commit 4185231
Show file tree

Hide file tree

Showing 5 changed files with 155 additions and 60 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -378,3 +378,8 @@
     - add cloudwatch_log_prefix, to be set when log policy > 5120, to /aws/vendedlogs per AWS documentation at
       https://docs.aws.amazon.com/step-functions/latest/dg/bp-cwl.html.  May also be possible with a cloudwatch log policy
       resource, but unclear how that affects all the other log policies listed
+
+* 2.9.15 -- 2024-05-07
+  - flowlogs-transit-gateway
+    - remove splunk
+    - add outputs (for generating subscription external to module)
diff --git a/common/version.tf b/common/version.tf
@@ -1,5 +1,5 @@
 locals {
-  _module_version = "2.9.14"
+  _module_version = "2.9.15"
   _module_names = {
     "_main_" = "aws-vpc-setup"
 

diff --git a/flowlogs-transit-gateway/README.md b/flowlogs-transit-gateway/README.md
@@ -103,3 +103,94 @@ No modules.
 | Name | Description |
 |------|-------------|
 | <a name="output_kinesis_flowlog_arn"></a> [kinesis\_flowlog\_arn](#output\_kinesis\_flowlog\_arn) | VPC Flowlog Kinesis stream ARN |
+
+<!-- BEGIN_TF_DOCS -->
+# About aws-vpc-setup :: flowlogs-transit-gateway
+
+AWS [recently announced](https://aws.amazon.com/about-aws/whats-new/2022/07/amazon-vpc-flow-logs-transit-gateway-improved-visibility-monitoring/) support for
+VPC flow logs on transt gateways and transit gateway attachmetns.
+
+This submodule creates VPC flow logs for a transit gateway.  This shoudl be used per transit gateway, one in each region
+You will need to run the flow logs role once before (aws-vpc-setup/flowlogs-role).
+
+This also creates cloudwatch logs and cloudwatch streams (Kinesis), if enabled.  Previously, files for Splunk were created. As we have removed
+Splunk from our environment, this feature no longer exists.
+
+# Usage
+
+```hcl
+module "flowlogs-transit-gateway" {
+  source               = "git@github.e.it.census.gov:terraform-modules/aws-vpc-setup.git//flowlogs-transit-gateway?ref=tf-upgrade"
+  label                = "ent-gov-sa"
+  # account_alias        = "ent-gov-network-sa"
+  transit_gateway_id   = aws_transit_gateway.gateway.id
+  flowlog_bucket_arn   = data.terraform_remote_state.common.infrastructure_east.flowlogs_arn
+  flowlog_role_arn     = data.terraform_remote_state.common.outputs.role_flowlogs_arn
+  ## optional
+  # use_flowlog_custom_format = true
+  # flowlog_custom_format = "${account-id} ${interface-id} ${srcaddr} ${dstaddr} ${srcport} ${dstport} ${protocol} ${packets} ${bytes} ${start} ${end} ${action} ${log-status} ${flow-direction} ${traffic-path}"
+
+  tags = {}
+}
+```
+
+## Requirements
+
+| Name | Version |
+|------|---------|
+| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.13 |
+| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 3.66.0 |
+| <a name="requirement_ldap"></a> [ldap](#requirement\_ldap) | >= 0.5.4 |
+| <a name="requirement_local"></a> [local](#requirement\_local) | >= 1.0.0 |
+| <a name="requirement_null"></a> [null](#requirement\_null) | >= 3.0 |
+| <a name="requirement_random"></a> [random](#requirement\_random) | >= 3.0 |
+| <a name="requirement_template"></a> [template](#requirement\_template) | >= 2.0 |
+
+## Providers
+
+| Name | Version |
+|------|---------|
+| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 3.66.0 |
+
+## Modules
+
+No modules.
+
+## Resources
+
+| Name | Type |
+|------|------|
+| [aws_cloudwatch_log_group.flowlog](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_group) | resource |
+| [aws_cloudwatch_log_subscription_filter.flowlog](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_subscription_filter) | resource |
+| [aws_flow_log.flowlog_cloudwatch](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/flow_log) | resource |
+| [aws_flow_log.flowlog_s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/flow_log) | resource |
+| [aws_kinesis_stream.flowlog](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kinesis_stream) | resource |
+| [aws_arn.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/arn) | data source |
+| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
+| [aws_iam_account_alias.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_account_alias) | data source |
+| [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |
+
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|------|---------|:--------:|
+| <a name="input_account_alias"></a> [account\_alias](#input\_account\_alias) | AWS Account Alias (default: will pull from current account\_alias) | `string` | `""` | no |
+| <a name="input_account_id"></a> [account\_id](#input\_account\_id) | AWS Account ID (default: will pull from current user) | `string` | `""` | no |
+| <a name="input_enable_kinesis"></a> [enable\_kinesis](#input\_enable\_kinesis) | Flag to enable AWS Kinesis streams for flow logs | `bool` | `true` | no |
+| <a name="input_flowlog_bucket_arn"></a> [flowlog\_bucket\_arn](#input\_flowlog\_bucket\_arn) | S3 Bucket to hold the VPC flowlogs | `string` | n/a | yes |
+| <a name="input_flowlog_custom_format"></a> [flowlog\_custom\_format](#input\_flowlog\_custom\_format) | If use\_flowlog\_custom\_format defined, use the custom format listed here. See https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs.html | `string` | `"${account-id} ${vpc-id} ${subnet-id} ${interface-id} ${start} ${end} ${pkt-srcaddr} ${srcaddr} ${srcport} ${pkt-dstaddr} ${dstaddr} ${dstport} ${protocol} ${packets} ${bytes} ${tcp-flags} ${log-status} ${action} ${type} ${flow-direction} ${traffic-path}"` | no |
+| <a name="input_flowlog_role_arn"></a> [flowlog\_role\_arn](#input\_flowlog\_role\_arn) | IAM Role with proper permissions to allow writing VPC flowlogs to cloudwatch logs and streamss | `string` | n/a | yes |
+| <a name="input_label"></a> [label](#input\_label) | Text label associated with the Transit Gateway | `string` | n/a | yes |
+| <a name="input_override_prefixes"></a> [override\_prefixes](#input\_override\_prefixes) | Override built-in prefixes by component. This should be used primarily for common infrastructure things | `map(string)` | `{}` | no |
+| <a name="input_retention_in_days"></a> [retention\_in\_days](#input\_retention\_in\_days) | Number of days to keep cloudwatch logs (default is 180). See the documentation for available values. | `number` | `180` | no |
+| <a name="input_tags"></a> [tags](#input\_tags) | AWS Tags to apply to appropriate resources (S3, KMS).  Do not include safeguard tags here, use the data\_safeguard field for such things. | `map(string)` | `{}` | no |
+| <a name="input_transit_gateway_id"></a> [transit\_gateway\_id](#input\_transit\_gateway\_id) | ID of the Transit Gateway | `string` | n/a | yes |
+| <a name="input_use_flowlog_custom_format"></a> [use\_flowlog\_custom\_format](#input\_use\_flowlog\_custom\_format) | Flag to control the use of a custom format. See https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs.html | `bool` | `false` | no |
+
+## Outputs
+
+| Name | Description |
+|------|-------------|
+| <a name="output_flowlog_log_group_arn"></a> [flowlog\_log\_group\_arn](#output\_flowlog\_log\_group\_arn) | ARN of TGW VPC Flow Log |
+| <a name="output_flowlog_log_group_name"></a> [flowlog\_log\_group\_name](#output\_flowlog\_log\_group\_name) | Name of TGW VPC Flow Log |
+<!-- END_TF_DOCS -->
diff --git a/flowlogs-transit-gateway/main.tf b/flowlogs-transit-gateway/main.tf
@@ -7,21 +7,9 @@
 * This submodule creates VPC flow logs for a transit gateway.  This shoudl be used per transit gateway, one in each region
 * You will need to run the flow logs role once before (aws-vpc-setup/flowlogs-role).
 *
-* This also creates cloudwatch logs and cloudwatch streams (Kinesis), and drops a configuration file in `setup/`
-* to be used in Splunk. Distribute this configuration to the splunk team for flowlog ingestion.
+* This also creates cloudwatch logs and cloudwatch streams (Kinesis), if enabled.  Previously, files for Splunk were created. As we have removed
+* Splunk from our environment, this feature no longer exists.
 * 
-* ```
-* # setup/aws_kinesis_tasks.lgs-tgw_ent-gov-sa_us-gov-east-1.conf
-* [ent-gov-network-sa-flowlogs-us-gov-east-1_lg_ent-gov-sa_us-gov-east-1]
-* account = 123123123123-ent-gov-network-sa
-* format = CloudWatchLogs
-* index = aws_vpc_flow_logs
-* region = us-gov-east-1
-* sourcetype = aws:cloudwatchlogs:vpcflow
-* init_stream_position = LATEST
-* stream_names = lgs-tgw_ent-gov-sa_us-gov-east-1
-* ```
-*
 * # Usage
 *
 * ```hcl
@@ -132,45 +120,46 @@ resource "aws_cloudwatch_log_subscription_filter" "flowlog" {
   distribution    = "ByLogStream"
 }
 
-#---
-# generate splunk inputs file
-#---
-data "template_file" "splunk_flowlog" {
-  count    = var.enable_kinesis ? 1 : 0
-  template = file("${path.module}/templates/aws_kinesis_tasks.conf.tpl")
-  vars = {
-    account_id          = local.account_id
-    account_alias       = local.account_alias
-    region              = local.region
-    flowlog_name        = aws_cloudwatch_log_group.flowlog.name
-    flowlog_stream_name = local.flowlog_stream_name
-    label               = var.label
-  }
-}
-
-resource "null_resource" "splunk_flowlog" {
-  count = var.enable_kinesis ? 1 : 0
-  triggers = {
-    filename  = format("aws_kinesis_tasks.%v-%v.%v.%v.conf", local.account_id, local.account_alias, local.region, local.flowlog_stream_name)
-    directory = format("%v/setup", path.root)
-  }
-  provisioner "local-exec" {
-    command = "test -d ${self.triggers.directory} || mkdir ${self.triggers.directory}"
-  }
-
-  #   provisioner "local-exec" {
-  #     working_dir = path.root
-  #     command     = "test -d setup || mkdir setup"
-  #   }
-  #   provisioner "local-exec" {
-  #     working_dir = "${path.root}/setup"
-  #     command     = "echo '${data.template_file.splunk_flowlog_tasks_flowlog.rendered}' > aws_kinesis_tasks.${local.flowlog_stream_name}.conf"
-  #   }
-}
-
-resource "local_file" "splunk_flowlog" {
-  count           = var.enable_kinesis ? 1 : 0
-  content         = var.enable_kinesis ? data.template_file.splunk_flowlog[0].rendered : ""
-  file_permission = "0644"
-  filename        = var.enable_kinesis ? format("%v/%v", null_resource.splunk_flowlog[0].triggers.directory, null_resource.splunk_flowlog[0].triggers.filename) : "__kinesis_disbaled__"
-}
+## splunk is gone, remove from code
+## #---
+## # generate splunk inputs file
+## #---
+## data "template_file" "splunk_flowlog" {
+##   count    = var.enable_kinesis ? 1 : 0
+##   template = file("${path.module}/templates/aws_kinesis_tasks.conf.tpl")
+##   vars = {
+##     account_id          = local.account_id
+##     account_alias       = local.account_alias
+##     region              = local.region
+##     flowlog_name        = aws_cloudwatch_log_group.flowlog.name
+##     flowlog_stream_name = local.flowlog_stream_name
+##     label               = var.label
+##   }
+## }
+## 
+## resource "null_resource" "splunk_flowlog" {
+##   count = var.enable_kinesis ? 1 : 0
+##   triggers = {
+##     filename  = format("aws_kinesis_tasks.%v-%v.%v.%v.conf", local.account_id, local.account_alias, local.region, local.flowlog_stream_name)
+##     directory = format("%v/setup", path.root)
+##   }
+##   provisioner "local-exec" {
+##     command = "test -d ${self.triggers.directory} || mkdir ${self.triggers.directory}"
+##   }
+## 
+##   #   provisioner "local-exec" {
+##   #     working_dir = path.root
+##   #     command     = "test -d setup || mkdir setup"
+##   #   }
+##   #   provisioner "local-exec" {
+##   #     working_dir = "${path.root}/setup"
+##   #     command     = "echo '${data.template_file.splunk_flowlog_tasks_flowlog.rendered}' > aws_kinesis_tasks.${local.flowlog_stream_name}.conf"
+##   #   }
+## }
+## 
+## resource "local_file" "splunk_flowlog" {
+##   count           = var.enable_kinesis ? 1 : 0
+##   content         = var.enable_kinesis ? data.template_file.splunk_flowlog[0].rendered : ""
+##   file_permission = "0644"
+##   filename        = var.enable_kinesis ? format("%v/%v", null_resource.splunk_flowlog[0].triggers.directory, null_resource.splunk_flowlog[0].triggers.filename) : "__kinesis_disbaled__"
+## }
diff --git a/flowlogs-transit-gateway/outputs.tf b/flowlogs-transit-gateway/outputs.tf
@@ -1,4 +1,14 @@
-output "kinesis_flowlog_arn" {
-  description = "VPC Flowlog Kinesis stream ARN"
-  value       = var.enable_kinesis ? aws_kinesis_stream.flowlog[0].arn : ""
+## output "kinesis_flowlog_arn" {
+##   description = "VPC Flowlog Kinesis stream ARN"
+##   value       = var.enable_kinesis ? aws_kinesis_stream.flowlog[0].arn : ""
+## }
+
+output "flowlog_log_group_name" {
+  description = "Name of TGW VPC Flow Log"
+  value       = aws_cloudwatch_log_group.flowlog.name
+}
+
+output "flowlog_log_group_arn" {
+  description = "ARN of TGW VPC Flow Log"
+  value       = aws_cloudwatch_log_group.flowlog.arn
 }