update tag passing, variables

README.eicar.md

@@ -0,0 +1,10 @@
+EICAR test file
+
+https://www.eicar.org/download-anti-malware-testfile/
+
+
+
+https://www.eicar.org/download/eicar-com/?wpdmdl=8840&refresh=687011f18864d1752175089
+https://www.eicar.org/download/eicar-com-2/?wpdmdl=8842&refresh=687011f37fd6d1752175091
+https://www.eicar.org/download/eicar_com-zip/?wpdmdl=8847&refresh=687011f5747b81752175093
+https://www.eicar.org/download/eicar-com-2-2/?wpdmdl=8848&refresh=687011f75ca9b1752175095

README.md

@@ -0,0 +1,105 @@
+# DARHTS/DAPPS S3 Workflow
+
+## Environments
+
+| Environment | Account |
+|-------------|---------|
+| dev | ma41 |
+| ite | adsd-dapps-ite |
+| uat | adsd-dapps-test |
+| stage | adsd-dapps-stage |
+| prod | adsd-dapps-prod |
+| train | adsd-dapps-prod |
+
+## DARHTS buckets
+
+Three buckets per environment, for the following purposes:
+
+* in: uploaded files from DARHTS Salesforce
+* clean: after s3 scan, files which do not have threats are moved here via EventBridge and Lambda
+* quarantine: after s3 scan, files which may have threats or are unknown types (not no_threats) are moved here via EventBridge and Lambda
+
+v-s3-ditd-darhts-{env}-in-{account}-{region-short}
+v-s3-ditd-darhts-{env}-clean-{account}-{region-short}
+v-s3-ditd-darhts-{env}-quarantine-{account}-{region-short}
+
+The `clean` bucket will need versioning turned on for replication to the DAPPS bucket (below)
+All 3 buckets will use a bucket-specific KMS key.
+All 3 buckets will use bucket keys
+All 3 buckets will have finops tags for the DARHTS project accountable by DITD.
+
+The `in` bucket will have GuardDuty S3 Malware scanning enabled, which requires an IAM role and IAM Policy to allow the service
+to scan and tag. These will have the following names:
+
+p-{in-bucketname}-gd
+r-{in-bucketname}-gd
+
+where {in-bucketname} is the bucket id of the `in` bucket id, without the `v-s3-` prefix.
+These IAM objects will have finops tags for the DARHTS project accountable by DITD.
+Need to determine where the scan events go, and how OIS will be notified.
+
+An EventBridge will be setup for the GuardDuty scan with three targets.  It will be named ditd-darhts-{env}-in-guardduty.
+
+1. Cloudwatch Log (format /aws/eventbridge/gd-{in-bucketname})
+1. DARHTS API
+   * needs URL per environment
+   * needs credentials per environment
+   * credentials into AWS Secret under /apps/darhts/{env}/api-credentials
+1. Lambda (for the move, format guardduty-move-files-{in-bucketname})
+   * BUCKET_IN = arn of in bucket
+   * BUCKET_CLEAN = arn of clean bucket
+   * BUCKET_QUARANTINE = arn of quarantine bucket
+   * with a log /aws/lambda/{lambda-name}
+   * Logic:
+     * triggered by event bridge after scan
+     * if scanned object tag is NO_THREATS_FOUND, copy to `clean` bucket, delete from `in` bucket
+     * if scanned object tag is anything else, copy to `quarantine` bucket, delete from `in` bucket
+
+On creation in the `clean` bucket, S3 notification even will trigger another EventBridge named ditd-darhts-env-in-guardduty to send indication
+to DARHTS API the object has arrived in the clean bucket.
+
+### Access to Buckets
+
+An IAM service use will be created and it will be allowed to assume a role.
+
+IAM service:
+
+* Name: s-ditd-darhts-{env}-s3
+* finops tags for DARHTS project accountable by DITD
+* need to determine how to pass and rotate credentials every 90 days
+* will neeed contact (Census) name and email address (group desired)
+* permission allow it to assume the role for the file activity (below)
+
+IAM role:
+
+* Name: r-ditd-darhts-{env}-s3
+* finops tags for DARHTS project accountable by DITD
+* permissions to PUT into `in` bucket
+* permissions to GET and TAG for `clean` bucket
+
+## DAPPS buckets
+
+Same environments as above.
+
+Nne bucket per environment, for the following purposes:
+
+* clean: replicated files from DARHTS in bucket if no_threats AND sync-to-dapps tag set
+
+The format is:
+
+v-s3-adsd-dapps-{env}-clean-{account}-{region-short}
+
+Bucket replication from the v-s3-ditd-darhts-{env}-clean bucket to the v-s3-adsd-dapps-{env}-clean buckets
+will be handled by an IAM role with the name
+
+r-ditd-darhts-{env}-clean-{account}-{region-short}-replication
+
+Which has permission for GETs on the source bucket and PUTs on the target bucket.
+It has a rule which replicates all prefixes where TWO tag values match.
+
+# CHANGELOG
+
+* 1.0.0 -- 2025-07-11
+  - initial
+
+

TODO.md

@@ -0,0 +1,6 @@
+# To Do
+
+* make replication work on tags
+* package layer code into zip file in repo
+* allow individual lambdas to make own zip files
+

base.tf

@@ -0,0 +1,4 @@
+module "base" {
+  source   = "git@github.e.it.census.gov:terraform-modules/boc-nts//base-label"
+#  filename = format("%v/%v", path.module, "base.yml")
+}

data.tf

@@ -1,7 +1,7 @@
-data "aws_caller_identity" "current" {}
-
-data "aws_arn" "current" {
-  arn = data.aws_caller_identity.current.arn
+data "aws_kms_key" "s3_key" {
+  key_id = "alias/aws/s3"
 }
 
-data "aws_region" "current" {}
+data "aws_s3_bucket" "log_bucket" {
+  bucket = format("inf-logs-%v-%v", var.account_id, local.region)
+}

eventbridge.guardduty.tf

@@ -0,0 +1,162 @@
+locals {
+  short_files_in = "files_in"
+}
+
+# https://repost.aws/knowledge-center/cloudwatch-log-group-eventbridge
+# must start with /aws/events
+
+resource "aws_cloudwatch_log_group" "guardduty_event_log" {
+  #  name              = format("/aws/events/%v-%v-%v/%v/%v", var.app_info.organization, var.app_info.name, var.app_info.environment, "gd", local.short_files_in)
+  name              = format("/aws/events/%v/%v-%v", var.input_resource_label, "in", "guardduty")
+  retention_in_days = var.log_retention_in_days
+
+  tags = merge(
+    local.base_tags,
+    var.tags,
+    local.input_finops_roles["log"],
+  )
+}
+
+data "aws_iam_policy_document" "guardduty_event_log" {
+  statement {
+    sid    = "TrustEventsToStoreLogEvent"
+    effect = "Allow"
+    actions = [
+      "logs:CreateLogStream",
+      "logs:PutLogEvents",
+    ]
+    principals {
+      type = "Service"
+      identifiers = [
+        "events.amazonaws.com",
+        "delivery.logs.amazonaws.com",
+      ]
+    }
+    condition {
+      test     = "StringEquals"
+      variable = "aws:SourceAccount"
+      values   = [data.aws_caller_identity.current.account_id]
+    }
+    resources = [format("%v:log-stream:*", aws_cloudwatch_log_group.guardduty_event_log.arn)]
+  }
+}
+
+resource "aws_cloudwatch_log_resource_policy" "guardduty_event_log" {
+  policy_document = data.aws_iam_policy_document.guardduty_event_log.json
+  policy_name     = format("%v-%v", var.input_resource_label, "guardduty")
+}
+
+module "eventbridge_guardduty" {
+  source = "terraform-aws-modules/eventbridge/aws"
+  #  role_name = format("r-%v-%v-%v-%v-%v", var.app_info.organization, var.app_info.name, var.app_info.environment, "gd", local.short_files_in)
+  role_name = format("%v%v-%v-%v", try(module.base.prefixes.role, ""), var.input_resource_label, "in", "guardduty")
+
+  append_rule_postfix = false
+  create_bus          = false
+  create_role         = true
+  create_permissions  = true
+
+  ##  create_connections  = true
+  ##  create_api_destinations       = true
+  ##  attach_api_destination_policy = true
+
+  attach_lambda_policy = true
+  lambda_target_arns = [
+    module.lambda_move.lambda_function_arn,
+    module.lambda_notify.lambda_function_arn,
+  ]
+
+  attach_cloudwatch_policy = true
+  cloudwatch_target_arns = [
+    aws_cloudwatch_log_group.guardduty_event_log.arn,
+    #    aws_cloudwatch_log_group.log.arn,
+  ]
+
+  rules = {
+    format("%v-guardduty", var.input_resource_label) = {
+      description = "GuardDuty Scan Events"
+      state       = "ENABLED"
+      #      role_arn = true
+      event_pattern = jsonencode({
+        "source"      = ["aws.guardduty"]
+        "detail-type" = ["GuardDuty Malware Protection Object Scan Result"]
+      })
+    }
+  }
+  targets = {
+    format("%v-guardduty", var.input_resource_label) = [
+      {
+        name = "log-to-cloudwatch"
+        arn  = aws_cloudwatch_log_group.guardduty_event_log.arn
+      },
+      {
+        name            = "lambda-status-move"
+        arn             = module.lambda_move.lambda_function_arn
+        attach_role_arn = true
+        dead_letter_arn = module.lambda_move_failure.queue_arn
+        retry_policy = {
+          maximum_event_age_in_seconds = var.dlq_event_age
+          maximum_retry_attempts       = var.dlq_retry_attempts
+        }
+      },
+      {
+        name            = "lambda-status-notify"
+        arn             = module.lambda_notify.lambda_function_arn
+        attach_role_arn = true
+        dead_letter_arn = module.lambda_notify_failure.queue_arn
+        retry_policy = {
+          maximum_event_age_in_seconds = var.dlq_event_age
+          maximum_retry_attempts       = var.dlq_retry_attempts
+        }
+      },
+    ]
+  }
+
+  ## not supported in govcloud
+  ## https://docs.aws.amazon.com/govcloud-us/latest/UserGuide/govcloud-eventbridge.html
+  ##
+  ##   api_destinations = {
+  ##     darhts = {
+  ##       description                      = "DARHTS Salesforce API"
+  ##       invocation_endpoint              = "https://census-darhts--ppandya2.sandbox.my.salesforce.com//services/data/v63.0/sobjects/GuardDutyObjectScan__e/"
+  ##       http_method                      = "POST"
+  ##       invocation_rate_limit_per_second = 300
+  ##     }
+  ##   }
+  ## 
+  ##   connections = {
+  ##     darhts = {
+  ##       authorization_type = "OAUTH_CLIENT_CREDENTIALS"
+  ##       auth_parameters = {
+  ##         oauth = {
+  ##           authorization_endpoint = "https://census-darhts--ppandya2.sandbox.my.salesforce.com/services/oauth2/token"
+  ##           http_method            = "POST"
+  ## 
+  ##           client_parameters = {
+  ##             client_id     = "3MVG9T8tY5rhBtS0ULMDTdhcFlvMrWqhtPHqfb.zlDpiigxGqdJoT3PGVmWzA99lxW2e4LYxYkrKpTU0V4s7d"
+  ##             client_secret = "Pass1234!"
+  ##           }
+  ##           oauth_http_parameters = {
+  ##             body = [{
+  ##               key             = "grant_type"
+  ##               value           = "client_credentials"
+  ##               is_value_secret = false
+  ##             }]
+  ##           }
+  ##         }
+  ##       }
+  ##     }
+  ##   }
+
+  role_tags = merge(
+    local.base_tags,
+    var.tags,
+    local.input_finops_roles["role"],
+  )
+
+  tags = merge(
+    local.base_tags,
+    var.tags,
+    local.input_finops_roles["guardduty"],
+  )
+}