From c7dded9a47e852a862cafd18eb13f7e40b0b664d Mon Sep 17 00:00:00 2001 From: badra001 Date: Fri, 11 Jul 2025 15:52:12 -0400 Subject: [PATCH] copy from shared-sa ref implementation --- aws/projects/ditd-darhts/README.md | 105 +++++++++++++++++++++++++++++ aws/projects/ditd-darhts/s3.md | 11 --- 2 files changed, 105 insertions(+), 11 deletions(-) create mode 100644 aws/projects/ditd-darhts/README.md delete mode 100644 aws/projects/ditd-darhts/s3.md diff --git a/aws/projects/ditd-darhts/README.md b/aws/projects/ditd-darhts/README.md new file mode 100644 index 00000000..452b42c7 --- /dev/null +++ b/aws/projects/ditd-darhts/README.md @@ -0,0 +1,105 @@ +# DARHTS/DAPPS S3 Workflow + +## Environments + +| Environment | Account | +|-------------|---------| +| dev | ma41 | +| ite | adsd-dapps-ite | +| uat | adsd-dapps-test | +| stage | adsd-dapps-stage | +| prod | adsd-dapps-prod | +| train | adsd-dapps-prod | + +## DARHTS buckets + +Three buckets per environment, for the following purposes: + +* in: uploaded files from DARHTS Salesforce +* clean: after s3 scan, files which do not have threats are moved here via EventBridge and Lambda +* quarantine: after s3 scan, files which may have threats or are unknown types (not no_threats) are moved here via EventBridge and Lambda + +v-s3-ditd-darhts-{env}-in-{account}-{region-short} +v-s3-ditd-darhts-{env}-clean-{account}-{region-short} +v-s3-ditd-darhts-{env}-quarantine-{account}-{region-short} + +The `clean` bucket will need versioning turned on for replication to the DAPPS bucket (below) +All 3 buckets will use a bucket-specific KMS key. +All 3 buckets will use bucket keys +All 3 buckets will have finops tags for the DARHTS project accountable by DITD. + +The `in` bucket will have GuardDuty S3 Malware scanning enabled, which requires an IAM role and IAM Policy to allow the service +to scan and tag. These will have the following names: + +p-{in-bucketname}-gd +r-{in-bucketname}-gd + +where {in-bucketname} is the bucket id of the `in` bucket id, without the `v-s3-` prefix. +These IAM objects will have finops tags for the DARHTS project accountable by DITD. +Need to determine where the scan events go, and how OIS will be notified. + +An EventBridge will be setup for the GuardDuty scan with three targets. It will be named ditd-darhts-{env}-in-guardduty. + +1. Cloudwatch Log (format /aws/eventbridge/gd-{in-bucketname}) +1. DARHTS API + * needs URL per environment + * needs credentials per environment + * credentials into AWS Secret under /apps/darhts/{env}/api-credentials +1. Lambda (for the move, format guardduty-move-files-{in-bucketname}) + * BUCKET_IN = arn of in bucket + * BUCKET_CLEAN = arn of clean bucket + * BUCKET_QUARANTINE = arn of quarantine bucket + * with a log /aws/lambda/{lambda-name} + * Logic: + * triggered by event bridge after scan + * if scanned object tag is NO_THREATS_FOUND, copy to `clean` bucket, delete from `in` bucket + * if scanned object tag is anything else, copy to `quarantine` bucket, delete from `in` bucket + +On creation in the `clean` bucket, S3 notification even will trigger another EventBridge named ditd-darhts-env-in-guardduty to send indication +to DARHTS API the object has arrived in the clean bucket. + +### Access to Buckets + +An IAM service use will be created and it will be allowed to assume a role. + +IAM service: + +* Name: s-ditd-darhts-{env}-s3 +* finops tags for DARHTS project accountable by DITD +* need to determine how to pass and rotate credentials every 90 days +* will neeed contact (Census) name and email address (group desired) +* permission allow it to assume the role for the file activity (below) + +IAM role: + +* Name: r-ditd-darhts-{env}-s3 +* finops tags for DARHTS project accountable by DITD +* permissions to PUT into `in` bucket +* permissions to GET and TAG for `clean` bucket + +## DAPPS buckets + +Same environments as above. + +Nne bucket per environment, for the following purposes: + +* clean: replicated files from DARHTS in bucket if no_threats AND sync-to-dapps tag set + +The format is: + +v-s3-adsd-dapps-{env}-clean-{account}-{region-short} + +Bucket replication from the v-s3-ditd-darhts-{env}-clean bucket to the v-s3-adsd-dapps-{env}-clean buckets +will be handled by an IAM role with the name + +r-ditd-darhts-{env}-clean-{account}-{region-short}-replication + +Which has permission for GETs on the source bucket and PUTs on the target bucket. +It has a rule which replicates all prefixes where TWO tag values match. + +# CHANGELOG + +* 1.0.0 -- 2025-07-11 + - initial + + diff --git a/aws/projects/ditd-darhts/s3.md b/aws/projects/ditd-darhts/s3.md deleted file mode 100644 index e24b8046..00000000 --- a/aws/projects/ditd-darhts/s3.md +++ /dev/null @@ -1,11 +0,0 @@ -# DARHTS/DAPPS S3 Bucket Data Exchange - -1. per environment (dev, test, ite, uat, qa, stage, prod, needs to be listed) -1. setup 3 buckets: in, clean, quarantine -1. buckets use guardduty s3 -1. in bucket will be accessed from a diff govcloud account (darhts), ideally through a cross account role - * they asked for service account, I want to push back on that -1. after scan, lambda kicks in via event bridge - * if tagged with clean, moved to clean bucket - * if somethign else, move to quarantine -1. so we can get the requirements fully documented