Add ses domain (#9)

* initial setup of ses-domain * update * udpate docs * update docs * update docs * update docs * update docs again * fix * fix * fix * undo checking of availability * add error doc * update doc * update changelog
terraform-modules · Mar 16, 2021 · 77acb63 · 77acb63
1 parent a7015e6
commit 77acb63
Show file tree

Hide file tree

Showing 15 changed files with 453 additions and 2 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -28,3 +28,6 @@
 
 * v1.6.0 -- 20210302
   - module: iam-cloud-admin
+
+* v1.7.0 -- 20210316
+  - module: ses-domain
diff --git a/common/data.tf b/common/data.tf
@@ -18,6 +18,6 @@ data "aws_region" "current" {}
 #   value = data.aws_arn.current.partition
 # }
 #
-# output "account_region"name" {
+# output "account_region_name" {
 #   value = data.aws_region.current.name
 # }
diff --git a/common/version.tf b/common/version.tf
@@ -1,3 +1,3 @@
 locals {
-  _module_version = "1.6.0"
+  _module_version = "1.7.0"
 }
diff --git a/ses-domain/README.md b/ses-domain/README.md
@@ -0,0 +1,132 @@
+# aws-inf-setup :: ses-domain
+
+This sets up the domain identity for SES.  We create by default `{account_id}.aws.mail.census.gov`  
+as a sender domain, and generate the details which are to be submitted to TCO for inclusion in DNS.  
+The contents of the file `setup/ses_dns.md` contain the text which goes to TCO.  A future enhancement  
+to this will include the ability to automatically incorporate the changes into DNS.
+
+## Select proper region
+
+Note that at this time, this needs to be done from the `us-gov-west-1` region for GovCloud.  For East/West, any US region  
+will work. Ideally, this will check that the west zone is selected for gov, and not create any resources if in the incorrect region.  
+However, that's not current possible with Terraform 0.12, and we need to get to 0.13 to make that possible.  You will see this  
+error if in the incorrect region:
+```console
+% tf-apply
+module.ses.aws_ses_domain_identity.this: Creating...
+
+Error: Error requesting SES domain identity verification: SignatureDoesNotMatch: Credential should be scoped to a valid region, not 'us-gov-east-1'.
+        status code: 403, request id: dff573a8-961a-4a5d-a9f8-1cf6c3beba46
+
+  on .terraform/modules/ses/ses-domain/main.tf line 106, in resource "aws_ses_domain_identity" "this":
+ 106: resource "aws_ses_domain_identity" "this" {
+```
+
+## Request TCO/OIS approval for 3rd Party Domain
+
+A submission of a form authorizing 3rd party mail is required to TCO, who then puts in a ticket to OIS.  This has  
+been done for the AWS accounts as whole, using the default format.  Any variation (that is, using a custom
+`ses_domain_name` requires this form be filled out.  Details are available on the [TCO Wiki Mail Section](https://wiki.apps.tco.census.gov/index.php/Services#Mail) or in [Sending Mail](https://wiki.apps.tco.census.gov/index.php/Sending_Mail).
+
+# Usage  
+This can be used without any variables to get the default configuration.
+
+```hcl
+module "ses" {
+  source = "git@github.e.it.census.gov:terraform-modules/aws-inf-setup.git//ses-domain"
+
+  # optional
+  ses_domain_name = "somerandomdomain.aws.mail.census.gov"
+  # while these can be changed, it is not advised
+  ses_base_domain_name = "aws.mail.census.gov"
+  ses_base_dkim_domain_name = "dkim.amazonses.com"
+}
+```
+
+# Sample Output
+```hcl
+ses_dkim_values = [
+  {
+    "domain" = "817869416306.aws.mail.census.gov"
+    "name" = "j4ixb5k4cum55axim6dhdn6nelsoigfm._domainkey.817869416306.aws.mail.census.gov"
+    "ttl" = 600
+    "type" = "CNAME"
+    "value" = "j4ixb5k4cum55axim6dhdn6nelsoigfm.dkim.amazonses.com"
+  },
+  {
+    "domain" = "817869416306.aws.mail.census.gov"
+    "name" = "njeeupi3lqslhjudeklt2utyhlggdrab._domainkey.817869416306.aws.mail.census.gov"
+    "ttl" = 600
+    "type" = "CNAME"
+    "value" = "njeeupi3lqslhjudeklt2utyhlggdrab.dkim.amazonses.com"
+  },
+  {
+    "domain" = "817869416306.aws.mail.census.gov"
+    "name" = "qdhk5oxqhqllj7hnxmvt74rk75zwwuhx._domainkey.817869416306.aws.mail.census.gov"
+    "ttl" = 600
+    "type" = "CNAME"
+    "value" = "qdhk5oxqhqllj7hnxmvt74rk75zwwuhx.dkim.amazonses.com"
+  },
+]
+ses_domain_verification = {
+  "domain" = "817869416306.aws.mail.census.gov"
+  "name" = "_amazonses.817869416306.aws.mail.census.gov"
+  "ttl" = 600
+  "type" = "TXT"
+  "value" = "/Pz6+wpIUfumhdG8l0NkdfLx+wHMp+/Za2Nf5jOQTos="
+}
+```
+
+# Sample ses\_dns.md output
+
+See [ses\_dns.md](example.ses\_dns.md).
+
+## Requirements
+
+No requirements.
+
+## Providers
+
+| Name | Version |
+|------|---------|
+| aws | n/a |
+| external | n/a |
+| null | n/a |
+
+## Modules
+
+No Modules.
+
+## Resources
+
+| Name |
+|------|
+| [aws_arn](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/arn) |
+| [aws_caller_identity](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) |
+| [aws_region](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) |
+| [aws_ses_domain_dkim](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ses_domain_dkim) |
+| [aws_ses_domain_identity](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ses_domain_identity) |
+| [aws_ses_domain_identity_verification](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ses_domain_identity_verification) |
+| [external_external](https://registry.terraform.io/providers/hashicorp/external/latest/docs/data-sources/external) |
+| [null_resource](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) |
+
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|------|---------|:--------:|
+| account\_alias | AWS Account Alias | `string` | `""` | no |
+| account\_id | AWS Account ID (default will pull from current user) | `string` | `""` | no |
+| override\_prefixes | Override built-in prefixes by component (efs, s3, ebs, kms, role, policy, security-group). This should be used primarily for common infrastructure things | `map(string)` | `{}` | no |
+| ses\_base\_dkim\_domain\_name | SES Base DKIM Domain Name | `string` | `"dkim.amazonses.com"` | no |
+| ses\_base\_domain\_name | SES Base Domain Name | `string` | `"aws.mail.census.gov"` | no |
+| ses\_domain\_name | SES Fully Qualified Domain Name (default: {account\_id}.aws.mail.census.gov) | `string` | `""` | no |
+| tags | AWS Tags to apply to appropriate resources (S3, KMS).  Do not include safeguard tags here, use the data\_safeguard field for such things. | `map(string)` | `{}` | no |
+
+## Outputs
+
+| Name | Description |
+|------|-------------|
+| ses\_dkim\_values | DNS entries to add for DKIM |
+| ses\_domain\_identity | SES Domain Identity |
+| ses\_domain\_identity\_arn | SES Domain Identity ARN |
+| ses\_domain\_verification | DNS entries to add for domain verification |
diff --git a/ses-domain/bin/dns_get_txt.sh b/ses-domain/bin/dns_get_txt.sh
@@ -0,0 +1,16 @@
+#!/bin/bash
+
+#set -e
+eval "$(jq -r '@sh "DOMAIN=\(.domain)"')"
+
+DIG="/usr/bin/dig"
+if [ -x $DIG ]
+then
+  result=$($DIG +short in txt $DOMAIN|sed -e 's/"//g')
+  status=$?
+else
+  result=""
+  status=1
+fi
+
+jq -n --arg domain "$DOMAIN" --arg value "$result" --arg status "$status" '{"domain":$domain,"value":$value,"status":$status}'
diff --git a/ses-domain/data.tf b/ses-domain/data.tf
@@ -0,0 +1 @@
+../common/data.tf
diff --git a/ses-domain/defaults.tf b/ses-domain/defaults.tf
@@ -0,0 +1 @@
+../common/defaults.tf
diff --git a/ses-domain/example.ses_dns.md b/ses-domain/example.ses_dns.md
@@ -0,0 +1,37 @@
+# DNS Validation
+
+Update DNS to complete SES verification with the following details. This must be done
+on all views: internal, Dmz, and Public, though Public is the most important one.
+
+* TXT records for validation:
+
+```
+$ORIGIN 817869416306.aws.mail.census.gov.
+_amazaonses  600 in txt "/Pz6+wpIUfumhdG8l0NkdfLx+wHMp+/Za2Nf5jOQTos="
+
+# OR fully-qualified
+_amazaonses.817869416306.aws.mail.census.gov. 600 in txt "/Pz6+wpIUfumhdG8l0NkdfLx+wHMp+/Za2Nf5jOQTos="
+
+```
+
+# DKIM Records
+
+Update DNS to set the DKIM records with the following details. This must be done
+on all views: internal, Dmz, and Public, though Public is the most important one.
+
+```
+$ORIGIN 817869416306.aws.mail.census.gov.
+j4ixb5k4cum55axim6dhdn6nelsoigfm._domainkey 600in cname j4ixb5k4cum55axim6dhdn6nelsoigfm.dkim.amazonses.com.
+njeeupi3lqslhjudeklt2utyhlggdrab._domainkey 600in cname njeeupi3lqslhjudeklt2utyhlggdrab.dkim.amazonses.com.
+qdhk5oxqhqllj7hnxmvt74rk75zwwuhx._domainkey 600in cname qdhk5oxqhqllj7hnxmvt74rk75zwwuhx.dkim.amazonses.com.
+
+# OR fully-qualified
+j4ixb5k4cum55axim6dhdn6nelsoigfm._domainkey.817869416306.aws.mail.census.gov. 600 in cname j4ixb5k4cum55axim6dhdn6nelsoigfm.dkim.amazonses.com.
+njeeupi3lqslhjudeklt2utyhlggdrab._domainkey.817869416306.aws.mail.census.gov. 600 in cname njeeupi3lqslhjudeklt2utyhlggdrab.dkim.amazonses.com.
+qdhk5oxqhqllj7hnxmvt74rk75zwwuhx._domainkey.817869416306.aws.mail.census.gov. 600 in cname qdhk5oxqhqllj7hnxmvt74rk75zwwuhx.dkim.amazonses.com.
+```
+
+# MX Records
+
+
+
diff --git a/ses-domain/main.tf b/ses-domain/main.tf
@@ -0,0 +1,149 @@
+/*
+* # aws-inf-setup :: ses-domain
+* 
+* This sets up the domain identity for SES.  We create by default `{account_id}.aws.mail.census.gov`
+* as a sender domain, and generate the details which are to be submitted to TCO for inclusion in DNS.
+* The contents of the file `setup/ses_dns.md` contain the text which goes to TCO.  A future enhancement
+* to this will include the ability to automatically incorporate the changes into DNS.
+* 
+* ## Select proper region
+*
+* Note that at this time, this needs to be done from the `us-gov-west-1` region for GovCloud.  For East/West, any US region
+* will work. Ideally, this will check that the west zone is selected for gov, and not create any resources if in the incorrect region.
+* However, that's not current possible with Terraform 0.12, and we need to get to 0.13 to make that possible.  You will see this
+* error if in the incorrect region:
+* ```console
+* % tf-apply
+* module.ses.aws_ses_domain_identity.this: Creating...
+* 
+* Error: Error requesting SES domain identity verification: SignatureDoesNotMatch: Credential should be scoped to a valid region, not 'us-gov-east-1'.
+*         status code: 403, request id: dff573a8-961a-4a5d-a9f8-1cf6c3beba46
+* 
+*   on .terraform/modules/ses/ses-domain/main.tf line 106, in resource "aws_ses_domain_identity" "this":
+*  106: resource "aws_ses_domain_identity" "this" {
+* ```
+* 
+* ## Request TCO/OIS approval for 3rd Party Domain
+* 
+* A submission of a form authorizing 3rd party mail is required to TCO, who then puts in a ticket to OIS.  This has
+* been done for the AWS accounts as whole, using the default format.  Any variation (that is, using a custom
+* `ses_domain_name` requires this form be filled out.  Details are available on the [TCO Wiki Mail Section](https://wiki.apps.tco.census.gov/index.php/Services#Mail) or in [Sending Mail](https://wiki.apps.tco.census.gov/index.php/Sending_Mail).
+* 
+* # Usage
+* This can be used without any variables to get the default configuration.
+*
+* ```hcl
+* module "ses" {
+*   source = "git@github.e.it.census.gov:terraform-modules/aws-inf-setup.git//ses-domain"
+*
+*   # optional
+*   ses_domain_name = "somerandomdomain.aws.mail.census.gov"
+*   # while these can be changed, it is not advised
+*   ses_base_domain_name = "aws.mail.census.gov"
+*   ses_base_dkim_domain_name = "dkim.amazonses.com"
+* }
+* ```
+*
+* # Sample Output
+* ```hcl
+* ses_dkim_values = [
+*   {
+*     "domain" = "817869416306.aws.mail.census.gov"
+*     "name" = "j4ixb5k4cum55axim6dhdn6nelsoigfm._domainkey.817869416306.aws.mail.census.gov"
+*     "ttl" = 600
+*     "type" = "CNAME"
+*     "value" = "j4ixb5k4cum55axim6dhdn6nelsoigfm.dkim.amazonses.com"
+*   },
+*   {
+*     "domain" = "817869416306.aws.mail.census.gov"
+*     "name" = "njeeupi3lqslhjudeklt2utyhlggdrab._domainkey.817869416306.aws.mail.census.gov"
+*     "ttl" = 600
+*     "type" = "CNAME"
+*     "value" = "njeeupi3lqslhjudeklt2utyhlggdrab.dkim.amazonses.com"
+*   },
+*   {
+*     "domain" = "817869416306.aws.mail.census.gov"
+*     "name" = "qdhk5oxqhqllj7hnxmvt74rk75zwwuhx._domainkey.817869416306.aws.mail.census.gov"
+*     "ttl" = 600
+*     "type" = "CNAME"
+*     "value" = "qdhk5oxqhqllj7hnxmvt74rk75zwwuhx.dkim.amazonses.com"
+*   },
+* ]
+* ses_domain_verification = {
+*   "domain" = "817869416306.aws.mail.census.gov"
+*   "name" = "_amazonses.817869416306.aws.mail.census.gov"
+*   "ttl" = 600
+*   "type" = "TXT"
+*   "value" = "/Pz6+wpIUfumhdG8l0NkdfLx+wHMp+/Za2Nf5jOQTos="
+* }
+* ```
+*
+* # Sample ses_dns.md output
+*
+* See [ses_dns.md](example.ses_dns.md).
+*/
+
+locals {
+  account_id          = var.account_id != "" ? var.account_id : data.aws_caller_identity.current.account_id
+  account_environment = data.aws_arn.current.partition == "aws-us-gov" ? "gov" : "ew"
+  region              = data.aws_region.current.name
+
+  ses_available = local.account_environment == "ew" ? length(regexall("us-", local.region)) > 0 : length(regexall("gov-west", local.region)) > 0
+
+  ses_domain       = var.ses_domain_name != "" ? var.ses_domain_name : format("%v.%v", local.account_id, var.ses_base_domain_name)
+  ses_dns_txt_name = format("_amazonses.%v", aws_ses_domain_identity.this.domain)
+  ses_dns_ttl      = 600
+  ses_dns_value    = aws_ses_domain_identity.this.verification_token
+
+  ses_output = templatefile("${path.module}/ses_dns.md.tpl", {
+    domain         = local.ses_domain
+    ttl            = local.ses_dns_ttl
+    validation_txt = local.ses_dns_value
+    dkim_tokens    = aws_ses_domain_dkim.this.dkim_tokens
+  })
+  ses_domain_ready = data.external.ses_dns_txt.result.value != "" ? true : false
+
+  base_tags = {
+    "boc:tf_module_version" = local._module_version
+    "boc:created_by"        = "terraform"
+  }
+}
+
+data "external" "ses_dns_txt" {
+  program = ["bash", "${path.module}/bin/dns_get_txt.sh"]
+  # output {object}.results.{domain,status,value}
+  query = {
+    "domain" = local.ses_dns_txt_name
+  }
+}
+
+resource "aws_ses_domain_identity" "this" {
+  #  count  = local.ses_available ? 1 : 0
+  domain = local.ses_domain
+}
+
+resource "aws_ses_domain_dkim" "this" {
+  #  count  = local.ses_available ? 1 : 0
+  domain = aws_ses_domain_identity.this.domain
+}
+
+resource "aws_ses_domain_identity_verification" "this" {
+  #  count  = local.ses_available && local.ses_domain_ready ? 1 : 0
+  count  = local.ses_domain_ready ? 1 : 0
+  domain = aws_ses_domain_identity.this.id
+}
+
+# resource "aws_ses_domain_mail_from" "this" {
+#   domain           = aws_ses_domain_identity.example.domain
+#   mail_from_domain = "bounce.${aws_ses_domain_identity.example.domain}"
+# }
+
+resource "null_resource" "main_output" {
+  provisioner "local-exec" {
+    command = "test -d ${path.root}/setup || mkdir ${path.root}/setup"
+  }
+  provisioner "local-exec" {
+    command = "echo '${local.ses_output}' > setup/ses_dns.md"
+  }
+}
+
diff --git a/ses-domain/outputs.tf b/ses-domain/outputs.tf
@@ -0,0 +1,32 @@
+output "ses_domain_identity" {
+  description = "SES Domain Identity"
+  value       = aws_ses_domain_identity.this.domain
+}
+
+output "ses_domain_identity_arn" {
+  description = "SES Domain Identity ARN"
+  value       = aws_ses_domain_identity.this.arn
+}
+
+output "ses_domain_verification" {
+  description = "DNS entries to add for domain verification"
+  value = {
+    "domain" = aws_ses_domain_identity.this.domain
+    "name"   = local.ses_dns_txt_name
+    "type"   = "TXT"
+    "ttl"    = local.ses_dns_ttl
+    "value"  = local.ses_dns_value
+  }
+}
+
+output "ses_dkim_values" {
+  description = "DNS entries to add for DKIM"
+  value = [for dk in aws_ses_domain_dkim.this.dkim_tokens : {
+    "domain" = aws_ses_domain_identity.this.domain
+    "name"   = format("%v._domainkey.%v", dk, aws_ses_domain_identity.this.domain)
+    "type"   = "CNAME"
+    "ttl"    = local.ses_dns_ttl
+    "value"  = format("%v.%v", dk, var.ses_base_dkim_domain_name)
+    }
+  ]
+}
diff --git a/ses-domain/prefixes.tf b/ses-domain/prefixes.tf
@@ -0,0 +1 @@
+../common/prefixes.tf