TXT records

[badra001]

I was looking at the external-dns in the k8s sigs project (https://github.com/kubernetes-sigs/external-dns) and they record text records using a tag of "heritage". It seems we can adopt the same style. The goal is to only remove records we added by the automation.

Ideas:

• add "heritage=dynamic-route53" (or dynr53)
• add each key-value pair as dynr53/{key}={value}
  • version={code_version}
  • account_id={aws_account_id}
  • region={aws_region}
  • instance_id={instance_id}
  • create_time={create_time_epoch}
• join all theses as a comma separated value into the TXT record (for A, AAAA, and PTR)
• when adding a cname, we need to also track that. We cannot add a TXT for a cname, so we have to add a record with a prefix (default: _info), using the same values as above
• we record the key/value pairs into the DDB table
• this seems to be extensible

If someone changes these tags, the DNS will not update.

[cho00013]

@badra001 - probably will work on this next week. So can we flush out the strategy?

1. I think above make sense but can you put an example of what a TXT record would look for particular Record?
2. A, CNAME and PTR
3. What field(s) would you want to maintain? account_id, region, instance_id?

I am not sure if I understand this statement below

we record the key/value pairs into the DDB table

[badra001]

For an A or an AAAA, we use the same name. We only need one txt per RR name (it may have an A, an AAAA, or both).

name in txt "heritage=dynr53,dynr53/version=0.0.8,dynr53/account_id=252999262699,dynr53/region=us-gov-west-1,dynr53/instance_id=i-0b03f8d3e89e9fbf1,dynr53/create_time=1644264267"

We will add the same thing to the PTR zone, but as a TXT record:

# $ORIGIN 10.188.10.in-addr.arpa.
# 88 IN PTR name.
88 in txt "(same as above)"

[badra001]

For any CNAMEs that get created, we will need a prefix (defined as an ENV, default "_txt"), because a CNAME can't have a TXT record. It'll be exactly the same stuff.

_txt.name in txt "same as above"

[cho00013]

OK. Getting fuller picture... few follow up questions.

For an A or an AAAA, we use the same name. We only need one txt per RR name (it may have an A, an AAAA, or both).

name in txt "heritage=dynr53,dynr53/version=0.0.8,dynr53/account_id=252999262699,dynr53/region=us-gov-west-1,dynr53/instance_id=i-0b03f8d3e89e9fbf1,dynr53/create_time=1644264267"

We will add the same thing to the PTR zone, but as a TXT record:

# $ORIGIN 10.188.10.in-addr.arpa.
# 88 IN PTR name.
88 in txt "(same as above)"

1. So for something that has A and AAAA record, we are creating a SINGLE TXT record. Should that info be captured in the TXT? Meaning, should we have something like dynr53/recordtype=A, dynr53/recordtype=AAAA dynr53/recordtype=BOTH or dynr53/recordtype=A-AAAA
2. And the CNAME record would be _txt.dnsname. I think we should NOT use period (so _txt_dndname?) It make the parsing out info tricky.
3. I wonder if we put the txt record in JSON format, that will be easier to program? {key=value}? Anyway, I forget whether "{}" are allowed.

[badra001]

For fun, I added somethign similar to the manual records I've setup with terraform:

das-sci001.ite.das.rm.census.gov has address 10.191.18.5

% host -t txt das-sci001.ite.das.rm.census.gov
das-sci001.ite.das.rm.census.gov descriptive text "heritage=terraform,terraform/account_id=252999262699,terraform/region=us-gov-west-1,terraform/instance_id=i-004efee02ae5f3932,terraform/create_time=1641483499"

% host 10.191.18.5
5.18.191.10.in-addr.arpa domain name pointer das-sci001.ite.das.rm.census.gov.

% host -t txt 5.18.191.10.in-addr.arpa
5.18.191.10.in-addr.arpa descriptive text "heritage=terraform,terraform/account_id=252999262699,terraform/region=us-gov-west-1,terraform/instance_id=i-004efee02ae5f3932,terraform/create_time=1641483499"

I'm going to pop this into a module, so this will be defined similarly as the automated route53 stuff, but with a different heritage.

[badra001]

1. So for something that has A and AAAA record, we are creating a SINGLE TXT record. Should that info be captured in the TXT? Meaning, should we have something like dynr53/recordtype=A, dynr53/recordtype=AAAA dynr53/recordtype=BOTH or dynr53/recordtype=A-AAAA

One text records per RR name. An RR may have multiple other records, like A, AAAA, other TXT, MX, a few other possibilities. We're not looking at any other ones at this time besides A and AAAA. And the TXT. Saving the A or AAAA in the TXT is redundant.

2. And the CNAME record would be _txt.dnsname. I think we should NOT use period (so _txt_dndname?) It make the parsing out info tricky.

Dots are standard for this stuff. Underscores are typically discouraged from DNS, as they aren't legal, at least by specification, but many things use the underscore: Active Directory, DMARC, domain validation, etc. This seems like a good use of it, and consistent with other uses.

3. I wonder if we put the txt record in JSON format, that will be easier to program? {key=value}? Anyway, I forget whether "{}" are allowed.

TXT allows nearly everything, but putting quotes there needed by json will be very icky. We'll go with the format specified.

[cho00013]

@badra001

Regarding the TXT record, we need to be mindful of the Route53 API limit (5 per second). Currently, A, PTR, and CNAME (optional) results in 3 ChangeRecordSet API calls. I've spaced them out (1 sec) to minimize the going over the limit, but if we add TXT records for each, then we have 6 API calls.

Few ways to work-around:
"Short" Term

For the TXT record, combine the TXT record into a single ChangeRecordSet (1 API call can include a list of changes). I would recommend this at a minimum a TXT record creation and deletion will be just a single 1 additional API.
Add a 1 sec sleep before TXT record creation/deletion API call. This is to spread out the API call.

Long Term

Modify the current code to combine ALL DNS updates into a single API call (Create ALL 4 records) and a single API call (Delete ALL 6 records). This is probably what we should do. This will be little more invasive change on the current code but this will significantly reduce the probability of the API calls going over the limit.
So the idea is to build the ChangeRecordSet for various entries (A, PTR, TXT, CNAME, etc) and make a single API call. I think this is probably the best route, and we can even build a error control to retry if that ChangeRecordSet fails.

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/route53.html#Route53.Client.change_resource_record_sets

[cho00013]

Actually, the ChangeRecordSet is per PHZ so we won't be able to bundle everything into a single API call. At best, 1 API call per PHZ (forward and reverse) and probably separate one for CNAME if the zoneid is different.

[badra001]

Closed in d37ecd6