Getting Json Loader when parsing DynamoDB item when terminating instance

[cho00013]

So I am seeing something weird in my lab. line 1425. I am getting 100% error when I run the code. The issue is that I haven't made any changes on that module (I don't think) and that has worked in the past (from the original code). I'm trying not to tweak too much as it may have "butterfly" effect 🙂.

return json.loads(item)

From the subroutine below...

https://github.e.it.census.gov/terraform-modules/aws-dynamic-route53/blob/master/code/ddns-lambda.py

def get_item_from_dynamodb_table(client, table, instance_id):
    """
    Get item from dynamodb table
    :param client:
    :param table:
    :param instance_id:
    :return:
    """
    try:
        # Fetch item from DynamoDB
        item = client.get_item(
            TableName=table,
            Key={
                'InstanceId': {
                    'S': instance_id
                }
            },
            AttributesToGet=[
                'InstanceAttributes'
            ]
        )

        if 'Item' in item:
            LOGGER.debug("returned item:"
                         " %s", str(item['Item']['InstanceAttributes']['S']) + lineno())
            item = item['Item']['InstanceAttributes']['S'].replace("'", '"')
            item = item.replace(" True,", ' "True",')
            item = item.replace(" False,", ' "False",')
            LOGGER.debug("item: %s", str(item) + lineno())
            return json.loads(item)
        return None
    except ClientError as err:
        LOGGER.error("instance: %s, unexpected error. %s\n",
                     instance_id, str(err) + lineno())

[cho00013]

I've narrowed the culprit in my lab error. I do not think it's the same issue as what you saw in the 50 instance case. It was an edge case where the True text was not replaced with "True" which basically caused the json load command to fail. I am looking at if there's a better way to parse the DDB item OR figure out how to fool-proof the way that DDB item is being being written (perhaps perform json.dumps before writing it so that it stays in the proper string format - currently it's just doing str which doesn't put into a proper json string that can be "loaded" - the code is doing some replace command to make it json like).

[cho00013]

Per @badra001

I think that we want this instead

https://towardsaws.com/making-use-of-boto3-out-of-the-box-dynamodb-serializers-1dffbc7deafe

so

 1418          if 'Item' in item:
  1419              LOGGER.debug("returned item:"
  1420                           " %s", str(item['Item']['InstanceAttributes']['S']) + lineno())
  1421              item = item['Item']['InstanceAttributes']['S'].replace("'", '"')
  1422              item = item.replace(" True,", ' "True",')
  1423              item = item.replace(" False,", ' "False",')
  1424              LOGGER.debug("item: %s", str(item) + lineno())
  1425              return json.loads(item)
  1426          return None
  1427      except ClientError as err:
  1428          LOGGER.error("instance: %s, unexpected error. %s\n",
  1429                       instance_id, str(err) + lineno())

Add

from boto3.dynamodb.types import TypeDeserializer, TypeSerializer
Replace 1421 through 1423 with

item = dynamo_obj_to_python_obj(item['item']['InstanceAttributes']['S'])
replace 1425 with

return item

[cho00013]

Fixed in PR #23