Skip to content

created structure to explain infrastructure #77

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
188 changes: 188 additions & 0 deletions docs/structure/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,188 @@
## Resource Structure
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file looks like it came directly from EDL and as such doesn't necessarily belong as part of this.




### S3 Bucket

We may/shoudl standardize on the _region_ being the compressed region format. This takes the first
character of each component of the region, split by the dash. us-gov-west-1 becomes ugw1. These
also happen to be what AWS uses behind the scenes. There is a 63 character limit for S3 bucket names
(a DNS limitation per hostname component).

## If a resource is deployed based on a DMS ID then see the below.

{prefix}-{project}-{env}-{project1}-{account_id}-{region}

| Field | Description | Type | Example |
|-------|-------------|------|---------|
| prefix | S3 bucket prefix standard | string | v-s3 |
| env | Environment: dev, test, ite, qa, stage, prod | enum(string) | dev |
| project | string | das, edl etc |
| project1 | DMS Project number | integer | 7512345 |
| account_id | AWS Account ID | integer(12) | 123123123123 |
| region | AWS Region Identifier, either full or compressed:<BR>us-gov-east-1, us-gov-west-1<BR>uge1, ugw1 | string |

Examples:

```script
v-s3-{project}-dev-7512345-account_id-us-gov-west-1
v-s3-{project}-ite-7512345-account_id-us-gov-west-1
v-s3-{project}-stage-7512345-account_id-us-gov-west-1
v-s3-{project}-prod-7512345-account_id-us-gov-west-1
v-s3-{project}-dev-7512345-account_id-ugw1
v-s3-{project}-ite-7512345-account_id-ugw1
v-s3-{project}-stage-7512345-account_id-ugw1
v-s3-{project}-prod-7512345-account_id-ugw1
```

## If a resource is deployed "NOT" based on a DMS ID then see the below.

{prefix}-{project}-{env}-{bucket_name}-{account_id}-{region}

| Field | Description | Type | Example |
|-------|-------------|------|---------|
| prefix | S3 bucket prefix standard | string | v-s3 |
| env | Environment: dev, test, ite, qa, stage, prod | enum(string) | dev |
| bucket_name|cdl-asset, logs, cumulus, data, dhc etc | enum(string) |
| project | string | das, edl etc |
| account_id | AWS Account ID | integer(12) | 123123123123 |
| region | AWS Region Identifier, either full or compressed:<BR>us-gov-east-1, us-gov-west-1<BR>uge1, ugw1 | string |

Examples:

```script
v-s3-{project}-dev-{bucket_name}-account_id-us-gov-west-1
v-s3-{project}-ite-{bucket_name}-account_id-us-gov-west-1
v-s3-{project}-stage-{account_id}-account_id-us-gov-west-1
v-s3-{project}-prod-{bucket_name}-account_id-us-gov-west-1
v-s3-{project}-dev-{bucket_name}-account_id-ugw1
v-s3-{project}-ite-{bucket_name}-account_id-ugw1
v-s3-{project}-stage-{bucket_name}-account_id-ugw1
v-s3-{project}-prod-{bucket_name}-account_id-ugw1
```

![S3 bucket path](images/s3_bucket.png)

### IAM Roles and Policies

* SAML Role

This role will be assigned to users of the project. As there is already a group for project users
in DMS (u-{project}), these two should be the same. We may wish to pursue some automation to enable this,
or see if we can enable a group as a member of this group.

Note that a service-based role would not also be a SAML role. See below for such roles. At this time, we
see no need for a service-labeled role and SAML.

{prefix}-{project}-{env}-{project1}

{prefix}-{project}


| Field | Description | Type | Example |
|-------|-------------|------|---------|
| prefix | Role prefix standard | string | r- |
| env | Environment: dev, test, ite, qa, stage, prod | enum(string) | dev |
| project | edl, das, dice-mojo, dice-centurion etc |string |
| project1 | DMS Project number | integer | 7512345 |

Examples:

```script
r-project-dev-7512345
r-project-ite-7512345
r-project-stage-7512345
r-project-prod-7512345
```

```script
r-project
```

![SAML Role path](images/saml_role.png)

* Policy for SAML Role

This will match the same name as the role, with the policy prefix `p-` instead of the role prefix. If this
policy will not be used by anything else, we may elect to attach it as an inline policy on
the role.

Note that a service-based role would not also be a SAML role, and the same for its associated policy. See below for such roles
and policies . At this time, we see no need for a service-labeled role policy.

{prefix}-{project}-{env}-{project1}

{prefix}-{project}

| Field | Description | Type | Example |
|-------|-------------|------|---------|
| prefix | Policy prefix standard | string | p- |
| env | Environment: dev, test, ite, qa, stage, prod | enum(string) | dev |
| project | string | edl, das, dice-mojo, dice-centurion etc |
| project1 | DMS Project number | integer | 7512345 |

Examples:

```script
p-project-dev-7512345
p-project-dev-7512345
p-project-dev-7512345
p-project-dev-7512345
```

```script
p-project
```

![Policy SAML Role path](images/saml_role.png)



### RDS

This does not have a _prefix_. Normally, there is expected to be only a single database
for a project, but we are permitting the use of mutiple databases (aka RDS instances). Most
of the RDS instances support multiple databases within the instances. However, datatabase
usage, performance characteristics, etc. may lead towards multiple instances of the same
engine.

rds_instance_name = {project}-{env}-{project1}-db{sequence}

rds_instance_name = {application}-{project}-{env}


| Field | Description | Type | Example |
|-------|-------------|------|---------|
| env | Environment: dev, test, ite, qa, stage, prod | enum(string) | dev |
|application |adsd, edl,das etc | string |
| project | adsd-dps, das, dice-mojo, dice-centurion etc | string |
| project1 | DMS Project number | integer | 7512345 |
| sequence | Sequence number of the RDS instance (>0) | integer | 1 |


![RDS path](images/rds.png)

## Security Groups

Security Groups have a prefix sg-. Normally security Groups are deployed under VPC (eg vpc(N))

![Security Groups path](images/security_groups.png)


## Subnet Groups

This does not have a _prefix_. Deployed under vpc(n)/apps/rds since subnets are created for RDS(postgres, oracle, mssql, mysql etc.

![Subnet Groups path](images/subnet_groups.png)

## Parameter Groups

This does not have a _prefix_. Deployed under top/common/{region}/apps/rds-apps since parameter groups are created for RDS(postgres, oracle, mssql, mysql etc.

![Parameter Groups path](images/parameter_groups.png)

## Option Groups

This does not have a _prefix_. Deployed under top/common/apps/{apps_dir}-region.

![Option Groups path](images/option_groups.png)
Binary file added docs/structure/images/option_groups.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/structure/images/parameter_groups.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/structure/images/rds.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/structure/images/s3_bucket.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/structure/images/saml_role.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/structure/images/security_groups.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/structure/images/subnet_groups.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.