Skip to content

Update readme #424

Open
wants to merge 14 commits into
base: master
Choose a base branch
from
93 changes: 92 additions & 1 deletion aws/projects/adsd-chris/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,18 @@
# ADSD CHRIS

This document lists some basic architecture and requirements of the ADSD CHRIS System
* CHRIS application is a .Net application that runs on windows servers with a MSSQL Server backend database.
* Application interfaces with other systems. Most of the interfaces are through windows shared file systems.
* It is tightly integrated with boc.ad.census.gov domain. It need access to boc.ad.census.gov on-prem file shares.
* Web and Application EC2 servers need to be joined to boc.ad.census.gov domain.
* Web and Application servers need common shared filesystem that can be mounted on both servers. FSx for windows AWS storage service is an ideal choice.

# RDS MSSQL Server engine:
* CHRIS database need native SQL Server Auditing enabled. We use this feature to comply with security baseline, track user activity and data access.
* CHRIS database need native SQL Server backup and restore functionality. We use native backups to move/migrate CHRIS databases to Cloud-RDS.
* CHRIS application also uses SSIS - SQL Server Integration Services feature to process interface data files.
* The options group (SQLSERVER_AUDIT , SQL_Server_Backup_Restore , SSIS) require tight integration with S3 storage.
* SSIS options group require tight integration with boc.ad.census.gov customer managed domain controller.

## Challenges

Expand All @@ -19,12 +31,91 @@ to any future MSSQL setups.

## RDS
* Security Group
* Paraameter Group
* Security groups and Parameter groups are common for all database engines.
* sg-adsd-chris-qa-mssql
* Port 1433
* Port 445
* Parameter Group
* census-sqlserver-se-15-0
* census-sqlserver-se-16-0
* Option Group
* list each desired setting name and value
* base-ms-sqlserver-ee-15-00
* base-ms-sqlserver-se-15-00
* base-ms-sqlserver-ee-16-00
* base-ms-sqlserver-se-16-00
* Settings
* SQLSERVER_AUDIT
* RETENTION_TIME
* 365
* S3_BUCKET_ARN
* S3 bucket/mssql
* IAM_ROLE_ARN
* IAM Role
* SQLSERVER_BACKUP_RESTORE
* IAM_ROLE_ARN
* IAM Role
* SSIS
* IAM_ROLE_ARN
* IAM Role
* AD Integration
* TDE

* Audit
* SQLSERVER_AUDIT
* Backup
* SQLSERVER_BACKUP_RESTORE
* Integration with boc.ad.census.gov
* SSIS

## The OG module for the mssql db does the following
1. creates the OG
2. if enable_s3_backups set, checks for options_group_bucket_id and options_group_role_arn
3. if enable_s3_audit set, checks for same as #2
4. if none of the things are set in #2, and create_resources is set
1. create s3 bucket (need to have a lifecycle on it)
2. with kms key
3. create iam role
5. uses same bucket, key, and role for both backups and audit
6. need to decide on defaults
## Formats
1. s3: v-s3-{blf}-mssql-{sqlinstance}-{account_id}-{regionshort}
2. iam: r-{blf}-mssql-{sqlinstance}
## SSO Permission
if the sc-dba needs to read/write the bucket, we would grant assume role access to the iam role
## Process
* is this write access a one time thing?
Not a one time thing. It will be needed until the CHRIS migration is completed. The CHRIS databases are currently on-prem and the backup files will need to be copied to the S3 Bucket so they can be restored to the AWS RDS DB instance.
* is it only from windows servers?
Yes, it will only be from windows servers
* do the windows servers have the aws cli and the ability to use sso and/or the aws sdk and assume roles?
Yes, the windows server I will be using has the aws cli and the ability to use sso.
* is there one bucket per mssql db and an associated role for it?
If you mean one bucket per environment (dev,test,stage,prod) then yes a S3 bucket is needed for each environment which I believe already exists.
would like to use the current S3 bucket "s3://v-s3-adsd-chris-dev-rds-dump-582222802695-uge1/mssql/" and create a new folder called "restore" or "restore fromonprem" for testing. The role which writes backups and audit, that has write access
* what folders are in use for the backups and for the audit?
This has nothing to do with the audit folders only backups and i would like to get a new folder created under the "mssql" folder (s3://v-s3-adsd-chris-de v-rds-dump-582222802695-uge1/mssql/) called "migration", "restore" or "restorefromonprem" and upload the on-prem backup files to that folder. The files will be deleted once the restore is completed.
* Backups
s3://v-s3-adsd-chris-dev-rds-dump-582222802695-uge1/mssql/backup/chris-dev-db1/
* diff/
* full/
* Audits
s3://v-s3-adsd-chris-dev-rds-dump-582222802695-uge1/mssql/chris-dev-db1/
* ADSD_Audit/
* Database_Audit/
* Database_Schema_Object_Change_Audit/
* DatabaseTables_Audit/
* Login_Audit/
* Login_Failed_Audit/
* RoleMemberChange_Audit/
* Schema_Object_Access_Audit/
* would it be a different folder for the migration?
Yes, I would like to get a new folder created called "migration", "restore" or "restorefromonprem"
* once migrated is that write access still needed?
No
* Read will be needed to validate.
* for the backups it is run from within the SQL Server Instance via a schedule job using an AWS RDS provided stored procedure "EXEC msdb.dbo.rds_backup_dat abase"
* the folder(s) get created from the "msdb.dbo.rds_backup_database" stored procedure if they do not exist already.

## FSx Windows
* Integration with boc.ad.census.gov