wip

terraform · Apr 1, 2025 · 60c40e7 · 60c40e7
1 parent 623b30f
commit 60c40e7
Show file tree

Hide file tree

Showing 376 changed files with 9,570 additions and 7,235 deletions.
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -11,7 +11,7 @@ repos:
     -   id: isort
 
 -   repo: https://github.com/pycqa/flake8
-    rev: 7.1.2
+    rev: 7.2.0
     hooks:
     -   id: flake8
         additional_dependencies: [flake8-docstrings]

diff --git a/README.prowler b/README.prowler
@@ -1,4 +1,3 @@
 git clone https://github.com/toniblyx/prowler.git
 
 git submodule add https://github.com/toniblyx/prowler.git prowler
-
diff --git a/docs/BOOTSTRAP.md b/docs/BOOTSTRAP.md
@@ -3,7 +3,7 @@
 Do all setup from (GETTING-STARTED.md).
 
 ```code
-git clone 
+git clone
 git-secret reveal
 ```
 
@@ -13,7 +13,7 @@ etc.
 
 We need to create an account in which to bootstap one's own user.  Generally when set up,
 we have an a-USER account, and access keys and setup is done under that user, and access
-keys mapped to the specific profile.  Since all the accounts are created under 
+keys mapped to the specific profile.  Since all the accounts are created under
 Terraform, it's not possible to create the same user that is running the configurations.
 
 More details coming.
@@ -22,10 +22,10 @@ More details coming.
 
 Some configuration files have dependencies on other parts of the configuration in
 other directories.  We've collected base setup files into common and infrastructure,
-where common are IAM things (users, groups, policies, roles) and infrastructure are 
+where common are IAM things (users, groups, policies, roles) and infrastructure are
 buckets, config, cloudtrail, and others.
 
-## 1. Common 
+## 1. Common
 
 We need to start with an empty remote_state.* for common and infrastructure
 
@@ -38,14 +38,14 @@ cd common
 ln -sf remote_state.common.tf.none remote_state.common.tf
 
 terraform init
-terraform plan 
-terraform apply 
+terraform plan
+terraform apply
 ```
 
 We are now in a state with a number of resources setup, and we can move on
 to the next step.
 
-## 2. Infrastructure 
+## 2. Infrastructure
 
 Now that we have a state, we'll link to it
 
@@ -61,8 +61,8 @@ cd ../infrastructure
 
 terraform init
 
-terraform plan 
-terraform apply 
+terraform plan
+terraform apply
 ```
 
 After this is completed, we now have an S3 bukcet for state, so we'll start using that.
@@ -115,7 +115,7 @@ Do you want to copy existing state to the new backend?
   Enter a value:
 ```
 
-## 3. Common 
+## 3. Common
 
 We'll now go into common and activate the remote state.
 
@@ -132,7 +132,7 @@ Let's pull in the rest of the stage2 configurations and activate them.
 
 ```code
 mv .stage2/config.tf .stage2/flowlog.tf ./
-terraform plan 
+terraform plan
 terraform apply
 ```
 

diff --git a/docs/GETTING-STARTED.md b/docs/GETTING-STARTED.md
@@ -2,7 +2,7 @@
 
 We have changed from a RHEL7 to a RHEL8 server in order to support some newer things needed for EKS.
 While RHEL7 will still work for most things, it will not work for any EKS deployments. This is because
-a utility callsed `skopeo` is needed, and the RHEL7 version of it is missing login capability. This 
+a utility callsed `skopeo` is needed, and the RHEL7 version of it is missing login capability. This
 capability exists in RHEL8.
 
 RHEL 8 Linux Server minimum
@@ -29,7 +29,7 @@ These are delivered through the OS (yum, dnf)
 
 This is delivered through either the OS, via download, or the Terraform application package:
 
-* awscli v2 
+* awscli v2
 
 These are delivered through the Terraform application package.
 
@@ -87,7 +87,7 @@ aws_secret_access_key={secret_access_key}
 region = us-gov-west-1
 ```
 
-This should be reflective of your IAM account `a-{username}` in the `g-inf-cloud-admin` group, so that 
+This should be reflective of your IAM account `a-{username}` in the `g-inf-cloud-admin` group, so that
 the proper permissions are available to run the various terraform.  At some point, there may be a specific
 set of terraform permissions scoped out and narrowed down for more granular control.
 
@@ -139,4 +139,3 @@ ln -sf ../outputs.common.tf ./
 ln -sf ../variables.common.auto.tfvars ./
 ln -sf ../variables.common.tf ./
 ```
-
diff --git a/docs/GPG.md b/docs/GPG.md
@@ -11,7 +11,7 @@ Setup the `/etc/sysconfig/rngd` file as follows:
 
 ```
 # /etc/sysconfig/rngd
-EXTRAOPTIONS="-n 0 
+EXTRAOPTIONS="-n 0
 ```
 
 Then enable and start
@@ -173,24 +173,24 @@ GPG_EMAIL="other-name@census.gov" GPG_USERNAME="other" setup-gpg.sh
 
 To set up GPG keys for a service account:
 
-1. Submit LDAP Service Account Remedy ticket to have an enterprise service account created, if not already done. 
+1. Submit LDAP Service Account Remedy ticket to have an enterprise service account created, if not already done.
     * Include in the ticket details that the service account needs to be able to authenticate with Enterprise Github and must have POSIX/Unix attributes enabled so Linux based OS can recognize it.
-2. Once the Service Account is created, submit an Application Account Remedy ticket to add the service account to your GitHub repositories. 
-    * In the ticket details, include the service account name and the repositories to which it will require access. Once the service account is added, you will need to log into GitHub using the service account name/password in a PrivateTab window using https://id-provider.tco.census.gov/nidp/saml2/sso. 
-3. Once the Service Account is created, submit a ticket for sudo access to the service account from the iebcloud server. 
-    * Include the service account name in the ticket and what it is being used for. 
-4. From the iebcloud server, sudo into the the service account. 
+2. Once the Service Account is created, submit an Application Account Remedy ticket to add the service account to your GitHub repositories.
+    * In the ticket details, include the service account name and the repositories to which it will require access. Once the service account is added, you will need to log into GitHub using the service account name/password in a PrivateTab window using https://id-provider.tco.census.gov/nidp/saml2/sso.
+3. Once the Service Account is created, submit a ticket for sudo access to the service account from the iebcloud server.
+    * Include the service account name in the ticket and what it is being used for.
+4. From the iebcloud server, sudo into the the service account.
     * Run `sudo su - {service_account_name}` where **{service_account_name}** is the service account name.
 5. As the service account, create a **gpg-files** directory in the service account home direcotry and copy the the [setup-gpg.sh](../local-app/bin/setup-gpg.sh) file into it or refer to it at `/apps/terraform/bin/setup-gpg.sh` if present.
 6. Run the gpg script as the service account.
-    * Run the command `GPG_EMAIL={example_email_distro} setup-gpg.sh` where **{example_email_distro}** is a Census email distribution list corresponding to users who manage the service account. *Do not use an individual user's email address.* 
+    * Run the command `GPG_EMAIL={example_email_distro} setup-gpg.sh` where **{example_email_distro}** is a Census email distribution list corresponding to users who manage the service account. *Do not use an individual user's email address.*
 9. Verify the resulting **{service_account_name}.gpg.asc** file maps to exactly ONE pub, sub, and **{service_account_name}** uid with corresponding **{GPG_EMAIL}** email distribution. If the following command return more than one of those things, there are too many keys in this file and it won't work.
-    * Run the command `gpg {service_account_name}.gpg.asc`. 
+    * Run the command `gpg {service_account_name}.gpg.asc`.
 
     **Example**
-    
+
     ```console
-    % gpg edl-svc-tf-deploy.gpg.asc 
+    % gpg edl-svc-tf-deploy.gpg.asc
     gpg: WARNING: no command supplied.  Trying to guess what you mean ...
     pub   rsa4096 2024-05-06 [SCEA]
           XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

diff --git a/docs/api-reference.md b/docs/api-reference.md
@@ -275,10 +275,10 @@ from tf_upgrade.reporters.base import BaseReporter
 class CustomReporter(BaseReporter):
     def __init__(self):
         super().__init__()
-        
+
     def add_section(self, title, content):
         # Custom implementation
-        
+
     def generate_report(self):
         # Custom implementation
         return "Generated report"
@@ -357,11 +357,11 @@ class CustomUpgrader(BaseUpgrader):
             source_version="0.12",
             target_version="0.13"
         )
-        
+
     def pre_upgrade_check(self, directory):
         # Check if directory is ready for upgrade
         return True
-        
+
     def upgrade(self, directory):
         # Implement upgrade logic
         return self.upgrade_result(success=True)
@@ -439,38 +439,38 @@ class CustomV013Upgrader(BaseUpgrader):
             target_version="0.13"
         )
         self.transformer = HCLTransformer()
-        
+
         # Add custom rules for Census patterns
         self.transformer.add_rule(
-            name="census_module_source", 
+            name="census_module_source",
             pattern=r'source\s+=\s+"git::https://github\.com/census-bureau-internal/([^"]+)\.git\?ref=([^"]+)"',
             replacement=lambda m: f'source = "git::https://github.com/census-bureau-internal/{m.group(1)}.git?ref=tf-upgrade"'
         )
-        
+
     def upgrade(self, directory):
         # Create backup
         self.create_backup(directory)
-        
+
         try:
             # Apply custom transformations
             self.transformer.transform_directory(directory, "*.tf")
-            
+
             # Run standard 0.13upgrade command
             self.terraform_runner.run_command(
                 directory=directory,
                 command=["0.13upgrade", "-yes"]
             )
-            
+
             # Validate the upgraded configuration
             validation = self.terraform_runner.validate(directory)
             if not validation.success:
                 return self.upgrade_result(
                     success=False,
                     error=f"Validation failed: {validation.stderr}"
                 )
-                
+
             return self.upgrade_result(success=True)
-            
+
         except Exception as e:
             # Restore from backup in case of error
             self.restore_backup(directory)
@@ -483,7 +483,7 @@ The tool uses custom exceptions for better error handling:
 
 ```python
 from tf_upgrade.exceptions import (
-    UpgradeError, 
+    UpgradeError,
     ValidationError,
     TerraformExecutionError,
     BackupError
@@ -594,7 +594,7 @@ def custom_scan(directory, custom_option):
     """Perform a custom scan of Terraform configurations."""
     click.echo(f"Scanning {directory} with {custom_option}")
     # Custom scan implementation
-    
+
 @cli_group.command("custom-upgrade")
 @click.argument("directory", type=click.Path(exists=True))
 @click.option("--special-handling", is_flag=True, help="Use special handling")

diff --git a/docs/app-setup.md b/docs/app-setup.md
@@ -42,7 +42,7 @@ cd my-app-name
 
 Then:
 
-1. copy remote-state.yml from the parent directory 
+1. copy remote-state.yml from the parent directory
 1. update the `directory:` line to include the new app directory at the end
 ```
 directory: "common/app/my-app-name"

diff --git a/docs/app-setup/.terraform-docs.yml b/docs/app-setup/.terraform-docs.yml
@@ -5,7 +5,7 @@ footer-from: ""
 
 sections:
 ##  hide: []
-  show: 
+  show:
     - data-sources
     - header
     - footer
@@ -15,7 +15,7 @@ sections:
     - providers
     - requirements
     - resources
-    
+
 output:
   file: README.md
   mode: inject
@@ -27,11 +27,11 @@ output:
 ## output-values:
 ##   enabled: false
 ##   from: ""
-## 
+##
 ## sort:
 ##   enabled: true
 ##   by: name
-## 
+##
 ## settings:
 ##   anchor: true
 ##   color: true

diff --git a/docs/census-examples.md b/docs/census-examples.md
@@ -54,7 +54,7 @@ For EDL modules, follow this version reference pattern after upgrading to Terraf
 ```terraform
 module "edl_processing" {
   source = "git::https://github.com/census-bureau-internal/terraform-aws-edl-workflow.git?ref=tf-upgrade"
-  
+
   // Rest of module configuration
 }
 ```
@@ -73,7 +73,7 @@ resource "aws_security_group" "app_sg" {
   name        = "${var.environment}-${var.app_name}-sg"
   description = "Security group for ${var.app_name}"
   vpc_id      = var.vpc_id
-  
+
   tags = {
     "boc:application" = var.app_name
     "boc:environment" = var.environment
@@ -86,7 +86,7 @@ resource "aws_security_group" "app_sg" {
   name        = "${var.environment}-${var.app_name}-sg"
   description = "Security group for ${var.app_name}"
   vpc_id      = var.vpc_id
-  
+
   tags = {
     "boc:application" = var.app_name
     "boc:environment" = var.environment
@@ -105,30 +105,30 @@ Census Bureau IAM roles follow a standard pattern. Here's how they're upgraded:
 # Before upgrade (0.12.x)
 module "app_role" {
   source = "git::https://github.com/census-bureau-internal/terraform-aws-iam-role.git?ref=v0.5.0"
-  
+
   name               = "${var.environment}-${var.app_name}-role"
   assume_role_policy = data.aws_iam_policy_document.assume_role.json
-  
+
   policy_arns = [
     "arn:aws-us-gov:iam::aws:policy/AmazonS3ReadOnlyAccess",
     aws_iam_policy.custom_policy.arn
   ]
-  
+
   tags = var.tags
 }
 
 # After upgrade (1.x)
 module "app_role" {
   source = "git::https://github.com/census-bureau-internal/terraform-aws-iam-role.git?ref=v1.0.0"
-  
+
   name               = "${var.environment}-${var.app_name}-role"
   assume_role_policy = data.aws_iam_policy_document.assume_role.json
-  
+
   policy_arns = [
     "arn:aws-us-gov:iam::aws:policy/AmazonS3ReadOnlyAccess",
     aws_iam_policy.custom_policy.arn
   ]
-  
+
   tags = var.tags
 }
 ```
@@ -203,7 +203,7 @@ terraform {
   backend "s3" {
     # These values are populated from environment variables
   }
-  
+
   required_version = ">= 1.0.0"
   required_providers {
     aws = {