working on scripts

external-dependencies.md

@@ -0,0 +1,137 @@
+# External Dependencies Documentation
+
+This document lists all external dependencies that are not managed by Terraform in the aws-image-pipeline project.
+
+## Hardcoded Values
+
+### Build Configuration
+- Builder Image: `aws/codebuild/standard:7.0`
+- Terraform Version: `1.8.5`
+- Packer Version: `1.10.3` (default)
+- SSH User: `ec2-user`
+
+### AMI Configuration
+- Base AMI ID: `ami-03fadeeea589a106b`
+- Instance Type: `t2.micro`
+
+### Repository Names (Defaults)
+- `linux-image-pipeline`
+- `image-pipeline-ansible-playbooks`
+- `image-pipeline-goss-testing`
+
+### Network Configuration
+- Proxy Server: `proxy.tco.census.gov:3128`
+- Allowed Domains:
+  - `.census.gov`
+  - `.eks.amazonaws.com`
+  - `.s3.amazonaws.com`
+  - `.amazonaws.com`
+  - `.gcr.io`
+  - `.pkg.dev`
+  - `downloads.morpheusdata.com`
+  - `169.254.169.254`
+  - Various internal network ranges (148.129.*, 10.*, 172.18-25.*)
+
+## External AWS Resources
+
+### AWS Parameter Store Values
+- AMI configuration parameters
+- Subnet configuration
+- Security group settings
+- Region settings
+- Source AMI information
+- Instance type configurations
+- SSH user parameters
+- Docker repository configurations
+- AWS account ID parameters
+- Shared accounts parameters
+- Userdata parameters
+
+### S3 Bucket Artifacts
+- Packer configurations (`linux-image-pipeline.zip`)
+- Ansible playbooks (`image-pipeline-ansible-playbooks.zip`)
+- Goss testing files (`image-pipeline-goss-testing.zip`)
+
+### AWS Secrets Manager
+- WinRM credentials
+- AWS credentials for build process
+- SSH private keys
+
+### Security Groups
+- Existing security group: `it-linux-base`
+
+### IAM Dependencies
+- AWS managed policies (referenced in IAM policy documents)
+- Cross-account roles and permissions
+
+### AWS Service Dependencies
+- AWS Partition data (`aws_partition.current`)
+- AWS Caller Identity (`aws_caller_identity.current`)
+- AWS Region data (`aws_region.current`)
+- KMS keys (referenced via ARNs)
+
+### VPC Dependencies
+- Pre-existing Security Group IDs
+- Pre-existing Subnet IDs
+- Pre-existing VPC ID
+
+### State Backend Requirements
+- Pre-existing S3 bucket for state storage
+- Pre-existing DynamoDB table for state locking
+
+### Cross-Account Resources
+- AMI sharing account IDs
+- Cross-region S3 bucket replication configurations
+
+## Build Dependencies
+
+### Environment Variables
+- HTTP_PROXY
+- HTTPS_PROXY
+- NO_PROXY
+
+### Source Control
+- CodeCommit repository ARNs (when not using S3)
+- Default branch names (defaulting to "main")
+
+### Build Resources
+- Pre-existing ECR/Docker images
+  - `aws/codebuild/standard:7.0`
+
+### Configuration Files
+- Ansible playbook: `hello-world.yaml`
+- Goss profile: `base-test`
+
+## Workspace-Managed Resources
+Resources that are created in this workspace but outside the terraform-aws-image-pipeline module:
+
+### S3 Resources
+- Assets bucket (`aws_s3_bucket.assets_bucket`) - Used to store pipeline artifacts
+- Associated bucket policies and access controls
+
+### IAM Resources
+- Morpheus build user policy (`aws_iam_user_policy.morpheus_build_user`)
+- AMI sharing policies and roles
+
+### Parameter Store Resources
+- RHEL9 AMI parameters
+- Base image parameters
+- Ansible-related parameters
+
+### VPC Resources
+- VPC endpoints for AWS services
+- Associated security groups and routing configurations
+
+### Pipeline Configurations
+- Multiple pipeline definitions:
+  - Amazon Linux pipeline
+  - RHEL pipeline
+  - Morpheus application pipeline
+  - Docker image pipeline
+  - GitHub runner pipeline
+
+### Volume Configurations
+- Custom EBS volume mappings (e.g., for Morpheus deployments)
+  - Root volumes
+  - Application volumes
+  - Data volumes

external-dependencies.tf

@@ -0,0 +1,25 @@
+module "external_dependencies" {
+  source = "../terraform-aws-image-pipeline-external"
+
+  project_name      = "aws-image-pipeline"
+  assets_bucket_name = aws_s3_bucket.assets_bucket.bucket
+  state_bucket_name = local.state_config.bucket
+
+  pipeline_iam_arns = [
+    module.amazon_linux.iam_arn,
+    module.morpheus.iam_arn
+  ]
+
+  vpc_config = {
+    vpc_id             = local._vpc_config.vpc_id
+    region             = local._vpc_config.region
+    security_group_ids = local._vpc_config.security_group_ids
+    subnets           = local._vpc_config.subnets
+  }
+
+  # Add common tags
+  tags = {
+    Project     = "aws-image-pipeline"
+    Environment = local.environment
+  }
+}

linux-images.code-workspace

@@ -0,0 +1,13 @@
+{
+    "folders": [
+        {
+            "path": "."
+        },
+        {
+            "path": "../terraform-aws-image-pipeline"
+        },
+        {
+            "path": "../../terraform-aws-image-pipeline-external"
+        }
+    ]
+}

morpheus.tf

@@ -43,7 +43,6 @@ module "morpheus" {
     morpheus_version  = "7.0.10-1",
     shutdown_behavior = "stop"
   }
-  assets_bucket_name = aws_s3_bucket.assets_bucket.bucket
   image_volume_mapping = [
     {
       device_name           = "/dev/sda1"  # Root device

moved.tf

@@ -0,0 +1,41 @@
+# S3 Bucket moves
+moved {
+  from = aws_s3_bucket.state_bucket
+  to   = module.external_dependencies.aws_s3_bucket.state_bucket
+}
+
+moved {
+  from = aws_s3_bucket.assets_bucket
+  to   = module.external_dependencies.aws_s3_bucket.assets_bucket
+}
+
+moved {
+  from = aws_s3_bucket_server_side_encryption_configuration.state_bucket_encryption["state_bucket"]
+  to   = module.external_dependencies.aws_s3_bucket_server_side_encryption_configuration.state_bucket_encryption
+}
+
+moved {
+  from = aws_s3_bucket_server_side_encryption_configuration.state_bucket_encryption["assets_bucket"]
+  to   = module.external_dependencies.aws_s3_bucket_server_side_encryption_configuration.assets_bucket_encryption
+}
+
+moved {
+  from = aws_s3_bucket_policy.assets_bucket_policy
+  to   = module.external_dependencies.aws_s3_bucket_policy.assets_bucket_policy
+}
+
+# Security group moves
+moved {
+  from = aws_security_group.allow_amznlinux_cdn
+  to   = module.external_dependencies.aws_security_group.pipeline_security_group
+}
+
+moved {
+  from = aws_vpc_security_group_egress_rule.allow_all_traffic_ipv4
+  to   = module.external_dependencies.aws_vpc_security_group_egress_rule.allow_all_traffic_ipv4
+}
+
+moved {
+  from = aws_vpc_security_group_ingress_rule.allow_all_between_self
+  to   = module.external_dependencies.aws_vpc_security_group_ingress_rule.allow_self_traffic
+}