Skip to content

Commit

Permalink
refactor and add rds
Browse files Browse the repository at this point in the history
  • Loading branch information
morga471 committed May 28, 2026
1 parent 4a4f46b commit 09b08cc
Show file tree
Hide file tree
Showing 70 changed files with 1,736 additions and 401 deletions.
40 changes: 40 additions & 0 deletions .github/workflows/terraform-validate.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: Terraform Validate

on:
pull_request:
push:
branches:
- main

jobs:
validate:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
dir:
- modules/product
- modules/ec2
- modules/ec2/linux
- modules/ec2/windows
- modules/rds
- modules/s3
- examples/ec2/linux
- examples/rds
- examples/s3
- examples/ecs

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Setup Terraform
uses: hashicorp/setup-terraform@v3
with:
terraform_version: 1.9.8

- name: Terraform Init
run: terraform -chdir=${{ matrix.dir }} init -backend=false -input=false -no-color

- name: Terraform Validate
run: terraform -chdir=${{ matrix.dir }} validate -no-color
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Versions

* v1.0.0 -- {{ yyyy-mm-dd }}
* v1.0.0 -- 2026-05-28
- initial creation

88 changes: 79 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,10 @@ This module provides simplified Terraform wrappers for provisioning AWS Service

The module uses a layered architecture:
- **product module** - Base module handling Service Catalog provisioning (internal use)
- **ec2 module** - EC2-specific wrapper with instance parameters and VPC networking
- **ec2 module** - Shared EC2 implementation with parameter mapping and networking resolution
- **ec2/linux module** - Linux EC2 wrapper that preserves Linux defaults
- **ec2/windows module** - Windows EC2 wrapper that preserves Windows defaults
- **rds module** - RDS-specific wrapper with database parameters
- **s3 module** - S3-specific wrapper with bucket parameters and data classification

Each wrapper module provides:
Expand All @@ -30,29 +33,38 @@ Each wrapper module provides:
aws-servicecatalog/
├── modules/
│ ├── product/ # Base Service Catalog provisioning module (internal)
│ ├── ec2/ # EC2 instance provisioning wrapper
│ ├── ec2/
│ │ ├── linux/ # Linux EC2 compatibility wrapper
│ │ └── windows/ # Windows EC2 compatibility wrapper
│ ├── rds/ # RDS provisioning wrapper
│ └── s3/ # S3 bucket provisioning wrapper
└── examples/
├── ec2/linux/ # EC2 Linux instance example
├── rds/ # RDS instance example
└── s3/ # S3 bucket example
```

### Currently Implemented

-**modules/ec2/** - Provision EC2 instances from Service Catalog
-**modules/ec2/** - Shared EC2 provisioning implementation
-**modules/ec2/linux/** - Linux EC2 compatibility wrapper
-**modules/ec2/windows/** - Windows EC2 compatibility wrapper
-**modules/rds/** - Provision RDS instances from Service Catalog
-**modules/s3/** - Provision S3 buckets from Service Catalog
-**modules/product/** - Base module for Service Catalog provisioning (internal use only)

### Planned/Future Enhancements

-**RDS module** - Provision RDS databases
-**ECS module** - Provision ECS services
-**Service actions** - Execute service actions (start, stop, reboot)

## Usage

See individual module README files for detailed documentation:
- [EC2 Module Documentation](modules/ec2/README.md)
- [EC2 Shared Module Documentation](modules/ec2/README.md)
- [EC2 Linux Module Documentation](modules/ec2/linux/README.md)
- [EC2 Windows Module Documentation](modules/ec2/windows/README.md)
- [RDS Module Documentation](modules/rds/README.md)
- [S3 Module Documentation](modules/s3/README.md)
- [Product Module Documentation](modules/product/README.md)

Expand All @@ -64,14 +76,18 @@ module "ec2_instance" {
# Product identity
provisioned_product_name = "my-web-server"
portfolio_id = "port-xxxxxxxxxxxx"
product_id = "prod-xxxxxxxxxxxx"
# EC2-specific parameters
instance_type = "t3.small"
os_name = "RHEL9"
requires_backup = "yes"
power_schedule = "Always_On"
# Networking (auto-discovers VPC/subnets by name tag)
# Networking (explicit IDs take precedence over lookup values)
subnet_ids = ["subnet-0123456789abcdef0"]
vpc_id = "vpc-0123456789abcdef0"
vpc_name = "my-vpc"
subnets_name = "*-apps-*"
Expand Down Expand Up @@ -102,6 +118,8 @@ module "s3_bucket" {
# Product identity
provisioned_product_name = "my-data-bucket"
portfolio_id = "port-xxxxxxxxxxxx"
product_id = "prod-xxxxxxxxxxxx"
# S3-specific parameters
bucket_name = "my-data-lake"
Expand All @@ -127,20 +145,72 @@ module "s3_bucket" {
- Terraform >= 1.0
- AWS Provider >= 6.0

## Discovering Service Catalog IDs With AWS CLI

If you do not already know `portfolio_id` and `product_id`, you can resolve them from your current AWS environment.

Prerequisites:
- AWS CLI configured for the target account/org
- IAM permission to read Service Catalog portfolios/products

```bash
# Set your context
AWS_REGION="us-east-1"
PORTFOLIO_NAME="My Service Catalog Portfolio"
PRODUCT_NAME="My Product Name"

# 1) Get portfolio ID by display name
PORTFOLIO_ID=$(aws servicecatalog list-portfolios \
--region "$AWS_REGION" \
--query "PortfolioDetails[?DisplayName=='${PORTFOLIO_NAME}'].Id | [0]" \
--output text)

# 2) Get product ID by product name within that portfolio
PRODUCT_ID=$(aws servicecatalog search-products-as-admin \
--region "$AWS_REGION" \
--portfolio-id "$PORTFOLIO_ID" \
--query "ProductViewDetails[?ProductViewSummary.Name=='${PRODUCT_NAME}'].ProductViewSummary.ProductId | [0]" \
--output text)

# 3) Optional: get latest active provisioning artifact ID
PROVISIONING_ARTIFACT_ID=$(aws servicecatalog list-provisioning-artifacts \
--region "$AWS_REGION" \
--product-id "$PRODUCT_ID" \
--query "ProvisioningArtifactDetails[?Active==\`true\`]|sort_by(@,&CreatedTime)[-1].Id" \
--output text)

# 4) Optional: get launch path ID
PATH_ID=$(aws servicecatalog list-launch-paths \
--region "$AWS_REGION" \
--product-id "$PRODUCT_ID" \
--query "LaunchPathSummaries[0].Id" \
--output text)

echo "portfolio_id=$PORTFOLIO_ID"
echo "product_id=$PRODUCT_ID"
echo "provisioning_artifact_id=$PROVISIONING_ARTIFACT_ID"
echo "path_id=$PATH_ID"
```

Note:
- If `search-products-as-admin` is restricted in your org, use the Service Catalog console to locate product IDs or ask your portfolio admins for read access.

## Examples

Complete working examples with terraform.tfvars.example files are available in:
- [examples/ec2/linux/](examples/ec2/linux/) - EC2 Linux instance provisioning
- [examples/rds/](examples/rds/) - RDS instance provisioning
- [examples/s3/](examples/s3/) - S3 bucket provisioning

## Architecture

This module follows a base + specialized wrapper pattern:

1. **product module** provides generic Service Catalog provisioning
2. **ec2/s3 modules** add product-specific parameters and data sources
3. Each wrapper handles parameter mapping to CloudFormation format
4. Networking data sources are only in modules that need them (EC2, not S3)
2. **ec2/rds/s3 modules** add product-specific parameters and data sources
3. **ec2/linux and ec2/windows wrappers** provide OS-specific defaults on top of shared ec2
4. Each wrapper handles parameter mapping to CloudFormation format
5. Networking data sources are used only in modules that need them (EC2 and RDS, not S3)

## Contributing

Expand Down
2 changes: 1 addition & 1 deletion REQUIREMENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
- Terraform >= 1.0

## Provider Requirements
- AWS Provider >= 5.0
- AWS Provider >= 6.0

## Permissions

Expand Down
26 changes: 18 additions & 8 deletions examples/ec2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@

This example demonstrates how to provision an EC2 instance from AWS Service Catalog.

Networking behavior is standardized across wrappers:
- explicit `subnet_ids`/`vpc_id` take precedence
- lookup via `vpc_name`/`subnets_name` is used when explicit IDs are not provided

## Usage

```bash
Expand All @@ -14,17 +18,19 @@ terraform apply

```hcl
module "ec2_instance" {
source = "../../ec2"
source = "../../modules/ec2/linux"
provisioned_product_name = "my-web-server"
portfolio_name_pattern = "edl-portfolio"
product_name_pattern = "linux-product"
portfolio_id = "port-xxxxxxxxxxxx"
product_id = "prod-xxxxxxxxxxxx"
# Explicit networking (preferred for deterministic plans)
subnet_ids = ["subnet-12345"]
vpc_id = "vpc-12345"
parameters = {
InstanceType = "t3.medium"
KeyName = "my-key"
SubnetId = "subnet-12345"
VpcId = "vpc-12345"
}
tags = {
Expand All @@ -48,15 +54,19 @@ output "instance_private_ip" {
```hcl
# Provision the instance
module "ec2_instance" {
source = "../../ec2"
source = "../../modules/ec2/linux"
provisioned_product_name = "my-web-server"
product_name_pattern = "linux-product"
portfolio_id = "port-xxxxxxxxxxxx"
product_id = "prod-xxxxxxxxxxxx"
# Lookup networking
vpc_name = "my-vpc"
subnets_name = "*-apps-*"
parameters = {
InstanceType = "t3.medium"
KeyName = "my-key"
SubnetId = "subnet-12345"
}
}
Expand Down
49 changes: 40 additions & 9 deletions examples/ec2/linux/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@ This example demonstrates how to provision a Census RHEL EC2 instance using the

## CloudFormation Template Reference

This example implements the parameters from the **2-0-2.yaml** CloudFormation template, which provides a comprehensive Census EC2 instance provisioning workflow including:
This example implements the parameters from the EC2 Service Catalog product template, with a standardized networking model shared across wrappers.

- **EC2 Configuration**: Instance type, hostname, OS version (RHEL8/RHEL9), disk sizing
- **Network Setup**: VPC, availability zone, security groups
- **Network Setup**: Explicit subnet_ids/vpc_id inputs or lookup via vpc_name/subnets_name
- **Storage**: Apps disk plus optional extra volumes for /data mounts
- **Creator/Contact Info**: JBID, email addresses for notifications
- **Compliance Tags**: FISMA ID, Title Data classifications
Expand All @@ -28,7 +28,10 @@ This example implements the parameters from the **2-0-2.yaml** CloudFormation te
```

2. Edit `terraform.tfvars` with your specific values:
- Set your VPC ID and Availability Zone
- Provide required Service Catalog identifiers: portfolio_id and product_id
- Choose one networking approach:
- explicit `subnet_ids` and `vpc_id`, or
- lookup via `vpc_name` and `subnets_name`
- Provide census project identifier
- Set creator/contact email addresses
- Customize instance type, hostname, OS version as needed
Expand All @@ -40,17 +43,46 @@ This example implements the parameters from the **2-0-2.yaml** CloudFormation te
terraform apply
```

### Lookup IDs (AWS CLI)

```bash
AWS_REGION="us-east-1"
PORTFOLIO_NAME="My Service Catalog Portfolio"
PRODUCT_NAME="My Product Name"
PORTFOLIO_ID=$(aws servicecatalog list-portfolios \
--region "$AWS_REGION" \
--query "PortfolioDetails[?DisplayName=='${PORTFOLIO_NAME}'].Id | [0]" \
--output text)
PRODUCT_ID=$(aws servicecatalog search-products-as-admin \
--region "$AWS_REGION" \
--portfolio-id "$PORTFOLIO_ID" \
--query "ProductViewDetails[?ProductViewSummary.Name=='${PRODUCT_NAME}'].ProductViewSummary.ProductId | [0]" \
--output text)
echo "portfolio_id=$PORTFOLIO_ID"
echo "product_id=$PRODUCT_ID"
```

See the main module docs for optional provisioning artifact/path lookups: [Discovering Service Catalog IDs With AWS CLI](../../../README.md#discovering-service-catalog-ids-with-aws-cli).

## Key Parameters

### Required Parameters
- `project_name` - Census project identifier (org_project_env_type-accountid)
- `vpc_id` - VPC where instance will be deployed
- `availability_zone` - AZ for the instance
- `instance_hostname` - Local hostname (3-16 chars)
- `creator_jbid` - Census user ID of creator
- `contact_email` - Provisioning user's email
- `incident_poc_email` - Team distribution list email
### Networking Parameters
- `subnet_ids` + `vpc_id` - Explicit network placement (takes precedence)
- `vpc_name` + `subnets_name` - Lookup-based placement when explicit IDs are not set
- `availability_zone` - Derived from the first selected subnet
### Other Required Parameters
- `instance_hostname` - Local hostname (3-16 chars)
### Optional Parameters
- `instance_type` - EC2 type (default: t3.small)
- `os_name` - OS version (default: RHEL9)
Expand Down Expand Up @@ -109,6 +141,5 @@ Note: If `enable_deletion_protection` is set to true, you must change it to fals

## Related Documentation

- CloudFormation Template: `../../templates/products/ec2/instance-linux/2-0-2.yaml`
- Module Documentation: `../../ec2/README.md`
- Parent Module: `../../ec2/main.tf`
- Module Documentation: `../../../modules/ec2/linux/README.md`
- Parent Module: `../../../modules/ec2/linux/main.tf`
4 changes: 3 additions & 1 deletion examples/ec2/linux/census-rhel-instance.tf
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,12 @@
# Aligns with the 2-0-2.yaml CloudFormation product template.

module "ec2_instance" {
source = "../../../modules/ec2"
source = "../../../modules/ec2/linux"

# === Product Identity ===
provisioned_product_name = var.provisioned_product_name
portfolio_id = var.portfolio_id
product_id = var.product_id

# === Required EC2-Specific Parameters ===
instance_type = var.instance_type
Expand Down
1 change: 0 additions & 1 deletion examples/ec2/linux/main.tf

This file was deleted.

Loading

0 comments on commit 09b08cc

Please sign in to comment.