add readme; exclude org-based services

local-app/python-tools/aws-service-linked-roles/README.md

@@ -0,0 +1,108 @@
+# AWS Service Linked Role to Terraform Generator
+
+A Python-based utility to audit AWS Service-Linked Roles (SLRs) and generate Terraform import blocks and variable files. This tool is designed to help teams transition existing, manually-created service-linked roles into Terraform management using `for_each` patterns.
+
+## How It Works
+
+The script performs the following steps:
+1. **AWS Audit**: Connects to your AWS account and lists all IAM roles located under the `/aws-service-role/` path.
+2. **Partition & Account Detection**: Dynamically identifies whether the account is in the `aws` (Commercial) or `aws-us-gov` (GovCloud) partition via STS to ensure generated ARNs are correct.
+3. **Principal Extraction**: Inspects the Trust Relationship of each role to identify the specific AWS Service Principal (e.g., `s3.amazonaws.com`).
+4. **Code Generation**: 
+   - Generates an `import.tf` file containing Terraform `import` blocks using the correct partition-aware ARN format.
+   - Generates a `.tfvars` file containing a list of service names.
+
+## Installation
+
+### Prerequisites
+- Python 3.8+
+- AWS CLI configured with permissions: `iam:ListRoles`, `iam:GetRole`, and `sts:GetCallerIdentity`.
+
+### Setup
+1. Save the script as `aws-slr-generate.py`.
+2. Install dependencies:
+   ```bash
+   pip install boto3
+   ```
+
+## Usage
+
+### Basic Usage
+```bash
+python aws-slr-generate.py
+```
+
+### Advanced Usage (GovCloud / Custom Files)
+```bash
+python aws-slr-generate.py --profile gov-admin --region us-gov-west-1 --import-file slr_imports.tf
+```
+
+### Dry Run
+```bash
+python aws-slr-generate.py --dry-run
+```
+
+### Usage with Filtering
+To run the script while excluding specific organization-level services and respecting existing Terraform tags:
+
+```bash
+# Using the default internal ignore list
+python aws-slr-generate.py --profile my-profile
+
+# Using a custom CSV list of services to ignore
+python aws-slr-generate.py --organization-services org-managed-list.csv
+```
+
+### Filtering Logic
+The script now applies a three-tier filter before creating an import:
+1.  **Tag Check**: Looks for `boc:created_by: terraform`.
+2.  **Service Check**: Compares the service principal against the internal `DEFAULT_IGNORE_SERVICES` list.
+3.  **Manual Overrides**: Includes any services found in the optional `--organization-services` CSV.
+
+### Command Line Arguments
+- `--profile`: AWS CLI profile to use.
+- `--region`: AWS region.
+- `--import-file`: Filename for import blocks (default: `import.tf`).
+- `--variables-file`: Filename for tfvars (default: `variables.service-linked-roles.auto.tfvars`).
+- `--dry-run`: Output to terminal instead of files.
+- `--organization-services`: Organization managed service linked roles (excluded from import and list)
+
+## Terraform Integration
+
+```hcl
+resource "aws_iam_service_linked_role" "roles" {
+  for_each         = toset(var.service_linked_roles)
+  aws_service_name = each.key
+}
+```
+
+## Changelog
+
+### [1.0.5] - 2026-04-14
+* **Added**: Built-in ignore list for organization-managed services (e.g., GuardDuty, Security Hub, SSO).
+* **Added**: Support for `--organization-services` CSV file to dynamically expand the ignore list.
+* **Added**: Tag-based filtering. Roles with the tag `boc:created_by = terraform` are now skipped to prevent duplicate management.
+* **Fixed**: ARN generation logic. The script now uses the full role ARN returned by AWS instead of a constructed string, ensuring the role name is included in the Terraform `import` block.
+* **Improved**: Added inline comments in the generated `import.tf` file to explain why specific roles were skipped.
+
+### [1.0.4] - 2026-04-14
+* **Fixed**: Dynamic Partition Detection. Replaced the hardcoded `aws` partition with a dynamic lookup via STS.
+* **Improved**: The script now correctly generates `arn:aws-us-gov` for GovCloud environments and `arn:aws` for commercial environments.
+* **Updated**: Improved `get_session_info` function to reliably extract partition data from the caller identity ARN.
+
+### [1.0.3] - 2026-04-14
+- **Added**: Dynamic Partition Detection (supports `aws`, `aws-us-gov`, etc.).
+- **Updated**: Renamed script to `aws-slr-generate.py` in documentation.
+- **Improved**: ARN generation logic to use detected partition.
+
+### [1.0.2] - 2026-04-14
+- **Added**: `--dry-run` option.
+- **Improved**: Summary output formatting.
+
+### [1.0.1] - 2026-04-14
+- **Added**: CLI arguments for profile, region, and custom filenames.
+- **Added**: Automatic Account ID detection.
+
+### [1.0.0] - 2026-04-14
+- **Initial Release**: Basic SLR scraping and file generation.
+

local-app/python-tools/aws-service-linked-roles/aws-slr-generate.py

@@ -1,119 +1,138 @@
-#!/bin/env python
-
 import boto3
 import argparse
 import sys
+import csv
 from botocore.exceptions import ClientError, ProfileNotFound
 
-VERSION = "1.0.4"
+VERSION = "1.0.5"
+
+# Default list of organization-managed services to ignore
+DEFAULT_IGNORE_SERVICES = [
+    "access-analyzer", "guardduty", "inspector2", "acm", "cloudtrail",
+    "compute-optimizer", "member.org.stacksets.cloudformation",
+    "config-multiaccountsetup", "fms", "ipam", "organizations",
+    "securityhub", "sso"
+]
 
 def get_session_info(session):
-    """
-    Retrieves the Account ID and Partition from the current session.
-    """
     sts = session.client('sts')
     identity = sts.get_caller_identity()
-    arn = identity['Arn']
-
-    # ARN format: arn:partition:service:region:account-id:resource-type/resource-id
-    # We split by ':' and take the second element for the partition.
-    arn_parts = arn.split(':')
-    partition = arn_parts[1]
-    account_id = identity['Account']
-
-    return account_id, partition
+    partition = identity['Arn'].split(':')[1]
+    return identity['Account'], partition
+
+def load_ignored_services(csv_path):
+    ignored = set(DEFAULT_IGNORE_SERVICES)
+    if csv_path and os.path.exists(csv_path):
+        try:
+            with open(csv_path, mode='r') as f:
+                reader = csv.reader(f)
+                for row in reader:
+                    if row:
+                        ignored.add(row[0].strip())
+        except Exception as e:
+            print(f"Warning: Could not read organization services file: {e}")
+    return ignored
 
 def generate_slr_terraform():
     parser = argparse.ArgumentParser(description="Audit AWS SLRs and generate TF import/vars files.")
     parser.add_argument("--profile", help="AWS CLI profile to use")
-    parser.add_argument("--region", help="AWS region (e.g., us-gov-west-1)")
+    parser.add_argument("--region", help="AWS region")
     parser.add_argument("--import-file", default="import.tf", help="Output filename for import blocks")
     parser.add_argument("--variables-file", default="variables.service-linked-roles.auto.tfvars", help="Output filename for tfvars")
+    parser.add_argument("--organization-services", help="CSV file containing org services to ignore")
     parser.add_argument("--dry-run", action="store_true", help="Print content to stdout instead of files")
     parser.add_argument("--version", action="version", version=f"%(prog)s {VERSION}")
 
     args = parser.parse_args()
 
     try:
-        # Initialize Session
         session = boto3.Session(profile_name=args.profile, region_name=args.region)
         iam = session.client('iam')
-
-        # Detect Partition and Account ID dynamically
         account_id, partition = get_session_info(session)
+        ignored_services = load_ignored_services(args.organization_services)
 
-        service_names = []
-        slr_count = 0
+        service_names_to_import = []
+        import_content = [f"# Generated by aws-slr-generate v{VERSION}\n"]
+        slr_found_count = 0
+        imported_count = 0
 
-        # Paginate through IAM roles with the SLR path
         paginator = iam.get_paginator('list_roles')
         for page in paginator.paginate(PathPrefix='/aws-service-role/'):
             for role in page['Roles']:
+                slr_found_count += 1
+                role_name = role['RoleName']
+                role_arn = role['Arn']
+
+                # 1. Check Trust Policy for Service Principal
                 trust_policy = role.get('AssumeRolePolicyDocument', {})
+                service_principals = []
                 for statement in trust_policy.get('Statement', []):
                     principal = statement.get('Principal', {}).get('Service')
                     if principal:
-                        slr_count += 1
-                        # Service can be a string or a list
                         if isinstance(principal, list):
-                            service_names.extend(principal)
+                            service_principals.extend(principal)
                         else:
-                            service_names.append(principal)
+                            service_principals.append(principal)
+
+                if not service_principals:
+                    continue
+
+                # Using the first service principal as the primary key
+                primary_service = service_principals[0]
 
-        # De-duplicate and sort
-        unique_services = sorted(list(set(service_names)))
+                # 2. Check for Tags (boc:created_by == terraform)
+                try:
+                    tags_response = iam.list_role_tags(RoleName=role_name)
+                    role_tags = {tag['Key']: tag['Value'] for tag in tags_response.get('Tags', [])}
+                except ClientError:
+                    role_tags = {}
 
-        # Prepare File Content
-        import_content = [f"# Generated by aws-slr-generate v{VERSION}\n"]
-        for service in unique_services:
-            import_content.append(f'import {{\n')
-            import_content.append(f'  to = aws_iam_service_linked_role.roles["{service}"]\n')
-            # Use the detected partition here
-            import_content.append(f'  id = "arn:{partition}:iam::{account_id}:role/aws-service-role/{service}"\n')
-            import_content.append(f'}}\n\n')
+                # 3. Filtering Logic
+                is_ignored = any(ignored in primary_service for ignored in ignored_services)
+                is_managed_by_tf = role_tags.get('boc:created_by') == 'terraform'
+
+                if is_ignored:
+                    import_content.append(f"# Skipping {primary_service}: Organization-managed service\n")
+                elif is_managed_by_tf:
+                    import_content.append(f"# Skipping {primary_service}: Role '{role_name}' already managed by terraform (tag detected)\n")
+                else:
+                    import_content.append(f'import {{\n')
+                    import_content.append(f'  to = aws_iam_service_linked_role.roles["{primary_service}"]\n')
+                    import_content.append(f'  id = "{role_arn}"\n')
+                    import_content.append(f'}}\n\n')
+                    service_names_to_import.append(primary_service)
+                    imported_count += 1
 
+        # Prepare Vars Content
+        service_names_to_import = sorted(list(set(service_names_to_import)))
         vars_content = [f"# Generated by aws-slr-generate v{VERSION}\n", "service_linked_roles = [\n"]
-        for service in unique_services:
+        for service in service_names_to_import:
             vars_content.append(f'  "{service}",\n')
         vars_content.append("]\n")
 
-        # Write or Print
         if args.dry_run:
             print("\n" + "="*20 + " DRY RUN: IMPORT FILE " + "="*20)
             print("".join(import_content))
             print("="*20 + " DRY RUN: VARIABLES FILE " + "="*20)
             print("".join(vars_content))
-            print("="*65 + "\n")
         else:
             with open(args.import_file, 'w') as f:
                 f.writelines(import_content)
             with open(args.variables_file, 'w') as f:
                 f.writelines(vars_content)
 
-        # Summary Output
         print("-" * 45)
         print(f"SLR-to-TF Generator v{VERSION}")
         print("-" * 45)
         print(f"AWS Partition:    {partition}")
         print(f"AWS Account:      {account_id}")
-        print(f"Roles Found:      {slr_count}")
-        print(f"Unique Services:  {len(unique_services)}")
-
-        if args.dry_run:
-            print(f"Mode:             DRY RUN (Files not created)")
-        else:
-            print(f"Import File:      {args.import_file}")
-            print(f"Variables File:   {args.variables_file}")
+        print(f"Total SLRs Found: {slr_found_count}")
+        print(f"Imports Created:  {imported_count}")
+        print(f"Ignored/Skipped:  {slr_found_count - imported_count}")
         print("-" * 45)
 
-    except ProfileNotFound:
-        print(f"Error: Profile '{args.profile}' not found.")
-        sys.exit(1)
-    except ClientError as e:
-        print(f"AWS Error: {e}")
-        sys.exit(1)
-    except Exception as e:
-        print(f"Unexpected error: {e}")
+    except (ProfileNotFound, ClientError) as e:
+        print(f"Error: {e}")
         sys.exit(1)
 
 if __name__ == "__main__":