From b3c75d4d54fd52a3f41e858512446ed646b39a01 Mon Sep 17 00:00:00 2001 From: badra001 Date: Wed, 4 Mar 2026 08:57:00 -0500 Subject: [PATCH] add README --- .../python-tools/aws_config_editor/README.md | 589 ++++++++++++++++++ 1 file changed, 589 insertions(+) create mode 100755 local-app/python-tools/aws_config_editor/README.md diff --git a/local-app/python-tools/aws_config_editor/README.md b/local-app/python-tools/aws_config_editor/README.md new file mode 100755 index 00000000..52969400 --- /dev/null +++ b/local-app/python-tools/aws_config_editor/README.md @@ -0,0 +1,589 @@ +# aws_config_editor + +**Version 1.0.1** + +A command-line tool for bulk editing AWS CLI `config` and `credentials` files. Built for engineers managing dozens — or hundreds — of AWS profiles across commercial and GovCloud partitions who need more than a text editor but less than a full identity platform. + +--- + +## Why This Tool Exists + +The AWS CLI config file is an INI-style flat file. For a single developer with two or three accounts it's easy to manage by hand. For anyone running a large AWS Organization — 50, 150, 400+ accounts across commercial and GovCloud — it becomes a serious operational burden. + +Common pain points this tool addresses: + +- **Bulk cleanup**: SSO token refresh creates dozens of `[profile temp-*]` entries. Deleting them by hand, or with a fragile `sed` one-liner, risks corrupting the file. +- **Mass updates**: Rotating a base/source profile name, updating `role_session_name` conventions, or changing a region setting means touching every profile individually. +- **Consistent provisioning**: When you onboard a new account it should get the same standard profile structure as every other account in its class — without copy-paste errors. +- **Comment preservation**: Every other approach — Python's `configparser`, `aws configure set`, manual edits — silently discards your `#` comments and section dividers. This tool doesn't. + +--- + +## Features + +| Command | Description | +|----------|-------------| +| `list` | List profiles, optionally filtered by regex or substring. Optionally show all key=value pairs. | +| `delete` | Delete all profiles matching a pattern. | +| `edit` | Sed-style find/replace on a specific key's value across all matching profiles. | +| `create` | Create a new profile from inline values, by cloning an existing profile, or from a Jinja2 template. | + +**Safety features present on every mutating command:** + +- Automatic timestamped `.bak` backup created before any write +- `--dry-run` mode to preview all changes without touching the file +- Interactive confirmation prompt (bypass with `--yes` / `-y` for scripting) + +--- + +## Requirements + +### Python version + +Python **3.10 or newer** is required (uses `match`-compatible type union syntax internally). + +Verify your version: + +```bash +python3 --version +``` + +### Standard library dependencies + +The core tool uses only Python standard library modules — no `pip install` required for basic usage: + +| Module | Purpose | +|--------|---------| +| `argparse` | CLI argument parsing | +| `re` | Regex matching and substitution | +| `shutil` | File backup (copy) | +| `dataclasses` | Internal data model | +| `pathlib` | Cross-platform file paths | +| `datetime` | Backup filename timestamps | + +### Optional dependency: Jinja2 + +Jinja2 is only needed if you use `--template` or Jinja2 expressions in `--name`. It is **not required** for `list`, `delete`, `edit`, or plain `create`. + +```bash +pip install jinja2 + +# or, in a GovCloud / air-gapped environment where PyPI is unavailable: +pip install jinja2 --no-index --find-links /path/to/local/wheelhouse +``` + +--- + +## Installation + +The tool is a single self-contained Python script. There is no package to install. + +```bash +# Download the script +curl -O https://your-internal-repo/aws_config_editor.py +# or just copy it into place + +# Make it executable (optional) +chmod +x aws_config_editor.py + +# Verify +python3 aws_config_editor.py --version +# aws_config_editor.py 1.0.1 +``` + +**Recommended**: put it on your `$PATH` so you can call it from anywhere: + +```bash +cp aws_config_editor.py ~/bin/aws-config-editor +chmod +x ~/bin/aws-config-editor +aws-config-editor --version +``` + +--- + +## File Targeting + +By default the tool operates on `~/.aws/config`. Use `--file` / `-f` to target any file, including `~/.aws/credentials` or a non-standard path. + +```bash +# Default (config file) +python3 aws_config_editor.py list + +# Credentials file +python3 aws_config_editor.py --file ~/.aws/credentials list + +# Arbitrary path (e.g. a staging copy) +python3 aws_config_editor.py --file /tmp/aws-config-backup list + +# Short form +python3 aws_config_editor.py -f ~/.aws/credentials list +``` + +The tool auto-detects whether it's operating on a `config` file (which uses `[profile name]` section headers) or a `credentials` file (which uses bare `[name]` headers) based on whether the filename contains the word `credentials`. + +--- + +## Command Reference + +### `list` — Inspect profiles + +List all profiles, or filter to a subset. Non-destructive, never modifies the file. + +``` +list [--pattern REGEX] [--substring] [--show-keys] +``` + +**List everything:** + +```bash +python3 aws_config_editor.py list +``` + +**Filter by regex (full match against profile name):** + +```bash +# All profiles starting with "govcloud-" +python3 aws_config_editor.py list --pattern "govcloud-.*" + +# Profiles for a specific account ID anywhere in the name +python3 aws_config_editor.py list --pattern ".*123456789012.*" + +# Profiles ending in "-admin" or "-readonly" +python3 aws_config_editor.py list --pattern ".*-(admin|readonly)" +``` + +**Filter by substring (simpler, no regex):** + +```bash +python3 aws_config_editor.py list --pattern "govcloud" --substring +python3 aws_config_editor.py list --pattern "123456789012" --substring +``` + +**Show each profile's key=value pairs:** + +```bash +python3 aws_config_editor.py list --pattern "prod-.*" --show-keys +``` + +Example output: +``` +prod-east-app + region = us-east-1 + role_arn = arn:aws:iam::111122223333:role/AppRole + source_profile = base-commercial + role_session_name = ops-session + output = json +prod-east-db + region = us-east-1 + role_arn = arn:aws:iam::111122224444:role/DBRole + source_profile = base-commercial + role_session_name = ops-session + output = json + +2 profile(s). +``` + +--- + +### `delete` — Remove profiles matching a pattern + +``` +delete --pattern REGEX [--substring] [--dry-run] [--yes] +``` + +**Always dry-run first:** + +```bash +python3 aws_config_editor.py delete --pattern "temp-.*" --dry-run +``` + +Example output: +``` +3 profile(s) matched 'temp-.*': + - temp-audit-20240115 + - temp-audit-20240201 + - temp-migration-east + +DRY RUN — no changes written. +``` + +**Then apply:** + +```bash +python3 aws_config_editor.py delete --pattern "temp-.*" +# Prompts: Delete these 3 profile(s)? [y/N] +``` + +**Skip the prompt (for scripts and CI):** + +```bash +python3 aws_config_editor.py delete --pattern "temp-.*" --yes +``` + +**Real-world examples:** + +```bash +# Remove all profiles referencing a decommissioned account +python3 aws_config_editor.py delete --pattern ".*999988887777.*" + +# Clean up all GovCloud staging profiles +python3 aws_config_editor.py delete --pattern "govcloud-staging-.*" + +# Remove a specific named profile +python3 aws_config_editor.py delete --pattern "old-legacy-account" + +# Remove profiles whose names contain a ticket number (substring mode) +python3 aws_config_editor.py delete --pattern "PLAT-4821" --substring +``` + +A timestamped backup is written before any deletion, e.g. `~/.aws/config.20250115_142035.bak`. + +--- + +### `edit` — Bulk find/replace on key values + +``` +edit --pattern REGEX --key KEY --search TEXT --replace TEXT + [--regex] [--substring] [--dry-run] [--yes] +``` + +This is the workhorse command. It finds all profiles matching `--pattern`, locates the specified `--key` in each one, and performs a find/replace on its value. The rest of the file — other keys, comments, blank lines, unmatched profiles — is untouched. + +**Plain string replacement:** + +```bash +# Rename a base profile across all govcloud profiles +python3 aws_config_editor.py edit \ + --pattern "govcloud-.*" \ + --key source_profile \ + --search "base-govcloud-old" \ + --replace "base-govcloud-v2" + +# Update role_session_name across all profiles +python3 aws_config_editor.py edit \ + --pattern ".*" \ + --key role_session_name \ + --search "alice" \ + --replace "svc-automation" + +# Change default region for all prod profiles +python3 aws_config_editor.py edit \ + --pattern "prod-.*" \ + --key region \ + --search "us-east-1" \ + --replace "us-west-2" +``` + +**Regex substitution with `--regex`:** + +Uses Python `re.sub()` semantics. Back-references are supported with `\1`, `\2`, etc. + +```bash +# Rename session convention: "user_alice" → "svc_alice" +python3 aws_config_editor.py edit \ + --pattern ".*" \ + --key role_session_name \ + --search "^user_(.+)" \ + --replace "svc_\1" \ + --regex + +# Replace account ID embedded in a role ARN +python3 aws_config_editor.py edit \ + --pattern "acct-old-.*" \ + --key role_arn \ + --search "arn:aws:iam::111122223333:" \ + --replace "arn:aws:iam::444455556666:" \ + +# Upgrade role name suffix across all profiles (regex capture) +python3 aws_config_editor.py edit \ + --pattern ".*" \ + --key role_arn \ + --search "(arn:aws[^:]*:iam::[0-9]+:role/)OldRoleName" \ + --replace "\1NewRoleName" \ + --regex +``` + +**Preview before applying:** + +```bash +python3 aws_config_editor.py edit \ + --pattern "govcloud-.*" \ + --key source_profile \ + --search "old-base" \ + --replace "new-base" \ + --dry-run +``` + +Example output: +``` +4 profile(s) would be updated: + govcloud-prod-app + source_profile: 'old-base' → 'new-base' + govcloud-prod-db + source_profile: 'old-base' → 'new-base' + govcloud-staging-app + source_profile: 'old-base' → 'new-base' + govcloud-staging-db + source_profile: 'old-base' → 'new-base' + +DRY RUN — no changes written. +``` + +--- + +### `create` — Add a new profile + +``` +create --name NAME + [--source PROFILE] + [--template PATH] + [--set KEY=VALUE ...] + [--var KEY=VALUE ...] + [--dry-run] [--yes] +``` + +Three source modes — use one, or combine `--source`/`--template` with `--set` overrides: + +#### Mode 1: Inline key=value pairs + +Start from a blank profile and specify every key with `--set`: + +```bash +python3 aws_config_editor.py create \ + --name "acct-555566667777-admin" \ + --set "region=us-east-1" \ + --set "output=json" \ + --set "role_arn=arn:aws:iam::555566667777:role/AdminRole" \ + --set "source_profile=base-commercial" \ + --set "role_session_name=ops-session" +``` + +#### Mode 2: Clone an existing profile + +Copy all keys from an existing profile, then override what differs. Ideal when you have a "template profile" pattern or when onboarding accounts that differ only by account ID and role ARN: + +```bash +# Clone prod-east-app and update account-specific fields +python3 aws_config_editor.py create \ + --name "acct-888899990000-app" \ + --source "acct-111122223333-app" \ + --set "role_arn=arn:aws:iam::888899990000:role/AppRole" + +# Clone a GovCloud template profile +python3 aws_config_editor.py create \ + --name "govcloud-new-account-admin" \ + --source "govcloud-template-admin" \ + --set "role_arn=arn:aws-us-gov:iam::112233445566:role/AdminRole" \ + --set "role_session_name=ops-new-account" +``` + +#### Mode 3: Jinja2 template file + +Write a `.j2` template once and render it for every new account. Requires `pip install jinja2`. + +Create a template file, e.g. `~/.aws/templates/commercial_role.j2`: + +```jinja2 +region = {{ region | default('us-east-1') }} +output = json +role_arn = arn:aws:iam::{{ account_id }}:role/{{ role }} +source_profile = {{ source_profile | default('base-commercial') }} +role_session_name = {{ session_name | default('ops-session') }} +``` + +Then render it: + +```bash +python3 aws_config_editor.py create \ + --name "acct-{{ account_id }}-{{ role | lower }}" \ + --template ~/.aws/templates/commercial_role.j2 \ + --var "account_id=777788889999" \ + --var "role=AdminRole" +# Creates profile: acct-777788889999-adminrole + +# With an explicit name (no Jinja2 in --name) +python3 aws_config_editor.py create \ + --name "acct-777788889999-admin" \ + --template ~/.aws/templates/commercial_role.j2 \ + --var "account_id=777788889999" \ + --var "role=AdminRole" \ + --var "session_name=deploy-automation" +``` + +GovCloud template example (`govcloud_role.j2`): + +```jinja2 +region = us-gov-west-1 +output = json +role_arn = arn:aws-us-gov:iam::{{ account_id }}:role/{{ role }} +source_profile = {{ source_profile | default('base-govcloud') }} +role_session_name = {{ session_name | default('ops-session') }} +``` + +```bash +python3 aws_config_editor.py create \ + --name "govcloud-{{ account_id }}-{{ role | lower }}" \ + --template ~/.aws/templates/govcloud_role.j2 \ + --var "account_id=112233445566" \ + --var "role=ReadOnly" +``` + +**Dry-run a create:** + +```bash +python3 aws_config_editor.py create \ + --name "acct-new-test" \ + --source "acct-existing-prod" \ + --set "role_arn=arn:aws:iam::000011112222:role/TestRole" \ + --dry-run +``` + +Output: +``` +New profile 'acct-new-test': +[profile acct-new-test] +region = us-east-1 +output = json +role_arn = arn:aws:iam::000011112222:role/TestRole +source_profile = base-commercial +role_session_name = ops-session + +DRY RUN — no changes written. +``` + +--- + +## Comment Preservation + +This tool's parser is line-aware. Comments (lines starting with `#` or `;`) and blank lines inside and between profiles are preserved exactly as written. + +Example — given this config: + +```ini +# AWS CLI config — managed centrally, do not edit manually + +# ── Commercial ───────────────────────────────────────── +[profile prod-app] +region = us-east-1 +# role_arn points to the shared services account +role_arn = arn:aws:iam::111122223333:role/AppRole +source_profile = base +``` + +After running an `edit` or `delete`, the output file retains every comment exactly as it was. This matters for files that are committed to version control or that contain operational notes. + +> **Note:** `configparser` — the standard Python library module, and the underlying mechanism for `aws configure set` — silently drops all comments on every write. This tool does not use `configparser` for writes. + +--- + +## Backup Behavior + +Every command that modifies the file creates an automatic backup before writing. Backups are named with a timestamp suffix: + +``` +~/.aws/config.20250115_142035.bak +``` + +To restore a backup: + +```bash +cp ~/.aws/config.20250115_142035.bak ~/.aws/config +``` + +To list backups: + +```bash +ls -lt ~/.aws/config.*.bak +``` + +To clean up old backups (keep last 5): + +```bash +ls -t ~/.aws/config.*.bak | tail -n +6 | xargs rm -f +``` + +--- + +## Scripting and Automation + +The `--yes` / `-y` flag suppresses all interactive prompts, making the tool safe to use in scripts, CI pipelines, and Terraform/Ansible provisioners. + +```bash +# Non-interactive delete +python3 aws_config_editor.py delete --pattern "temp-.*" --yes + +# Non-interactive create +python3 aws_config_editor.py create \ + --name "acct-${ACCOUNT_ID}-admin" \ + --template ~/.aws/templates/commercial_role.j2 \ + --var "account_id=${ACCOUNT_ID}" \ + --var "role=AdminRole" \ + --yes + +# Loop over a list of account IDs +while IFS=',' read -r account_id account_name; do + python3 aws_config_editor.py create \ + --name "govcloud-${account_name}-admin" \ + --template ~/.aws/templates/govcloud_role.j2 \ + --var "account_id=${account_id}" \ + --var "role=AdminRole" \ + --yes +done < new_accounts.csv +``` + +Exit codes follow Unix conventions: `0` on success, `1` on error. Dry-run exits `0`. + +--- + +## Pattern Matching Reference + +All `--pattern` arguments are **Python regex full-match** by default, meaning the pattern must match the entire profile name. Add `--substring` for simple string containment. + +| Pattern | Mode | Matches | +|---------|------|---------| +| `govcloud-.*` | regex | Any profile starting with `govcloud-` | +| `.*-admin` | regex | Any profile ending with `-admin` | +| `acct-[0-9]{12}-.*` | regex | Profiles with a 12-digit account ID | +| `.*-(admin\|readonly)` | regex | Profiles ending in `-admin` or `-readonly` | +| `123456789012` | substring | Any profile whose name contains that account ID | +| `temp` | substring | Any profile whose name contains `temp` | + +> **Tip:** Always run `list --pattern "..."` first to confirm what will be affected before running `delete` or `edit`. + +--- + +## Global Options + +``` +--file PATH, -f PATH Target a specific config or credentials file + (default: ~/.aws/config) +--version Print version and exit +--help, -h Show help +``` + +Each subcommand also accepts: + +``` +--dry-run Preview without writing (delete, edit, create) +--yes, -y Skip confirmation prompt (delete, edit, create) +--pattern REGEX Filter by profile name +--substring Use plain substring matching instead of regex +``` + +--- + +## Limitations and Notes + +- **Multi-line values** (continuation lines indented with whitespace, valid in some INI parsers) are stored as raw lines and preserved, but the `edit` command will not match them — only single-line `key = value` entries are eligible for substitution. +- **`[sso-session]` blocks** are treated as ordinary sections. The tool will not corrupt them but also does not have special awareness of their semantics. +- **Credentials file** — when targeting `~/.aws/credentials` with `--file`, section headers use bare `[profile-name]` format (no `profile ` prefix). The tool handles this automatically. +- **Encoding** — files are read and written as UTF-8. If your config file uses a different encoding, conversion is required first. + +--- + +## Version History + +| Version | Notes | +|---------|-------| +| 1.0.1 | Fixed `--help` argparse conflict on `--pattern`; added `--version` flag | +| 1.0.0 | Initial release: `list`, `delete`, `edit`, `create` with comment preservation |