-
Notifications
You must be signed in to change notification settings - Fork 8
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
16 changed files
with
1,669 additions
and
1,061 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,171 +1,101 @@ | ||
| # AWS Resource Management Utilities | ||
| # AWS Resource Management Tool | ||
|
|
||
| A collection of Python utilities for managing and interacting with AWS resources across multiple accounts, with special support for GovCloud environments and AWS SSO. | ||
| ## Overview | ||
| The AWS Resource Management Tool is a Python-based utility designed to discover, start, and stop AWS resources across multiple accounts, primarily in GovCloud environments. Its primary purpose is cost management by automating resource state management. | ||
|
|
||
| ## Features | ||
|
|
||
| - Cross-account resource management | ||
| - Dynamic AWS partition detection (supports commercial AWS, GovCloud, AWS China) | ||
| - AWS SSO profile integration | ||
| - Fallback mechanisms for authentication | ||
| - Support for AWS Organizations | ||
|
|
||
| ## Requirements | ||
|
|
||
| - Python 3.6+ | ||
| - boto3 | ||
| - AWS credentials configured through AWS SSO or with standard credentials | ||
|
|
||
| ## Installation | ||
|
|
||
| 1. Clone this repository or copy the files to your desired location | ||
| 2. Install required dependencies: | ||
| - Multi-account and multi-region support | ||
| - Resource discovery and management for EC2, RDS, EKS, EMR, and ECR | ||
| - AWS SSO and role-based authentication | ||
| - CLI for easy interaction | ||
| - Dry-run mode for safe testing | ||
| - CSV reporting for resource actions | ||
|
|
||
| ## Package Structure | ||
| ``` | ||
| pip install boto3 | ||
| aws_resource_management/ # Main package | ||
| ├── __init__.py | ||
| ├── aws_utils.py # AWS credential/authentication utilities | ||
| ├── cli.py # Command-line interface | ||
| ├── core.py # Main business logic (ResourceManager) | ||
| ├── discovery.py # Resource discovery logic | ||
| ├── managers/ # Resource-specific managers | ||
| │ ├── base.py # Base resource manager class | ||
| │ ├── ec2.py # EC2-specific implementation | ||
| │ ├── eks.py # EKS-specific implementation | ||
| │ ├── emr.py # EMR-specific implementation | ||
| │ ├── rds.py # RDS-specific implementation | ||
| │ └── ecr.py # ECR-specific implementation | ||
| ├── reporting.py # Reporting utilities | ||
| └── logging_setup.py # Logging configuration | ||
| ``` | ||
|
|
||
| ## Command-Line Usage | ||
|
|
||
| The package provides a command-line interface for managing AWS resources: | ||
|
|
||
| ```bash | ||
| # Stop all resources in dry-run mode | ||
| aws-resource-mgmt --stop --dry-run --resource-type all | ||
| ## Installation | ||
|
|
||
| # Start EC2 instances | ||
| aws-resource-mgmt --start --resource-type ec2 | ||
| 1. Clone the repository: | ||
| ```bash | ||
| git clone <repository-url> | ||
| cd <repository-folder> | ||
| ``` | ||
|
|
||
| # Stop RDS instances in specific regions | ||
| aws-resource-mgmt --stop --resource-type rds --region us-east-1 --region us-west-2 | ||
| ``` | ||
| 2. Install dependencies: | ||
| ```bash | ||
| make install-dev | ||
| ``` | ||
|
|
||
| ## Usage | ||
|
|
||
| ### Basic Usage | ||
|
|
||
| ```python | ||
| import aws_resource_management.aws_utils as aws_utils | ||
|
|
||
| # Get an EC2 client for a specific account | ||
| ec2_client = aws_utils.create_boto3_client_for_account( | ||
| '123456789012', # AWS account ID | ||
| 'ec2', # AWS service | ||
| 'us-gov-west-1' # AWS region | ||
| ) | ||
|
|
||
| # Use the client | ||
| instances = ec2_client.describe_instances() | ||
| ``` | ||
|
|
||
| ### Using AWS SSO Profiles | ||
|
|
||
| The library will automatically find and use appropriate AWS SSO profiles from your AWS config file (`~/.aws/config`). It will: | ||
|
|
||
| 1. Look for profiles matching the requested account ID | ||
| 2. If a role_name is specified, it will find a profile with that role | ||
| 3. Otherwise, it will prefer profiles with admin permissions | ||
| 4. Fall back to role assumption if no profile is found or profile authentication fails | ||
|
|
||
| ### Key Functions | ||
|
|
||
| #### `create_boto3_client_for_account(account_id, service, region=None, role_name=None)` | ||
|
|
||
| Creates a boto3 client for the specified AWS service in the target account. | ||
|
|
||
| - `account_id`: Target AWS account ID | ||
| - `service`: AWS service name (e.g., 'ec2', 'rds') | ||
| - `region`: AWS region (defaults to config.DEFAULT_REGION) | ||
| - `role_name`: Specific role to assume or look for in SSO profiles | ||
|
|
||
| #### `get_partition_for_region(region)` | ||
|
|
||
| Determines the correct AWS partition for a given region. | ||
|
|
||
| - `region`: AWS region name | ||
| - Returns: 'aws', 'aws-us-gov', or 'aws-cn' | ||
|
|
||
| #### `find_profile_for_account(account_id, role_name=None)` | ||
|
|
||
| Finds an SSO profile in the AWS config file for the specified account. | ||
|
|
||
| - `account_id`: AWS account ID to find profile for | ||
| - `role_name`: Optional preferred role name | ||
| - Returns: Profile name if found, None otherwise | ||
|
|
||
| #### `assume_role(account_id, region=None, role_name=None)` | ||
|
|
||
| Gets credentials for the specified account, trying SSO profiles first. | ||
|
|
||
| - `account_id`: AWS account ID | ||
| - `region`: AWS region for determining partition | ||
| - `role_name`: Role name to assume | ||
| - Returns: Credentials dictionary | ||
|
|
||
| #### `get_organization_accounts(exclude_accounts=None)` | ||
|
|
||
| Gets a list of accounts in the AWS organization. | ||
|
|
||
| - `exclude_accounts`: List of account IDs to exclude | ||
| - Returns: List of dictionaries with account ID and name | ||
|
|
||
| ## Configuration | ||
|
|
||
| Create a `config.py` file with the following variables: | ||
|
|
||
| ```python | ||
| # Default AWS region to use | ||
| DEFAULT_REGION = 'us-gov-east-1' | ||
|
|
||
| # Default role name to assume if one is not provided | ||
| ASSUME_ROLE_NAME = 'OrganizationAccountAccessRole' | ||
| ``` | ||
|
|
||
| ## Makefile Usage | ||
|
|
||
| The project includes a Makefile to simplify common operations: | ||
|
|
||
| ``` | ||
| # Install dependencies | ||
| make install | ||
| # Run tests | ||
| make test | ||
| # Run linting | ||
| make lint | ||
| # Format code | ||
| make format | ||
| # Clean up temporary files | ||
| make clean | ||
| # Build distribution package | ||
| make dist | ||
| # Deploy to development environment | ||
| make deploy-dev | ||
| # Deploy to production environment | ||
| make deploy-prod | ||
| ``` | ||
|
|
||
| The Makefile helps standardize development workflows and ensures consistent environment setup across different machines. To view all available commands: | ||
|
|
||
| ``` | ||
| make help | ||
| ``` | ||
|
|
||
| ## Tips for GovCloud Usage | ||
|
|
||
| When working with GovCloud: | ||
|
|
||
| 1. Ensure your AWS SSO is properly configured | ||
| 2. AWS SSO profiles should include the GovCloud regions in their configuration | ||
| 3. The library will automatically detect GovCloud regions and use the correct partition ('aws-us-gov') | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| - **Authentication failures**: Check that your SSO session is active (`aws sso login --profile your-profile`) | ||
| - **Profile not found**: Verify your `~/.aws/config` file contains the correct account IDs | ||
| - **Permission issues**: Ensure the roles in your SSO profiles have the necessary permissions | ||
| ### CLI Commands | ||
| The tool exposes a CLI through `aws-resource-mgmt` with the following options: | ||
|
|
||
| - `--start`/`--stop`: Specify the action to perform. | ||
| - `--resource-type`: Select resource type (ec2, rds, eks, emr, ecr, all). | ||
| - `--region`/`--exclude-region`: Specify regions to include or exclude. | ||
| - `--account`/`--exclude-account`: Specify accounts to include or exclude. | ||
| - `--dry-run`: Simulate actions without making changes. | ||
|
|
||
| #### Examples | ||
| - Run in dry-run mode: | ||
| ```bash | ||
| python -m aws_resource_management.cli --stop --dry-run --resource-type ec2 | ||
| ``` | ||
|
|
||
| - Stop all resources: | ||
| ```bash | ||
| python -m aws_resource_management.cli --stop --resource-type all | ||
| ``` | ||
|
|
||
| - Start EC2 instances only: | ||
| ```bash | ||
| python -m aws_resource_management.cli --start --resource-type ec2 | ||
| ``` | ||
|
|
||
| ## Development | ||
|
|
||
| ### Code Quality | ||
| - Run linters: | ||
| ```bash | ||
| make lint | ||
| ``` | ||
| - Format code: | ||
| ```bash | ||
| make format | ||
| ``` | ||
|
|
||
| ### Testing | ||
| - Run tests: | ||
| ```bash | ||
| make test | ||
| ``` | ||
|
|
||
| ### Build Distribution | ||
| - Build the package: | ||
| ```bash | ||
| make dist | ||
| ``` | ||
|
|
||
| ## Contributing | ||
| Contributions are welcome! Please see the `CONTRIBUTING.md` file for guidelines. | ||
|
|
||
| ## License | ||
| This project is licensed under the MIT License. See the LICENSE file for details. |
Oops, something went wrong.