Skip to content

Commit

Permalink
Add Terragrunt configurations for various EKS components
Browse files Browse the repository at this point in the history 
- Created Terragrunt configurations for Grafana, Istio, K8s Dashboard, Karpenter, Keycloak, Kiali, Loki, Metrics Server, OpenTelemetry, Prometheus, and Tempo.
- Each configuration includes dependencies on EKS and other relevant modules, with mock outputs for testing.
- Updated inputs to include AWS configuration, cluster details, and specific component configurations.
- Added README documentation for the Prometheus module detailing its components, dependencies, inputs, and outputs.
- Set up retry lock arguments for Terraform commands to enhance stability during deployments.
  • Loading branch information
Your Name committed Jun 26, 2025
1 parent 5f40497 commit cc49e86
Show file tree
Hide file tree
Showing 82 changed files with 806 additions and 3,056 deletions.
110 changes: 110 additions & 0 deletions .github/initialize.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
name: Initialize Repository

on:
# Run on pull requests that involve the repo-init branch
pull_request:
types: [opened, synchronize, reopened]
branches:
- main
- master

# Keep the manual trigger option as well for flexibility
workflow_dispatch:
inputs:
config_path:
description: 'Path to config.json file'
required: false
default: 'config.json'
type: string

jobs:
initialize:
name: Initialize Repository from Template
# Only run if the head branch is repo-init
if: github.head_ref == 'repo-init' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest

steps:
- name: Checkout Repository
uses: actions/checkout@v3
with:
fetch-depth: 0 # Fetch all history and tags
ref: ${{ github.head_ref || 'repo-init' }} # Explicitly checkout repo-init branch

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.10'

- name: Setup locale environment
run: |
echo "Setting up locale environment..."
sudo apt-get update
sudo apt-get install -y locales
sudo locale-gen en_US.UTF-8
echo "LC_ALL=en_US.UTF-8" >> $GITHUB_ENV
echo "LANG=en_US.UTF-8" >> $GITHUB_ENV
echo "LANGUAGE=en_US.UTF-8" >> $GITHUB_ENV
- name: Configure pip
uses: CSVD/pip-config@main

- name: Install Ansible
run: |
python -m pip install --upgrade pip
pip install ansible
- name: Install dependencies
run: |
if [ -f requirements.txt ]; then
pip install -r requirements.txt
fi
- name: Determine config path
id: config
run: |
if [ "${{ github.event_name }}" == "workflow_dispatch" ]; then
CONFIG_PATH="${{ github.event.inputs.config_path }}"
else
CONFIG_PATH="config.json"
fi
# Create absolute path to config file if needed
if [[ "$CONFIG_PATH" != /* ]]; then
CONFIG_PATH="${{ github.workspace }}/${CONFIG_PATH}"
fi
echo "CONFIG_PATH=${CONFIG_PATH}" >> $GITHUB_ENV
echo "Using config file: ${CONFIG_PATH}"
- name: Verify config.json exists
run: |
if [ ! -f "${{ env.CONFIG_PATH }}" ]; then
echo "Error: Config file '${{ env.CONFIG_PATH }}' not found!"
exit 1
fi
cat "${{ env.CONFIG_PATH }}"
- name: Run Ansible Playbook
env:
LC_ALL: en_US.UTF-8
LANG: en_US.UTF-8
LANGUAGE: en_US.UTF-8
run: |
ansible-playbook ansible/generate_hcl_files.yml -e "config_file=${{ env.CONFIG_PATH }}"
- name: Commit Changes
run: |
git config --local user.email "action@github.com"
git config --local user.name "GitHub Action"
git add -A
# Only commit if there are changes
if git diff --staged --quiet; then
echo "No changes to commit"
else
git commit -m "Initialize repository structure from template"
# Explicitly push to repo-init branch
git push origin HEAD:repo-init
fi
207 changes: 207 additions & 0 deletions ansible/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,207 @@
# HCL Generator for EKS Cluster

This tool provides a flexible way to generate Terragrunt HCL files from templates for EKS cluster deployments. It analyzes your existing HCL files and allows you to customize your deployment through a single JSON configuration file.

## Features

- Generates all required HCL files from templates
- Configures environment, region, and cluster names through a single JSON file
- Supports enabling/disabling specific EKS modules
- Provides sensible defaults while allowing customization
- Integrates with GitHub Actions for CI/CD support
- Supports renaming development, region, and cluster directories

## Prerequisites

- Ansible 2.9+
- Python 3.8+
- (Optional) GitHub Actions for CI/CD

## Getting Started

### Local Usage

1. Clone this repository
2. Create your configuration file (or use the example file as a template)
3. Run the Ansible playbook:

```bash
ansible-playbook ansible/generate_hcl_files.yml -e "config_file=path/to/your/config.json"
```

### GitHub Actions Usage

The repository includes GitHub Actions workflows to automate the generation of HCL files:

1. Create your configuration file and commit it to the repository
2. Go to the "Actions" tab in your repository
3. Select the "Generate EKS Cluster HCL Files" workflow
4. Click "Run workflow" and provide the required parameters:
- Config file path
- Output directory
- Environment
- Whether to commit and push changes

## Configuration Options

The JSON configuration file provides a flexible way to customize your EKS cluster deployment. Here's a sample configuration:

```json
{
"environment": "production",
"region": "us-gov-west-1",
"cluster_dir": "platform-cluster",
"enable_all_modules": true,

"account": {
"account_name": "prod-ew",
"aws_account_id": "123456789012",
"aws_profile": "123456789012-prod-gov",
"environment_abbr": "prod"
},

"vpc": {
"vpc_name": "vpc-prod",
"vpc_domain_name": "prod.csp2.census.gov"
},

"cluster": {
"cluster_name": "prod-eks-platform",
"cluster_mailing_list": "platform-team@census.gov",
"eks_instance_disk_size": 200,
"eks_ng_desired_size": 3,
"eks_ng_max_size": 10,
"eks_ng_min_size": 3,
"enable_cluster_creator_admin_permissions": true,
"tags": {
"slim:schedule": "always-on",
"environment": "production",
"owner": "platform-team"
}
},

"modules": {
"gogatekeeper": true,
"cert_manager": true,
"prometheus": true,
"grafana": true,
"istio": true
}
}
```

### Configuration Fields

| Field | Description | Default |
|-------|-------------|---------|
| `environment` | Environment name (e.g., development, production) | development |
| `region` | AWS region for deployment | us-gov-east-1 |
| `cluster_dir` | Name of the cluster directory | cluster |
| `enable_all_modules` | Whether to enable all modules | false |
| `account.*` | Account-specific configuration | See below |
| `vpc.*` | VPC-specific configuration | See below |
| `cluster.*` | Cluster-specific configuration | See below |
| `modules.*` | Module-specific enablement flags | false |

#### Account Configuration

| Field | Description | Default |
|-------|-------------|---------|
| `account_name` | Name of the AWS account | lab-dev-ew |
| `aws_account_id` | AWS account ID | 224384469011 |
| `aws_profile` | AWS profile to use | 224384469011-lab-dev-gov |
| `environment_abbr` | Environment abbreviation | dev |

#### VPC Configuration

| Field | Description | Default |
|-------|-------------|---------|
| `vpc_name` | Name of the VPC | vpc3-lab-dev |
| `vpc_domain_name` | Domain name for the VPC | dev.lab.csp2.census.gov |

#### Cluster Configuration

| Field | Description | Default |
|-------|-------------|---------|
| `cluster_name` | Name of the EKS cluster | platform-eng-eks-mcm |
| `cluster_mailing_list` | Email for cluster notifications | matthew.c.morgan@census.gov |
| `eks_instance_disk_size` | Disk size for EKS instances | 100 |
| `eks_ng_desired_size` | Desired size of node group | 2 |
| `eks_ng_max_size` | Maximum size of node group | 10 |
| `eks_ng_min_size` | Minimum size of node group | 2 |
| `enable_cluster_creator_admin_permissions` | Whether to enable admin permissions | true |
| `tags` | Tags to apply to resources | See example |

## Generated Directory Structure

The tool generates the following directory structure:

```
<environment>/
├── environment.hcl
└── <region>/
├── region.hcl
└── vpc/
├── vpc.hcl
└── <cluster_dir>/
├── cluster.hcl
├── eks/
│ └── terragrunt.hcl
├── eks-cert-manager/
│ └── terragrunt.hcl
└── ... (other modules)
```

## Using the Generated Files

After generating the HCL files:

1. Navigate to the generated directory structure
2. Run Terragrunt commands to plan and apply:

```bash
cd <environment>/<region>/vpc/<cluster_dir>/eks
terragrunt init
terragrunt plan
terragrunt apply
```

## Advanced Usage

### GitHub Actions CI/CD Pipeline

The repository includes a reusable workflow for generating HCL files via GitHub Actions. You can customize this workflow by editing the `.github/workflows/generate-eks-cluster.yml` file.

Example using the workflow from another repository:

```yaml
jobs:
generate-hcl:
uses: org/template-eks-cluster/.github/workflows/generate-hcl-files.yml@main
with:
config_file: 'config/production.json'
output_directory: 'infrastructure/eks'
push_changes: true
secrets:
ssh_key: ${{ secrets.DEPLOY_KEY }}
```
### Customizing Templates
You can customize the Jinja2 templates in the `ansible/templates` directory to meet your specific needs.

## Troubleshooting

### Common Issues

1. **Missing Required Variables**: Ensure your JSON configuration file includes all required variables.
2. **File Permissions**: Make sure Ansible has permission to read the configuration file and write to the output directory.
3. **GitHub Actions Secrets**: For CI/CD, ensure the `REPO_SSH_KEY` secret is properly configured if you're using the `push_changes` option.

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

## License

See the LICENSE file for details.

0 comments on commit cc49e86

Please sign in to comment.