Implement branch creation and pull request automation in GitHubClient…

…; add detailed implementation plan to README

design-docs/README.md

@@ -0,0 +1,83 @@
+# EKS Automation Lambda Implementation Plan
+
+## Project Map
+
+### template-eks-cluster
+This is the template repo that is generated from the lambda function, we're designing this whole system to be agnostic towards which repos it's creating; this will be the first example of this style of condiguration (pattern).
+
+### terraform-aws-template-automation
+This is the terraform module that is repsonsible for deploying our Lambda function along with any peripheral infrastructure that may be required, such as API Gateway. If SSM parameters or Secrets are required in the lambda function, they get deployed through this module.
+
+### template-automation-lambda
+This is actual Lambda function, this repo creates a Docker image that we deploy to Lambda. Our actual lambda code is in template_automation/app.py.
+
+## Overview
+This document outlines the implementation plan for enhancing the EKS Automation Lambda to improve its GitHub integration workflow and testing capabilities. Most of this work will take place in template_automation/app.py
+
+## Implementation Phases
+
+### Phase 1: Lambda Function Updates
+Updates to the Lambda function to improve repository management:
+
+- **Branch Management**
+  - Create new "init-cluster" branch instead of pushing directly to main
+  - Implement branch creation in GitHub client (template_automation/app.py)
+  - Add error handling for branch operations
+
+- **Pull Request Automation**
+  - Add automatic PR creation after pushing changes
+  - Include standard PR template and description
+  - Implement PR creation in GitHub client
+
+### Phase 2: GitHub Actions Workflow Updates
+Clean out current github actions in template repo (template-eks-cluster).
+Enhance the GitHub Actions workflow configuration:
+
+- **HCL File Generation**
+  - Add action to expand config.js into HCL files, this will be done through ansible. Review the generate_hcl_files.yml playbook.
+  - Implement Terraform plan action post-HCL generation
+  - Add validation steps for generated HCL
+
+- **Runner Configuration**
+  - Templateize GitHub Actions workflow files
+  - Configure runners based on AWS account IDs
+  - Add support for lab environment runners
+  - Implement account-specific runner selection
+
+### Phase 3: Testing Implementation
+Comprehensive testing setup:
+
+- **Lab Environment**
+  - Configure workflow for lab AWS account
+  - Set up isolated testing environment
+  - Create test cluster configurations
+
+- **End-to-End Testing**
+  - Implement full workflow testing
+  - Create demonstration environment
+  - Add integration tests for GitHub operations
+  - Implement validation checks
+
+### Phase 4: Manual Trigger Interface
+Short-term manual operation support:
+
+- **Documentation**
+  - Lambda invocation process
+  - Example payload templates
+  - Verification steps and checks
+
+- **Future Considerations**
+  - CRF integration planning
+  - Automation transition strategy
+
+## Success Criteria
+- Lambda successfully creates branches and PRs
+- GitHub Actions properly expand config and run Terraform plans
+- Workflows correctly target different AWS accounts
+- End-to-end testing works in lab environment
+- Clear documentation exists for manual processes
+
+## Dependencies
+- GitHub API access and permissions
+- AWS account access for testing
+- Runner configurations for different environments

template_automation/app.py

@@ -686,6 +686,91 @@ def set_team_permission(self, repo_name, team_name, permission):
             logger.error(error_message)
             raise Exception(error_message)
 
+    def create_branch(self, repo_name, branch_name, from_ref="main"):
+        """Create a new branch in the repository
+        
+        Args:
+            repo_name (str): Name of the repository
+            branch_name (str): Name of the branch to create
+            from_ref (str): Reference to create branch from (default: main)
+            
+        Returns:
+            dict: Branch creation response from GitHub API
+        """
+        try:
+            # Get the SHA of the reference we're branching from
+            base_sha = self.get_reference_sha(repo_name, f"heads/{from_ref}")
+
+            # Create the new branch reference
+            create_ref_url = f"{self.api_base_url}/repos/{self.org_name}/{repo_name}/git/refs"
+            ref_data = {
+                "ref": f"refs/heads/{branch_name}",
+                "sha": base_sha
+            }
+
+            response = requests.post(
+                create_ref_url,
+                headers=self.headers,
+                json=ref_data,
+                verify=False
+            )
+
+            if response.status_code == 201:
+                logger.info(f"Created branch {branch_name} in {repo_name}")
+                return response.json()
+            else:
+                error_message = f"Failed to create branch: {response.status_code} - {response.text}"
+                logger.error(error_message)
+                raise Exception(error_message)
+
+        except Exception as e:
+            error_message = f"Error creating branch: {str(e)}"
+            logger.error(error_message)
+            raise Exception(error_message)
+
+    def create_pull_request(self, repo_name, title, head_branch, base_branch="main", body=None):
+        """Create a pull request
+        
+        Args:
+            repo_name (str): Name of the repository
+            title (str): Title of the pull request
+            head_branch (str): Name of the branch containing changes
+            base_branch (str): Name of the branch to merge into
+            body (str, optional): Description of the pull request
+            
+        Returns:
+            dict: Pull request creation response from GitHub API
+        """
+        try:
+            create_pr_url = f"{self.api_base_url}/repos/{self.org_name}/{repo_name}/pulls"
+
+            pr_data = {
+                "title": title,
+                "head": head_branch,
+                "base": base_branch,
+                "body": body or "Created by Template Automation"
+            }
+
+            response = requests.post(
+                create_pr_url,
+                headers=self.headers,
+                json=pr_data,
+                verify=False
+            )
+
+            if response.status_code == 201:
+                logger.info(f"Created pull request in {repo_name}: {title}")
+                return response.json()
+            else:
+                error_message = f"Failed to create pull request: {response.status_code} - {response.text}"
+                logger.error(error_message)
+                raise Exception(error_message)
+
+        except Exception as e:
+            error_message = f"Error creating pull request: {str(e)}"
+            logger.error(error_message)
+            raise Exception(error_message)
+
 def generate_repository_name(project_name):
     """Generate repository name based on prefix or project name
     
@@ -833,16 +918,27 @@ def operate_github(new_repo_name, template_settings, trigger_init_workflow=False
         with open(version_file_path, "w") as file:
             file.write(source_version)
 
-    # Commit all files to the new repository's main branch explicitly
+    # Create init-cluster branch and commit changes there
+    branch_name = "init-cluster"
     commit_message = "Add template configuration by automation"
-    github.commit_repository_contents(actual_repo_name, work_dir, commit_message, branch="main")
+
+    # Create the init-cluster branch from main
+    github.create_branch(actual_repo_name, branch_name, from_ref="main")
+
+    # Commit all files to the init-cluster branch
+    github.commit_repository_contents(actual_repo_name, work_dir, commit_message, branch=branch_name)
+
+    # Create pull request to merge init-cluster into main
+    pr_title = "Initial cluster configuration"
+    pr_body = f"Automated pull request for initializing cluster configuration.\n\nProject: {actual_repo_name}"
+    github.create_pull_request(actual_repo_name, pr_title, branch_name, base_branch="main", body=pr_body)
 
     # Add configurable topics to the repository
     topics = DEFAULT_TOPICS
 
     github.update_repository_topics(actual_repo_name, topics)
 
-    logger.info(f"Successfully updated {actual_repo_name} repository")
+    logger.info(f"Successfully updated {actual_repo_name} repository and created PR from {branch_name} to main")
 
     # Trigger init workflow if requested
     if trigger_init_workflow:
@@ -860,7 +956,7 @@ def github_token():
     """
     secrets = boto3.client("secretsmanager")
     try:
-        secret = secrets.get_secret_value(SecretId=SECRET_NAME)
+        secret = secrets.get_secret_value(SecretId=GITHUB_TOKEN_SECRET_NAME)
         return secret["SecretString"]
     except ClientError as e:
         logger.error(f"Error occurred when retrieving GitHub token from Secrets Manager: {str(e)}")