diff --git a/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/README.md b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/README.md new file mode 100644 index 00000000..efc12a4a --- /dev/null +++ b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/README.md @@ -0,0 +1,9 @@ +# + +# Conversion + +```script +pandoc --extract-media=images/dns GCP\ Cloud\ DNS\ Service\ Architecture.docx -o dns.md +pandoc --extract-media=images/networking GCP\ Networking\ Solution\ Architecture.docx -o networking.md +pandoc --extract-media=images/private-service GCP\ Private\ Service\ Connect\ Service\ Design.docx -o private-service.md +``` diff --git a/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/dns.md b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/dns.md new file mode 100644 index 00000000..80688c0b --- /dev/null +++ b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/dns.md @@ -0,0 +1,439 @@ +**Department of Commerce (DOC)** + +**United States Census Bureau (USCB)** + +**Google Cloud Platform (GCP)** + +**Service Architecture** + +**\ +GCP Cloud DNS** + +![](images/dns/media/image1.png){width="6.5in" +height="1.6979166666666667in"} + +1 Revision History [1](#revision-history) + +2 Overview [2](#overview) + +2.1 Background [2](#background) + +2.2 Scope [2](#scope) + +2.3 Governance [2](#governance) + +2.4 Terms and Definitions [2](#terms-and-definitions) + +3 Service Design [3](#service-design) + +3.1 Value [3](#value) + +3.2 Capabilities, Features, and Requirements +[3](#capabilities-features-and-requirements) + +3.2.1 Capabilities, Features [3](#capabilities-features) + +3.2.2 Requirements [3](#requirements) + +3.3 Assumptions [3](#assumptions) + +3.4 Constraints [3](#constraints) + +3.5 Hybrid DNS [4](#hybrid-dns) + +3.6 Interfaces [5](#interfaces) + +3.7 Consumption [5](#consumption) + +3.8 Configuration Management [5](#configuration-management) + +3.9 Service Level Agreements [5](#service-level-agreements) + +3.10 Patching and Updates [5](#patching-and-updates) + +3.11 Roles and Responsibilities [5](#roles-and-responsibilities) + +3.12 Service Limits & Capacity Planning +[5](#service-limits-capacity-planning) + +3.13 Consumption, Licensing, and Assets +[5](#consumption-licensing-and-assets) + +4 Cost Consideration [5](#cost-consideration) + +5 Backup and Recovery [6](#backup-and-recovery) + +6 Security [6](#security) + +6.1 Authentication, Access, Authorization +[6](#authentication-access-authorization) + +6.2 Auditing [6](#auditing) + +6.3 Logging [6](#logging) + +7 References [6](#references) + +7.1 Tagging [6](#tagging) + +7.2 Infrastructure as Code [6](#infrastructure-as-code) + +7.3 Links [6](#links) + +List of Figures + +[Figure 1 DNS logical design [4](#_Toc157520308)](#_Toc157520308) + +List of Table + +[Table 1 Revision History [1](#_Toc157520309)](#_Toc157520309) + +[Table 2 Query Pricing [5](#_Toc157520310)](#_Toc157520310) + +[Table 3 Managed Zone Pricing [6](#_Toc157520311)](#_Toc157520311) + +# Revision History + +[]{#_Toc157520309 .anchor}Table 1 Revision History + + ----------------------------------------------------------------------------- + Version Date Author Description + --------- ------------ ------------ ----------------------------------------- + 0.01 8/3/2023 Ethan Rowe Initial template draft + + 0.02 10/11/2023 Michael Draft content + Willetts + + 0.03 12/13/2023 Ethan Rowe Updated DNS Suffix to csp3 and best + practices link. + + 1.00 1/30/2024 Ethan Rowe Baseline inf/core #426 + + + ----------------------------------------------------------------------------- + +# Overview + +## Background + +Google Cloud Platform (GCP) provides Domain Name Services (DNS) using +GCP Cloud DNS. + +## Scope + +This document details the design of GCP Cloud DNS. It describes the +design requirements and decisions necessary to achieve integration with +USCB Enterprise Domain Name Services. + +## Governance + +This document, its associated service(s), and guidelines regarding the +use of this service are governed by the Enterprise Cloud Governance +Board (CGB). Any proposed modifications to the GCP Cloud DNS service and +configurations must be presented by the Product Owner and/or Technical +Lead and approved by the CGB. + +## Terms and Definitions + +**Project:** A Google Cloud console project is a container for +resources, a domain for access control, and a place where billing is +configured and aggregated. + +**Managed Zone:** A Managed Zone in GCP holds the DNS records for the +same DNA name suffix (i.e., .csp3.census.gov). A project may have +multiple managed zones, but the names must be unique. The Managed Zone +models a standard DNS zone. All records in a Managed Zone are hosted on +the same Google-operated name servers. + +**Record:** An entry in the hosted zone which determines how the +internet traffic is routed for a domain name so that it reaches the +resources. + +**Public Zone:** A Managed Zone that is visible to the Internet. USCB +does not utilize GCP hosted Public Zones at this time. + +**Private Zone:** A Managed Zone that enables USCB to host custom +private domain names for VM instances, load balancers and other +resources with them being published to the Internet. Private Zones in +GCP do not support DNSSEC or custom resource record sets. + +**Forwarding Zone:** A forwarding zone is a type of Cloud DNS managed +private zone that forwards requests for that zone to the IP addresses of +its forwarding targets. Records may not be added into a Forwarding Zone +directly, instead they must be entered into the DNS system referenced. + +**Peering Zone:** A peering zone is a type of Cloud DNS managed private +zone that follows the name resolution order of another VPC network. You +can use it to resolve the names that are defined in the other VPC +network. Records may not be added into a Peering Zone directly, instead +they must be entered into the DNS system referenced. + +**DNS Server Policy:** A DNS server policy lets you access name +resolution services provided by Google Cloud in a VPC network with +inbound forwarding or supersede the VPC name resolution order with an +outbound server policy. + +**DNS query:** It is a request for information sent from DNS client to +the DNS server. + +# Service Design + +## Value + +GCP's Cloud DNS provides highly available DNS services. Cloud DNS is a +managed service and simplifies the cost and complexity of managing an +equivalent server footprint. Cloud DNS permits name registration and +resolution of census.gov assets from within GCP. + +## Capabilities, Features, and Requirements + +### Capabilities, Features + +Cloud DNS is a highly available and scalable Domain Name System (DNS) +web service. You can use Cloud DNS to perform three main functions in +any combination: domain registration, DNS resolution & routing, and +health checking. + +- **Highly Reliable:** Cloud DNS is provided as a managed service from + GCP. There is no management of servers or activities other than + configuration necessary to utilize Cloud DNS. + +- **Scalable:** Cloud DNS automatically scales to handle large traffic + spikes and can be configured to handle DNS for GCP assets, as well + as on-prem and even public if need. + +- **Secure:** Cloud DNS is integrated with IAM, the access to Cloud + DNS is secured by giving its permissions to only the authorized + users. + +- **Integrated:** Cloud DNS can be used to map domain names to GCP + resources including Cloud Storage Buckets, load balancers, and + virtual machines. + +```{=html} + +``` +- **Hybrid DNS:** Supports private DNS that spans on-prem and other + cloud environments. + +- **Traffic Routing:** According to the geolocation, latency, health, + and other factors, directs traffic to the optimal endpoint + available. + +### Requirements + +- All Cloud DNS Zones forwards requests to Census Enterprise On-Prem + Infoblox for name resolution of resources in the applicable domain. + +- VPC Cloud DNS resolves the local VPC domain and the Shared Service + VPC Prod domain. + +- Cloud DNS resolves DNS queries of GCP domains from Census on-prem + and AWS networks. + +- Route tables must advertise the Cloud DNS source range through the + VPN link for hybrid DNS to function correctly. + +- Cloud DNS endpoints must register in the Census Enterprise Infoblox. + +## Assumptions + +- DNS domain for GCP is approved. + +- DNS subdomain of GCP is approved for each VPC. + +## Constraints + +- Route tables must advertise the Cloud DNS source range through the + VPN link for hybrid DNS to function correctly. + +- Full DNS resolution among all Cloud DNS managed domains require a + full mesh implementation of peering zones. A design decision was + made to only peer with VPC's to provide routing from an originating + VPC. This design was reviewed by Google. + +## Hybrid DNS + +![Diagram Description automatically +generated](images/dns/media/image2.png){width="6.5in" +height="3.077777777777778in"} + +[]{#_Toc157520308 .anchor}Figure DNS logical design + +The GCP Design leverages a custom domain name, \*.csp3.census.gov, with +subdomains managed for each VPC. The GCP Cloud DNS design centralizes +the inbound and outbound endpoints in the central Network services +account. For inbound resolution the design centralizes the inbound to +the Prod Hub account. This approach is necessary as Cloud DNS requests +are sourced from 35.199.192.0/19[^1] and require that route to be +advertised back through the VPN link. Advertising this route from other +Hub accounts causes routing issues and impede DNS resolution. To resolve +domain names, a GCP Cloud DNS inbound server policy is configured for +DNS queries that are forwarded to the GCP VPCs. This allows resolution +within Cloud DNS of these requests in the VPC resolution order. All +Cloud DNS zones are peered with the Prod Hub to enable full lookup of +any GCP resource from an on-prem source. Forwarding zones are created in +each VPC to handle outbound queries to on-prem. + +The following points cover the traffic flow for how DNS resolution is +achieved for inbound and outbound queries to on-prem/ Infoblox. + +- **Outbound DNS Resolution** --- GCP Cloud DNS utilizes forwarding + zones registered in each VPC to register the DNS servers used for + external resolution. + +```{=html} + +``` +- **Inbound DNS resolution** -- An inbound server policy created in + the Prod Hub VPC to receive inbound DNS resolution requests. Each + GCP VPC Cloud DNS zone has a peering zone created in the Prod Hub + account to allow for resolution of any GCP resource from external + requests. + +- **GCP VPC DNS resolution** -- Services utilize the standard GCP + metadata DNS resolution order to query Cloud DNS. Forwarding Zones + are created to direct queries for zones external to Cloud DNS. + +## Interfaces + +GCP Cloud DNS interfaces with DNS resolvers through standard DNS +protocols over port 53. Cloud DNS provides a graphical user interface +through the GCP console that can be leveraged for management. Cloud DNS +supports API/CLI access. + +## Consumption + +Provisioning of names and policies within Cloud DNS are performed with +Terraform, managed in GitLab, and may be requested from the Remedy SRM +Cloud Request. + +## Configuration Management + +Configuration of Cloud DNS is managed and maintained using Terraform, +managed in GitLab. It may be called as part of an automated or +orchestrated workflow in a CI/CD pipeline. As tickets or provisioning +requests are approved, updates are performed using Terraform, following +the GCP iSDLC. One all validation is performed; the state is updated, +and a known-good configuration is available for redeployment. + +## Service Level Agreements + +GCP Cloud DNS as a managed service provides a Monthly Uptime Percentage +of serving DNS queries from at least one of the Google managed +Authoritative Name Servers to the customer at 100%. If the SLO is not +met and the customer has met other SLA provisions, they may request +financial credits. + +## Patching and Updates + +GCP Cloud DNS is a managed service provided by Google. All patching and +updates are handled by the provider and occur without disruption to the +customer. + +## *Roles and Responsibilities* + +- GCP (Cloud Service Provider) -- Maintain, patch, and update Cloud + DNS. + +- Cloud Engineering Team -- Develop and maintain the DNS Design and + IaC to leverage Cloud DNS. Utilize Infrastructure as Code to + provision DNS records for GCP resources. + +## Service Limits & Capacity Planning + +Cloud DNS does have service quotas and limits. Quotas may be adjusted. +Refer to the GCP quota links in the appendix. Capacity planning is a +monthly activity, detailed in the GCP Operations Plan. + +## Consumption, Licensing, and Assets + +GCP Cloud DNS is provided as a managed platform from GCP. Licensing +agreements are covered by agreements with GCP overall. When provisioning +resources from Cloud DNS all Enterprise GCP naming and tagging standards +must be followed to ensure proper cost accounting is maintained. + +# Cost Consideration + +GCP Cloud DNS pricing is a low-cost service. Pricing centers around the +number of queries and the number of managed zones maintained. Pricing +should always be checked against GCP documentation for the latest. As of +this writing, pricing and examples are provided in the tables below. + +[]{#_Toc157520310 .anchor}Table Query Pricing + + ----------------------------------------------------------------------- + Number of Queries Regular Queries Routing Policy Queries + ----------------------- ----------------------- ----------------------- + 0-1 Billion \$0.40/million per \$0.70/million per + month month + + Over 1 Billion \$0.20/million per \$0.35/million per + month month + ----------------------------------------------------------------------- + +[]{#_Toc157520311 .anchor}Table Managed Zone Pricing + + ----------------------------------------------------------------------- + Managed Zones Price + ----------------------------------- ----------------------------------- + 0-25 \$0.20/zone per month + + 26-10,000 \$0.10/zone per month + + Over 10,000 \$0.03/zone per month + ----------------------------------------------------------------------- + +# Backup and Recovery + +As a managed service GCP Cloud DNS does not provide a distinct backup. +All configuration of the service should be maintained in a version +controlled IaC repository to allow for rollback operations or complete +rebuilding as required. + +# Security + +## Authentication, Access, Authorization + +Administration of Cloud DNS is performed by t4 administrators utilizing +IaC. + +## Auditing + +See the GCP operations plan for schedule of audits and reporting. + +## Logging + +All logging is directed to Operations Suite Cloud Logging. DNS queries +are logged. Any Cloud DNS API commands are logged. + +# References + +## Tagging + +[Enterprise GCP Naming and Tagging +Standards](https://uscensus.sharepoint.com/:f:/s/DITDSIRS/EpbVeuUQbE1Ftjo-SJpUjj4BG9Nq15mRUQdNsUaQcRyeyw?e=ZMCX3r) + +## Infrastructure as Code + +*Platform:* + +*DNS Modules:* +[gcp/networking/modules/dns](https://gitlab.ditd.census.gov/inf/modules/-/tree/main/gcp/networking/modules/dns?ref_type=heads) + +## Links + +- [Google Cloud DNS SLA](https://cloud.google.com/dns/sla) + +- [Current Cloud DNS Pricing](https://cloud.google.com/dns/pricing) + +- [Cloud DNS Best + Practices](https://cloud.google.com/dns/docs/best-practices#reference_architectures_for_hybrid_dns) + +- [GCP Cloud DNS Quotas and + Limits](https://cloud.google.com/dns/quotas) + +- [GCP Operations + Plan](https://uscensus.sharepoint.com/:w:/s/DITDSIRS/EfHiSwwk1mFCtyuXFnCKGoMBjV158-8VEfxVUCVI37D3lQ?e=qordeA) + +[^1]: GCP Block from Cloud DNS. diff --git a/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/dns/media/image1.png b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/dns/media/image1.png new file mode 100644 index 00000000..a2c5de5e Binary files /dev/null and b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/dns/media/image1.png differ diff --git a/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/dns/media/image2.png b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/dns/media/image2.png new file mode 100644 index 00000000..f558fcd6 Binary files /dev/null and b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/dns/media/image2.png differ diff --git a/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/networking/media/image1.png b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/networking/media/image1.png new file mode 100644 index 00000000..39b9589b Binary files /dev/null and b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/networking/media/image1.png differ diff --git a/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/networking/media/image2.png b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/networking/media/image2.png new file mode 100644 index 00000000..db1b2f38 Binary files /dev/null and b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/networking/media/image2.png differ diff --git a/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/networking/media/image3.png b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/networking/media/image3.png new file mode 100644 index 00000000..eaa421cb Binary files /dev/null and b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/networking/media/image3.png differ diff --git a/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/networking/media/image4.png b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/networking/media/image4.png new file mode 100644 index 00000000..96c0a9b5 Binary files /dev/null and b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/networking/media/image4.png differ diff --git a/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/networking/media/image5.png b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/networking/media/image5.png new file mode 100644 index 00000000..e78299a8 Binary files /dev/null and b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/networking/media/image5.png differ diff --git a/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/networking/media/image6.png b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/networking/media/image6.png new file mode 100644 index 00000000..ed7958d5 Binary files /dev/null and b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/networking/media/image6.png differ diff --git a/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/private-service/media/image1.jpeg b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/private-service/media/image1.jpeg new file mode 100644 index 00000000..543fad64 Binary files /dev/null and b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/private-service/media/image1.jpeg differ diff --git a/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/private-service/media/image2.png b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/private-service/media/image2.png new file mode 100644 index 00000000..fa07e8cd Binary files /dev/null and b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/private-service/media/image2.png differ diff --git a/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/private-service/media/image3.png b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/private-service/media/image3.png new file mode 100644 index 00000000..fe077e1a Binary files /dev/null and b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/private-service/media/image3.png differ diff --git a/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/private-service/media/image7.png b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/private-service/media/image7.png new file mode 100644 index 00000000..e78299a8 Binary files /dev/null and b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/images/private-service/media/image7.png differ diff --git a/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/networking.md b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/networking.md new file mode 100644 index 00000000..80820c83 --- /dev/null +++ b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/networking.md @@ -0,0 +1,884 @@ +**Department of Commerce (DOC)** + +**United States Census Bureau (USCB)** + +**Google Cloud Platform (GCP)** + +**GCP Networking Design** + +![](images/networking/media/image1.png){width="6.5in" +height="1.70625in"} + +1 Revision History [1](#revision-history) + +2 Overview [2](#overview) + +2.1 Background [2](#background) + +2.2 Scope [2](#scope) + +2.3 Audience [2](#audience) + +2.4 Governance [2](#governance) + +2.5 Terms and Definitions [2](#terms-and-definitions) + +3 Service Design [2](#service-design) + +3.1 Value [2](#value) + +3.2 Capabilities, Features, and Requirements +[3](#capabilities-features-and-requirements) + +3.3 Assumptions [3](#assumptions) + +3.4 Constraints [3](#constraints) + +3.5 Conceptual Design [3](#conceptual-design) + +3.6 Logical Design [4](#logical-design) + +3.6.1 GCP Cloud Router/ Cloud HA VPN Design +[5](#gcp-cloud-router-cloud-ha-vpn-design) + +3.6.2 Host Project/ Hub VPC Design [5](#host-project-hub-vpc-design) + +3.6.3 VPC Network Peering Design [5](#vpc-network-peering-design) + +3.6.4 Shared VPC Design [6](#shared-vpc-design) + +3.6.5 VPC and Subnet CIDR Allocation +[7](#vpc-and-subnet-cidr-allocation) + +3.6.6 Private Service Connect Design +[8](#private-service-connect-design) + +3.7 Interfaces [9](#interfaces) + +3.8 Consumption [9](#consumption) + +3.9 Configuration Management [9](#configuration-management) + +3.10 Key Performance Indicators [9](#key-performance-indicators) + +3.11 Service Level Agreements [10](#service-level-agreements) + +3.12 Patching and Updates [10](#patching-and-updates) + +3.13 Roles and Responsibilities [10](#roles-and-responsibilities) + +3.14 Service Limits & Capacity Planning +[10](#service-limits-capacity-planning) + +3.15 Consumption, Licensing, and Assets +[11](#consumption-licensing-and-assets) + +4 Cost Consideration [12](#cost-consideration) + +5 Backup and Recovery [13](#backup-and-recovery) + +6 Security [13](#security) + +6.1 Authentication, Access, Authorization +[13](#authentication-access-authorization) + +6.2 Auditing [13](#auditing) + +6.3 Logging [13](#logging) + +6.4 Alerts [14](#alerts) + +7 References [14](#references) + +7.1 Tagging [14](#tagging) + +7.2 Infrastructure as Code [14](#infrastructure-as-code) + +7.3 Links [14](#links) + +7.4 Shared VPC Best Practices [14](#shared-vpc-best-practices) + +List of Figures + +[Figure 1 GCP Conceptual Networking Design +[4](#_Toc157695668)](#_Toc157695668) + +[Figure 2 Logical GCP Network Design +[4](#_Toc157695669)](#_Toc157695669) + +[Figure 3 Shared VPC Design [7](#_Ref153543324)](#_Ref153543324) + +[Figure 4 Example VPC Private Service Connection to GCP services +[9](#_Toc157695671)](#_Toc157695671) + +List of Table + +[Table 1 Revision History [1](#_Toc157695672)](#_Toc157695672) + +[Table 3 VPC Subnets [8](#_Toc157695673)](#_Toc157695673) + +[Table 4 GCP VPC Service Limits [10](#_Toc157695674)](#_Toc157695674) + +[Table 5 Cloud VPN Limits [11](#_Toc157695675)](#_Toc157695675) + +[Table 6 Virtual Private Cloud Pricing +[12](#_Ref153791171)](#_Ref153791171) + +[Table 7 Cloud VPN [12](#_Ref153791180)](#_Ref153791180) + +[Table 8 Private Service Connect [13](#_Ref153791215)](#_Ref153791215) + +# Revision History + +[]{#_Toc157695672 .anchor}Table 1 Revision History + + -------------------------------------------------------------------------------- + **Version** **Date** **Description** + ------------- ------------ ----------------------------------------------------- + 0.01 8/3/2023 Initial template draft + + 0.02 9/27/2023 Draft content for Network Design + + 0.03 10/3/2023 Broad content update for Review and Comment + + 0.04 10/11/2023 Peer reviewed. Removed references to AWS. Content + rewrite to focus on design decisions. Comments added. + + 0.05 12/15/2023 Grammar Updates + + 1.0 12/18/2023 Updates from CSvD review + + 1.01 1/5/2024 Update Logical Design drawing and CIDR Table -- + Michael Jones + + 1.02 1/31/2024 Baseline core/inf #426 + + 1.03 3/28/2024 Update shared vpc subnet layout + -------------------------------------------------------------------------------- + +# Overview + +## Background + +The GCP Network design details the design decisions to provide network +connectivity among Google Cloud Platform (GCP) cloud resources and USCB +Enterprise networks. The core components of GCP network design include +Virtual Private Cloud (VPC), Hub and Shared VPCs, and Cloud High +Availability (HA) Virtual Private Networks (VPN). Cloud DNS is detailed +in a separate design document. + +## Scope + +This document describes the Google Cloud Platform (GCP) networking +design in support of the Enterprise Cloud and the Cloud Concept of +Operations (ConOpS) for the United States Census Bureau (USCB). + +## Audience + +The target audience for this document are cloud, networking, and +security leaders, architects, engineers, and operators. + +## Governance + +This document, its associated service(s), and guidelines regarding the +use of this service are governed by the Enterprise Cloud Governance +Board (CGB) Changes to this service require must be presented by the +Product Owner and/or Technical Lead and approved by the CGB. + +## Terms and Definitions + +**Host Project** --- Equivalent of an Account in Amazon Web Services +(AWS) where Shared VPCs, VPN connections, and core networking resources +are defined and provisioned. + +**Service Project** --- A Service project resides outside the host +project but subscribes to the host project's network services (i.e., +Shared VPC). + +**Shared VPC** --- A Shared VPC allows an organization to connect +resources from multiple projects to a common Virtual Private Cloud (VPC) +network to communicate with each other securely and efficiently by using +internal IP addresses from that network. + +# Service Design + +## Value + +The USCB GCP Network Design provides reliable, scalable, and secure +communication services to GCP hosted resources. + +- **Highly Secure Connectivity --** Traditional networking solutions + require complex configurations to establish security and access + controls. To meet privacy and security requirements, GCP VPCs are + configured with advanced security features including Network + Firewalls and hierarchical firewall policies for performing inbound + and outbound filtering. GCP Private Service Connect is used to + establish secure network connections between VPCs and GCP services + by delivering traffic on the GCP backbone network. + +- **Scalable Networking --** GCP Hub VPC is utilized for centralized + management, monitoring, and routing of all bi-directional traffic + for each VPC and VPN. GCP Hub VPCs connect VPCs and USCB on-premises + networks networking by peering connections to a Shared VPC and VPN + connections for hybrid connectivity. + +- **Monitored --** VPC and Host VPC traffic flows are monitored for + security and network anomalies and assist with troubleshooting using + a Palo Alto IDS managed by OIS. + +## Capabilities, Features, and Requirements + +The USCB GCP Network design provides connectivity to resources in the +GCP cloud and to the Census on-prem network. This design utilizes +several GCP Network services including VPC, Cloud Router, Cloud HA VPN, +Cloud DNS, and Private Service Connect. The requirements for this design +include: + +- Network Connectivity in the cloud among applications, systems, and + services. + +- Network Connectivity to the Census on-prem environment. + +- Provide network segmentation among environments. + +- Secure, encrypted connections over any public interfaces or + circuits. + +- Ability to restrict and permit cloud communication with TCP/IP based + firewalls and access control lists. + +- To the greatest extent possible, perform cost optimization and + complexity reduction by leveraging the capability to share cloud + resources. + +- High availability, fault tolerance, and resilience through redundant + networking. + +- Integration with GCP IAM to control privileged access to cloud + networking resources and to support security compliance, auditing, + and logging. + +## Assumptions + +- Connectivity to the Census on-prem environment is required. + +- Routing to AWS networks is outside the scope at the time of this + writing. + +```{=html} + +``` +- IPv6 integration is out of scope. IPv6 addressing is allocated and + reserved for future integration when required by TCO. + +## Constraints + +- Access to GCP Networking resources is provided by GCP Cloud + Identity. Cloud Identity provides support for permission sets, + users, roles, groups, and policies required to manage and administer + GCP Networking services. + +- Due to the VRF design of the on-premises networking, it does not + permit the use of a centralized hub for reducing cost and complexity + of GCP to Enterprise connectivity. + +- Due to the lack of segmentation of test workloads on-prem, + end-to-end segmentation is not possible and a single VRF is utilized + for all test workloads including, but not limited to: QA, Project + Test, Integration Testing, and Test. + +## Conceptual Design + +The USCB GCP conceptual design uses multiple host projects, multiple +service projects, and multiple shared VPCs to achieve high availability +while also maintaining network segmentation between environments. +Connectivity to Census on-prem environments uses GCP HA VPN connections +to redundant locations. VPC peering connections are used to interconnect +the Shared VPCs in the host project to the service projects. + +![](images/networking/media/image2.png){width="6.5in" height="2.9125in"} + +[]{#_Toc157695668 .anchor}Figure 1 GCP Conceptual Networking Design + +## Logical Design + +![](images/networking/media/image3.png){width="5.0in" height="4.0in"} + +[]{#_Toc157695669 .anchor}Figure 2 Logical GCP Network Design + +### GCP Cloud Router/ Cloud HA VPN Design + +Cloud Router is a fully distributed and managed Google Cloud service +that uses the Border Gateway Protocol (BGP) to advertise IP prefixes. It +establishes dynamic routes based on the BGP advertisements that it +receives from a peer. When connecting to the Census on-prem network, +Cloud Router uses BGP to dynamically exchange routes between the Google +Cloud VPC network and the Census on-premises network. Prefix and next +hop changes automatically propagate between the GCP VPC network and the +other network without the need for static routes. Each Hub VPC has a GCP +Cloud Router and Cloud HA VPN. GCP provides end-to-end IPsec-encrypted +connections between Census on-prem networks and a Cisco ASR Cloud +routers at HQ and BCC locations. There are multiple VPN tunnels per +environment that that terminate on Cisco Cloud routers via a connection +over the Internet. Initially, the minimum number of tunnels are listed +below but may be increased as required. + +- Services -- 4 VPN tunnels + +- Dev --4 VPN tunnels + +- Test -- 4 VPN tunnels + +- Stage -- 4 VPN tunnels + +- Prod -- 4 VPN tunnels + +The VPN tunnels terminate on routers at two Census locations: Bowie +Computer Center (BCC) and Census Headquarters (HQ). The traffic flowing +to/from GCP will be secured by an on-prem Cisco firewall prior to being +forwarded to the on-prem Cisco router. On the cloud side, the Cloud HA +VPN in the Hub VPC will propagate routes from on-prem through the Cloud +Router to cloud VPCs. Likewise, the Cloud Router propagates per +environment routes learned from the VPCs to the on-prem environment. +Traffic over the VPN tunnels utilizes HQ as the preferred route. + +### Host Project/ Hub VPC Design + +Within the GCP organization, we designate projects as Shared VPC host +projects for each of the environments shown below: + +- Dev + +- Test + +- ITE + +- Stage + +- Prod + +- Non-routable Lab + +- Shared Services-Dev + +- Shared Services-Test + +- Shared Services-Prod + +The GCP organization uses Shared VPCs in common host projects. +Application and systems teams from peered service projects in the +organization create resources that use Shared VPC network subnets, +specifically dedicated for their appropriate environments. + +### VPC Network Peering Design + +Peering connections are established between the centralized Hub VPC +(Prod and Non-Prod) and the Shared VPCs in the host project for each +environment. Centralized services, hosted on the production shared +services VPC, are peered to all environmental shared VPCs. Subnet routes +for each peered VPC are automatically exchanged over the network peering +connection. There are a few important caveats about VPC Network Peering: + +- Resources in a peered VPC network cannot use DNS names created by a + local VPC network. A peered VPC network can\'t use Cloud DNS managed + private zones that are authorized for only a local VPC network, so + we deploy DNS cloud peering zones to address this issue. + +- VPC Network Peering does not exchange any VPC firewall rules. VPC + firewall rules in one VPC network can\'t specify targets or sources + using network tags or service accounts from the other VPC network. + However, the same target network tag can be used in both networks. + +### Shared VPC Design + +GCP allows resources in an organization to connect to a shared VPC +securely and efficiently. To deploy a Shared VPC, a host project must be +designated that contains the Shared VPC. Service projects are defined +that have access to the Shared VPC subnets in the host project. When an +existing project accesses the host project, the project becomes known as +a service project. New resources created in the service project, such as +instances, can use the subnets from the Shared VPC to communicate. For +GCP, we will create new service projects, and all resources created will +support connectivity via the subnets available from Shared VPC, as shown +in Figure 3 Shared VPC Design. + +![](images/networking/media/image4.png){width="4.210416666666666in" +height="5.375in"} + +[]{#_Ref153543324 .anchor}Figure 3 Shared VPC Design + +### VPC and Subnet CIDR Allocation + +The GCP network is allocated a /16 CIDR block for the GCP network: + +- IPv4: 10.36.0.0/16 GCP Cloud Internal + +- IPv6: 2610:20:2061::/48 GCP Cloud + +**Note:** Cloud-native applications are typically dynamic and require +sufficient IP space to support auto-scaling. Shared VPCs are intended +for use across all GCP projects and environments, so a sufficient IP +space allocation must exist. Capacity planning occurs monthly, and the +usage of the address space will be reviewed as outlined in the GCP +Operation Plan. + +A VPC spans multiple regions. VPC networks, including their firewall +rules and associated routes, are global resources. However, each subnet +is associated with a specific region. At the time of this writing, USCB +is configuring GCP so that resources are deployed within the +**us-east4** region. + +Each VPC is assigned a CIDR block. For additional details on the +allocation of addresses, see the GCP CIDR Applications link in the +reference section of this document. At the time of this writing, a +general VPC Subnet allocation for each environment is shown below: + +[]{#_Toc157695673 .anchor}Table 2 VPC Subnets + + ----------------------------------------------------------------------- + Subnets Mask per AZ Total Addresses + --------------------------------- --------------------- --------------- + App /22 1024 + + Load Balancer /24 256 + + Load Balancer Regional Proxy /24 256 + + SPA-Functions-1 /28 16 + + SPA-CloudRun-1 /28 16 + + + + + + + ----------------------------------------------------------------------- + +### Private Service Connect Design + +Private Service Connect establishes a private connection from a VPC to +GCP services over the GCP backbone. Private Service Connects are +deployed in each VPC and resources utilize Private Hosted Zones to +resolve the IP address of the Private Service Connect. + +When the Private Service Connect is created in each VPC, GCP-managed +Cloud DNS private hosted zone (PHZ) enables the resolution of public GCP +service endpoint to the private IP of the interface endpoint. The +managed PHZ is available in the VPC with the interface endpoint for each +environment (Dev, Test, Prod, etc.). + +The following figure depicts resources accessing a VPC Private Service +Connect to reach Google services over the Google backbone. + +![](images/networking/media/image5.png){width="6.401388888888889in" +height="4.090972222222222in"} + +[]{#_Toc157695671 .anchor}Figure 4 Example VPC Private Service +Connection to GCP services + +## Interfaces + +The GCP Network uses multiple VPN tunnels to establish BGP peering +connections to USCB on-prem Cisco ASR routers for IP connectivity and +routing. + +## Consumption + +The GCP Network will be managed and administered by the GCP Network +administration team. The consumers of the service are the application +and platform teams that will build tools, services, and applications in +GCP. + +## Configuration Management + +The GCP Network will be deployed following the GCP iSDLC and delivered +using infrastructure as Code. + +## Key Performance Indicators + +No KPIs are defined at this time. There are a variety of USCB standard, +GCP-native, and 3^rd^ party tools that will be used to measure +performance on the GCP Network including Cloud Monitoring, Ping, iPerf3, +and Solarwinds. The GCP Network key performance indicators for VPN, VPC +and Shared VPC traffic on the network are: + +- VPN Connection Status + +- Inbound Packet per Second + +- Outbound Packets per Second + +- Inbound Bytes per Second + +- Outbound Bytes per Second + +- Throughput + +- Round-Trip Time (RTT) + +- Retransmission Rate + +- Window Size + +These metrics and associated monitoring activities are detailed in the +USCB Enterprise GCP Operating Guide. + +## Service Level Agreements + +Network services fall under the Google Cloud Compute Engine SLA. + +## Patching and Updates + +The services described in this design document (Shared VPC, Cloud HA +VPN, Hub VPC and Cloud DNS) are offered by GCP and all upgrades are the +responsibility of Google. + +## Roles and Responsibilities + + ----------------------------------------------------------------------------------- + Role + ---------------------- ----------------- ------------------------------------------ + NetworkAdministrator gr-net.admin Requires admin level permissions to + create, modify and delete GCP cloud + networking resources including VPCs, + Shared VPC, Cloud DNS, VPN Connections, + Cloud Logging, and similar services. + + NetworkSupport gr-net.operator Permissions to read, describe and list GCP + cloud networking resources including VPCs, + Shared VPC, Cloud DNS, VPN Connections, + Cloud Monitoring, and similar services. + ----------------------------------------------------------------------------------- + +## Service Limits & Capacity Planning + +At the time of this writing, the following tables describe the current +service limits. Refer to official Google Cloud Platform documentation +when planning any future use cases. + +[]{#_Toc157695674 .anchor}Table 3 GCP VPC Service Limits + + ----------------------------------------------------------------------------------------------------- + **Shared VPC limits** + ---------------------------- -------------- --------------------------------------------------------- + The number of service + projects that can be + attached to a host project + is a configurable + per-project quota. + + **Item** **Limit** **Notes** + + Number of Shared VPC host 100 [To request an update to this limit, file a support + projects in a single case.](https://cloud.google.com/support-hub) + organization + + Number of host projects to 1 This limit cannot be increased. + which a service project can + attach + + + + **Per network** + + The following limits apply + to VPC networks. + + **Item** **Limit** **Notes** + + [Subnet IP + ranges]{.underline} + + Primary IP ranges per subnet 1 [Each subnet must have exactly one primary IP range (CIDR + block). This range is used for VM primary internal IP + addresses, VM alias IP ranges, and the IP addresses of + internal load balancers. This limit cannot be + increased.](https://cloud.google.com/vpc/docs/alias-ip) + + Maximum number of secondary 30 Optionally, you can define up to thirty secondary CIDR + IP ranges per subnet blocks per subnet. These secondary IP ranges can only be + used for alias IP ranges. This limit cannot be increased. + + [Routes]{.underline} + + Maximum number of network 256 The maximum number of network tags that you can associate + tags per route with a static route. This limit cannot be increased. + + + + **VPC Network Peering + limits** + + **Item** **Peering **Notes** + group limit** + + [Peering group]{.underline} + + Maximum number of 25 The maximum number of networks that can connect to a + connections to a single VPC given VPC network using VPC Network Peering. + network + ----------------------------------------------------------------------------------------------------- + +The following limits apply to Cloud VPN. In this table, *VPN tunnel* +means either a Classic VPN tunnel or an HA VPN tunnel. Unless otherwise +stated, these limits cannot be increased. + +[]{#_Toc157695675 .anchor}Table 4 Cloud VPN Limits + + --------------------------------------------------------------------------------------------------------------------------------------------------------- + Item Limit Notes + ----------------------- ----------------------- --------------------------------------------------------------------------------------------------------- + Bandwidth per VPN 250,000 packets per 250,000 packets per second is roughly equivalent to 1 Gbps to 3 Gbps, depending on the average packet + tunnel second for the sum of size within the tunnel. + ingress and egress + + Cloud VPN only throttles egress IPsec traffic. It does not throttle ingress traffic. + + [For more details, see Network + bandwidth.](https://cloud.google.com/network-connectivity/docs/vpn/concepts/overview#network-bandwidth) + --------------------------------------------------------------------------------------------------------------------------------------------------------- + +## Consumption, Licensing, and Assets + +- VPC hourly charge - Customers are charged for GCP VPC on an hourly + basis. + +- VPC data processing charge - Data Processing is charged to the VPC + owner who sends outbound traffic to a VPC. + +- VPC data processing charge for outbound inter-Region peering + attachments - Inbound inter-Region data transfer charges are free. + +- Private Service Connect + +- Cloud DNS + +- Cloud HA VPN + +# Cost Consideration + +At the time of this writing, the following pricing details are described +in Table 5 Virtual Private Cloud Pricing, Table 6 Cloud VPN, and Table 7 +Private Service Connect. Architects and engineers should refer to the +Google Published pricing data and confer with USCB FinOps administrators +to forecast and model pricing for ongoing consumption of GCP resources. +All assets are tagged and tracked in Apptio Cloudability. + +[]{#_Ref153791171 .anchor}Table 5 Virtual Private Cloud Pricing + +![](images/networking/media/image6.png){width="6.178844050743657in" +height="4.054864391951006in"} + +[]{#_Ref153791180 .anchor}Table 6 Cloud VPN + ++---------------------------+------------------------------------------+ +| Item | Price per hour (USD) | ++===========================+==========================================+ +| Hourly charge for each | \$0.055 | +| tunnel attached to the | | +| gateway.\ | | +| \ | | +| HA VPN only: For 99.99% | | +| availability, you must | | +| configure two tunnels. | | ++---------------------------+------------------------------------------+ +| IPsec traffic | You are charged as follows: | +| | | +| | If the Cloud VPN tunnel connects to | +| | another Cloud VPN gateway, you are | +| | charged egress pricing as described in | +| | VM-VM egress pricing within Google | +| | Cloud. Egress pricing is based on the IP | +| | addresses of the destination VPN | +| | gateway---not the destination VM | +| | address.\ | +| | \ | +| | If the source and destination Cloud VPN | +| | gateways are in the same Google Cloud | +| | region, egress traffic is billed as | +| | traffic between zones in the same | +| | region. | +| | | +| | If the Cloud VPN tunnel connects to a | +| | VPN gateway outside of Google Cloud, you | +| | are charged as described in Internet | +| | egress rates. | ++---------------------------+------------------------------------------+ +| External IP address for | You are charged as described in IP | +| VPN gateway | address pricing.\ | +| | \ | +| | An external IP address is charged only | +| | if it is not being used by a VPN tunnel. | ++---------------------------+------------------------------------------+ + +[]{#_Ref153791215 .anchor}Table 7 Private Service Connect + + --------------------------------------------------------------------------------------------------------------------------------- + Item Price per hour (USD) Price per GiB + processed,\ + both egress and ingress + (USD) + --------------------------------------------------------------------------------- ----------------------- ----------------------- + Private Service Connect endpoint (forwarding rule) used to [access Google \$0.01 No data charge + APIs](https://cloud.google.com/vpc/docs/configure-private-service-connect-apis) + + --------------------------------------------------------------------------------------------------------------------------------- + +# Backup and Recovery + +GCP Networking resources provide the transport and connectivity services +that permit cloud-based applications, systems, and services to +communicate, shared data and exchange information. GCP Networking +provides the infrastructure and services to support backup and recovery +services in the cloud including: + +- Global, Multi Region Network + +- Multiple Availability Zones + +- Autoscaling + +- Serverless Architecture + +- Load Balancers + +- Cloud DNS + +GCP networking configurations for USCB are deployed through +Infrastructure as Code. If required, recovery of networking +configurations use the latest release code base stored in the Gitlab +version-controlled repository. See the GCP Operations Plan for details +on business continuity. + +# Security + +## Authentication, Access, Authorization + +Access to GCP Networking resources is provided by via GCP Cloud +Identity. Cloud Identity provides support for permission sets, users, +roles, groups, and policies required to manage and administer GCP +Networking services. + +## Auditing + +Organizational security audit schedules and outcomes are described in +the GCP Operations Plan. + +## Logging + +GCP Cloud Logging is enabled to capture VPC flow logs. Flow log data is +captured by GCP Cloud Logging and stored in GCP Cloud Storage. After +creating a flow log, an administrator/operator can retrieve and view its +data in GCP Log Explorer and/or GCP BigQuery. Flow log data is collected +outside of the path of your network traffic, and therefore does not +affect network throughput or latency. You can create or delete flow logs +without any risk of impact to network performance. + +**Cloud Audit Logs** + +GCP Cloud Audit Logs will be used to capture detailed information about +the calls made to the Shared VPC API and store them as log files in GCP +Cloud Storage. You can use these Cloud Audit logs to determine which +calls were made, the source IP address where the call came from, who +made the call, when the call was made, and so on. + +## Alerts + +**Cloud Monitoring metrics** + +GCP Cloud Monitoring is used to retrieve statistics about data points +for your Shared VPCs as an ordered set of time series data, known as +*metrics*. You can use these metrics to verify that your system is +performing as expected. + +GCP VPC publishes data about VPCs to GCP Cloud Monitoring. We can +retrieve statistics about the VPCs as an ordered set of time-series +data, or metrics and this can be used to generate Cloud Monitoring. + +GCP is integrated with Solarwinds for monitoring VPN connectivity. + +# References + +## Tagging + +[Naming and Tagging +Standards](https://uscensus.sharepoint.com/:x:/s/DITDATO/EV12IGvaxTJLnkkB-vz7c3IB6F_FUFcEqgy75ep0vuQA2Q?e=9oN2NW) + +## Infrastructure as Code + + + +## Links + +- [GCP FedRAMP Services in + Scope](https://aws.amazon.com/compliance/services-in-scope/FedRAMP/) + +- [Compute Engine Service Level Agreement (SLA) \| Google + Cloud](https://cloud.google.com/compute/sla) + +- GCP Best Practices + + - [Shared VPC + Design](https://cloud.google.com/architecture/best-practices-vpc-design#shared-common-vpc) + + - [Liens](o%09https:/cloud.google.com/vpc/docs/provisioning-shared-vpc#protectsharedvpc) + + - [Accidental Shared VPC + Shutdowns](https://cloud.google.com/billing/docs/how-to/secure-project-billing-account-link) + + - [Constrain host project + attachments](https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints) + + - [Constrain the subnets in the host + project](https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints) + +```{=html} + +``` +- [GCP Solutions + Architecture](https://uscensus.sharepoint.com/:w:/s/DITDATO/EaDaa4eHgJhPqnt-9FlGq2YBnV_F83HIhbree1TuW219Zg?e=jK1hQv) + +- [GCP DNS Detailed + Design](https://uscensus.sharepoint.com/:w:/s/DITDATO/ESnMqy6UOX5CttAk5vaYXL4BSn5JVRPWAwQ_1nRhXkepsw?e=XchWOq) + +- [GCP Identity Detailed + Design](https://uscensus.sharepoint.com/:w:/s/DITDATO/ESt6lFhB4RhPnLnHtZEX9nABfZLUyrf9Oezy1sRX_ewqAQ?e=sD9sxk) + +- [GCP Operations Suite Cloud Logging Detailed + Design](https://uscensus.sharepoint.com/:w:/s/DITDATO/EYSWYQAs28pLmu2I1T8Gt7ABC3zzy3ikv2XgB2lHLXJZJQ?e=QSDXv0) + +- [GCP Standards & + Configurations](https://uscensus.sharepoint.com/:f:/s/DITDATO/EpC2S1BFNCVNqU93LkYHXzsBYFzBXygEJZEDYHJjqmBFbA?e=kNdbl8) + +- [GCP CIDR + Allocation](https://uscensus.sharepoint.com/:x:/s/DITDATO/EeY54Ec_dyFKhW77iTdWlUsBfLUpkoRy_m6ZbwZ-tQit7Q?e=cuQSB4) + +- [GCP Operating + Plan](https://uscensus.sharepoint.com/:w:/s/DITDATO/ETXm6FlSxm5PrqipdIqa1aQB3uIXt8gBvf01KZcd4wl-ow?e=9Q2SKd) + +## Shared VPC Best Practices + +- Create a VPC network for each autonomous team, with Services in a + common VPC network + + +- Automatically prevent accidental deletion of host projects with a + special lock---called a lien. Liens can also be placed upon a + project automatically. When Identity and Access Management (IAM) + service accounts are allowed from one project to be attached to + resources in other projects, a lien is placed upon the project where + the service accounts are located. + ( + ) + +- Prevent a possible occurrence of an accidental Shared VPC shutdown + due to inactive or disabled billing + () + +- Constrain host project attachments + () + +- Constrain the subnets in the host project that a service project can + use + () diff --git a/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/private-service.md b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/private-service.md new file mode 100644 index 00000000..9277173c --- /dev/null +++ b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/private-service.md @@ -0,0 +1,665 @@ +## ![](images/private-service/media/image1.jpeg){width="3.0in" height="2.1041666666666665in"}![](images/private-service/media/image2.png){width="2.7777777777777777in" height="2.0833333333333335in"}![](images/private-service/media/image3.png){width="2.7430555555555554in" height="2.1041666666666665in"} {#section .Alphabetical .unnumbered} + +[0](#_Toc161138597) + +1 Revision History [1](#revision-history) + +2 Overview [2](#overview) + +2.1 Background [2](#background) + +2.2 Scope [2](#scope) + +2.3 Audience [2](#audience) + +2.4 Governance [2](#governance) + +2.5 Terms and Definitions [2](#terms-and-definitions) + +3 Service Design [3](#service-design) + +3.1 Value [3](#value) + +3.2 Capabilities, Features, and Requirements +[3](#capabilities-features-and-requirements) + +3.3 Assumptions [3](#assumptions) + +3.4 Constraints [3](#constraints) + +3.5 Logical Design [4](#_Toc161138610) + +3.6 Interfaces [5](#interfaces) + +3.7 Consumption [5](#consumption) + +3.8 Configuration Management [5](#configuration-management) + +3.9 Key Performance Indicators [5](#key-performance-indicators) + +3.10 Service Level Agreements [5](#service-level-agreements) + +3.11 Patching and Updates [5](#patching-and-updates) + +3.12 Roles and Responsibilities [6](#roles-and-responsibilities) + +3.13 Service Limits & Capacity Planning +[6](#service-limits-capacity-planning) + +3.14 Consumption, Licensing, and Assets +[6](#consumption-licensing-and-assets) + +4 Cost Consideration [6](#cost-consideration) + +5 Backup and Recovery [7](#backup-and-recovery) + +6 Security [7](#security) + +6.1 Authentication, Access, Authorization +[7](#authentication-access-authorization) + +6.2 Auditing [7](#auditing) + +6.3 Logging [7](#logging) + +6.4 Alerts [7](#alerts) + +6.5 Acronyms [8](#acronyms) + +6.6 Tagging [8](#tagging) + +6.7 Infrastructure as Code [8](#infrastructure-as-code) + +6.8 Links [8](#links) + +6.9 Appendix A [9](#appendix-a) + +List of Figures + +**No table of figures entries found.** + +List of Table + +[Table 1 -- Revision History [1](#_Toc125379086)](#_Toc125379086) + +[Table 2 -- Acronyms [4](#_Toc125379087)](#_Toc125379087) + +# Revision History + +[]{#_Toc125379086 .anchor}Table 1 -- Revision History + + ------------------------------------------------------------------------------- + Version Date Author Description + --------- ------------ ----------- -------------------------------------------- + 0.01 8/3/2023 Ethan Rowe Initial template draft + + 0.02 03/06/2023 Nathaniel Private Service Connect initial content + Ely + + + ------------------------------------------------------------------------------- + + : Table 2 Private Google Access Zone Entry + +# Overview + +## Background + +Private Service Connect (PSC) is a capability of Google Cloud networking +that allows the Census Bureau access to GCP managed services and APIs +privately from inside the GCP VPC network. Private Service Connect +establishes a private connection from a VPC to GCP services over the GCP +backbone. Private Service Connects are deployed in each VPC and +resources utilize Private Hosted Zones to resolve the IP address of the +Private Service Connect. + +## Scope + +This document details the engineering and operations lifecycle for +Private Service Connect (PSC) Private Google Access. It is intended to +document design decisions and provide guidelines for implementation and +operation of this service for the United States Census Bureau (USCB). + +## Audience + +Decennial Infrastructure Engineering + +Decennial Operations Engineering + +Decennial IT Directorate (DITD) Stakeholders + +Telecommunications Organization (TCO) + +## Governance + +This document, its associated service(s), and guidelines regarding the +use of this service are governed by the DITD Engineering Working Group +(EWG). Changes to this service require must be presented by the Product +Owner and/or Technical Lead and approved by the EWG. + +## Terms and Definitions + +- **Private Google Access** --- Private Google provides the capability + to access [Google APIs and + services](https://developers.google.com/apis-explorer/#p/) in + Google\'s production infrastructure without traversing the public + internet. + +- **[Private Service Connect + Endpoints](https://cloud.google.com/vpc/docs/private-service-connect#endpoints)** + --- PSC Endpoints are IP addressable resources in GCP VPCs mapped to + the PSC Service, providing connectivity to the Google Cloud APIs. + These are equivalent to an Amazon Web Services (AWS) VPC endpoints. + +- **Host Project** --- Equivalent of an Account AWS where Shared VPCs, + VPN connections, and core networking resources are defined and + provisioned. + +- **Service Project** --- A Service project resides outside the host + project but subscribes to the host project's network services (i.e., + Shared VPC). + +- **Shared VPC** --- A Shared VPC allows an organization to connect + resources from multiple projects to a common Virtual Private Cloud + (VPC) network to communicate with each other securely and + efficiently by using internal IP addresses from that network. + +# Service Design + +## Value + +The USCB GCP Private Service Connect design provides reliable, scalable, +and secure communication services to Google APIs from VPCs using +Google's internal network. + +- **Secure Connectivity --** Traffic destined to Google APIs are kept + securely within the internal USCB GCP network and do not traverse + the public internet. Traffic communication across the PSC endpoints + is secured using https. + +- **Monitored --** VPC traffic flows are monitored for security and + network anomalies and assist with troubleshooting using a Palo Alto + IDS managed by OIS. + +- **Scalability --** GCP automatically scales its infrastructure to + meet demand for network traffic to Google APIs from the VPC + endpoint. Service limits apply per VPC, per region. + +## Capabilities, Features, and Requirements + +Private Service Connect is a component of the USCB GCP Network design +and provides connectivity to Google APIs in the GCP cloud. + +- Secure network connectivity to Google APIs using Private Service + Connect endpoints. + +- To the greatest extent possible, perform cost optimization and + complexity reduction by leveraging the capability to share cloud + resources.  + +- High availability, fault tolerance, and resilience through redundant + networking.  + +- Integration with GCP IAM to control privileged access to cloud + networking resources and to support security compliance, auditing, + and logging.  + +## Assumptions + +- Routing to GCP Private Service Connect endpoints from on-premise + networks is outside the scope at the time of this writing. + +```{=html} + +``` +- Support for IPv6 integration with Private Service Connect is out of + scope. IPv6 addressing and its support with PSC is reserved for + future integration when required by TCO. + +- Private Service Connect endpoints are required within each GCP VPC. + +## Constraints + +- At the time of this writing, Private Service Connect is only + configured to access Google APIs and services using the **ALL APIs** + bundle. + +- The design and usage of PSC is not intended to provide private + networking connections between VPCs and associated services in a + consumer / producer configuration. + +## Logical Design + +Private Service Connect is configured within GCP for access to Google +API without traversing the public internet, known as Private Google +Access. Private Google Access is enabled on a subnet-by-subnet basis. + +![](images/private-service/media/image7.png){width="6.401388888888889in" +height="4.090972222222222in"} + +Figure 1 PSC Private Google Access + +### DNS Configuration + +Each Private Service Connect must contain a corresponding entry in a +Google Cloud DNS Private Hosted zone for routing purposes. A PHZ is +created with a corresponding DNS name. By creating private endpoints +using a global internal IP address, DNS names are assigned to these +internal IP addresses. Endpoints within the VPC are then able to +communicate via IPv4 and forward traffic to the Google Services Network +and corresponding APIs. + + ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + Zone name DNS name Description Zone type + ----------------------------------------------------- ------------------- ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- ----------- + *goog-psc-vpcs-services-ssvcpr-7131329659510185153* p.googleapis.com. \[DO NOT DELETE\]\[Private Service Connect for Google APIs\] The private managed DNS zone for the Service Directory namespace Service + https://servicedirectory.googleapis.com/v1/projects/gcp-inf-vpcs-ssvcprod-x2vdlm4/locations/us-central1/namespaces/goog-psc-vpcs-services-ssvcpr-7131329659510185153. Directory + + ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + + : Table 4 Private Google Access Configuration Requirements + +## Interfaces + +In each VPC, an Private Service Connect endpoint is created which uses a +single IPv4 address, and the endpoint in each VPC uses the same IP +address across the USCB implementation of GCP, 10.36.119.241. + +A GCP-managed Cloud DNS private hosted zone (PHZ) enables the resolution +of public GCP service endpoint to the private IP of the interface +endpoint. The managed PHZ is available in the VPC with the interface +endpoint for each environment (Dev, Test, Prod, etc.). + +Systems within a VPC send packets to the external IP addresses of Google +APIs and services using Private Google Access if all these conditions +are met: + +- The subnet where traffic is originating has Private Google Access is + enabled. + +- The VPC network that contains the subnet meets the [network + requirements for Google APIs and + services](https://cloud.google.com/vpc/docs/configure-private-google-access#requirements). + +- The source IP address of packets on the subnet uses an internal IPv4 + address from an alias IP range + +## Consumption + +The GCP Network will be managed and administered by the GCP Network +administration team. The consumers of the service are the application +and platform teams that will build tools, services, and applications in +GCP. + +## Configuration Management + +GCP Network services, including Private Service Connect, will be +deployed following the GCP iSDLC and delivered using infrastructure as +Code. + +## Key Performance Indicators + +No KPIs are defined at this time. There are a variety of USCB standard, +GCP-native, and 3^rd^ party tools that will be used to measure +performance on the GCP Network including Cloud Monitoring, Ping, iPerf3, +and Solarwinds. + +## Service Level Agreements + +Network services, including Private Service Connect and Private Google +Access, fall under the Google Cloud Compute Engine SLA. + +## Patching and Updates + +GCP Private Service Connect and Private Google Access are features of +the Google Cloud Platform provided by Google. As this is a managed +service, all patches or upgrades are solely the responsibility of Cloud +Service Provider. However, any new VPCs and/or subnets created within +USCB's GCP boundary requires associated configurations to enable Private +Service Connect, Private Google Access, and associated DNS Private +Hosted Zones (PHZs). + +## *Roles and Responsibilities* + + ----------------------------------------------------------------------------------- + Role + ---------------------- ----------------- ------------------------------------------ + NetworkAdministrator gr-net.admin Requires admin level permissions to + create, modify and delete GCP cloud + networking resources including VPCs, + Shared VPC, Cloud DNS, Private Service + Connect, and Private Google Access. + + NetworkSupport gr-net.operator Permissions to read, describe and list GCP + cloud networking resources including VPCs, + Shared VPC, Cloud DNS, Private Service + Connect, and Private Google Access. + ----------------------------------------------------------------------------------- + +## Service Limits & Capacity Planning + +The following Quotas apply with the Private Service Connect service: + + ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + Quota Type Quota Description Quota name: Quota Value + --------------------------------------------------------------------------------------------------------------------------------------------------------- ------------------------- ----------------------------------------------------- ----------- + [PSC internal LB forwarding The maximum number of PSC-INTERNAL-LB-FORWARDING-RULES-per-project-region 1000 + rules](https://console.cloud.google.com/iam-admin/quotas?service=compute.googleapis.com&metric=compute.googleapis.com/psc_internal_lb_forwarding_rules) Private Service Connect + forwarding rules + (endpoints) that a + service consumer can + create to connect to + producer services. This + quota is per region, per + project. + + [Service attachments](https://console.cloud.google.com/iam-admin/quotas?service=compute.googleapis.com&metric=compute.googleapis.com/service_attachments) The maximum number of SERVICE-ATTACHMENTS-per-project-region 1000 + Private Service Connect + service attachments that + a service producer can + create. This quota is per + region, per project. + + [Network attachments](https://console.cloud.google.com/iam-admin/quotas?service=compute.googleapis.com&metric=compute.googleapis.com/service_attachments) The maximum number of NETWORK-ATTACHMENTS-per-project-region 1000 + network attachments that + a Private Service Connect + consumer can create. This + quota is per region, per + project. + ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + +## Consumption, Licensing, and Assets + +Each VPC is configured with a PSC VPC endpoint. There is no separate +licensing costs associated with Private Service Connect other than the +hourly CSP consumption charge for each endpoint. The endpoints, +forwarding rules, and configuration of these endpoints are maintained as +Infrastructure as Code (IaC) in Gitlab. + +# Cost Consideration + +At the time of this writing, the following details are published +regarding Private Service Connect endpoint pricing. Architects and +engineers should refer to the Google Published pricing data and confer +with USCB FinOps administrators to forecast and model pricing for +ongoing consumption of GCP resources. All assets are tagged and tracked +in Apptio Cloudability. + + ------------------------------------------------------------------------------------------------------------------------------ + Item Price per hour (USD) Price per GiB + processed, Data + In/Out charges + --------------------------------------------------------------------------------- ------------------------- ------------------ + Private Service Connect endpoint (forwarding rule) to access the regional \$0.01 No data charge + endpoints of Google APIs + + [Private Service Connect endpoint (forwarding rule) used to access Google \$0.01 No data charge + APIs](https://cloud.google.com/vpc/docs/configure-private-service-connect-apis) + ------------------------------------------------------------------------------------------------------------------------------ + +# Backup and Recovery + +GCP Networking resources provide the transport and connectivity services +that permit cloud-based applications, systems, and services to +communicate, shared data and exchange information. GCP Networking +provides the infrastructure and services to support backup and recovery +services for Private Service Connect, including: + +- Global, Multi Region Network + +- Cloud DNS + +- Serverless Architecture + +GCP networking configurations for USCB are deployed through +Infrastructure as Code. If required, recovery of networking +configurations use the latest release code base stored in the Gitlab +version-controlled repository. See the GCP Operations Plan for details +on business continuity. + +# Security + +## Authentication, Access, Authorization + +Access to GCP Networking resources is provided by via GCP Cloud +Identity. Cloud Identity provides support for permission sets, users, +roles, groups, and policies required to manage and administer GCP +Networking services. + +## Auditing + +Organizational security audit schedules and outcomes are described in +the GCP Operations Plan. + +## Logging + +GCP Cloud Logging is enabled to capture VPC flow logs. Flow log data is +captured by GCP Cloud Logging and stored in GCP Cloud Storage. After +creating a flow log, an administrator/operator is able to retrieve and +view data in GCP Log Explorer and/or GCP BigQuery. Flow log data is +collected outside of the path of network traffic, and therefore does not +affect network throughput or latency. There is no impact or risk to +network performance by enabling GCP Cloud Logging. + +**Cloud Audit Logs** + +Cloud Logging captures all API requests made from subnets that have +Private Google Access enabled. Log entries identify the source of the +API request as an internal IP address of the calling function, instance, +etc. Cloud Logging details are sent to the USCB SEIM tool, Microsoft Log +Analytics. + +## Alerts + +**Cloud Monitoring metrics** + +GCP Cloud Monitoring is used to retrieve statistics about data points in +VPCs as an ordered set of time series data, known as *metrics*. These +metrics are used to verify that a system or application is performing as +expected. GCP VPC publishes data about VPCs, including Private Service +Connect and the VPC endpoints, to GCP Cloud Monitoring. References + +## Acronyms + +[]{#_Toc125379087 .anchor}Table 3 -- Acronyms + + ----------------------------------------------------------------------- + Acronym Definition + ------------- --------------------------------------------------------- + DITD Decennial Information Technology Directorate + + + + + ----------------------------------------------------------------------- + +## Tagging + +[Decennial Naming and Tagging +Standards](https://uscensus.sharepoint.com/:f:/s/DITDSIRS/EpbVeuUQbE1Ftjo-SJpUjj4BG9Nq15mRUQdNsUaQcRyeyw?e=ZMCX3r) + +## Infrastructure as Code + + + +## *Links* + +- [GCP FedRAMP Services in + Scope](https://aws.amazon.com/compliance/services-in-scope/FedRAMP/) + +- [Compute Engine Service Level Agreement (SLA) \| Google + Cloud](https://cloud.google.com/compute/sla) + +- GCP Best Practices + + - [Shared VPC + Design](https://cloud.google.com/architecture/best-practices-vpc-design#shared-common-vpc) + + - [Liens](https://uscensus.sharepoint.com/sites/DITDSIRS/Shared%20Documents/General/C4.1.4%20IT%20Integration/Infrastructure/04_Solutions%20and%20Service%20Architecture/Google/GCP%20Networking%20Designs/o%09https:/cloud.google.com/vpc/docs/provisioning-shared-vpc#protectsharedvpc) + + - [Accidental Shared VPC + Shutdowns](https://cloud.google.com/billing/docs/how-to/secure-project-billing-account-link) + + - [Constrain host project + attachments](https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints) + + - [Constrain the subnets in the host + project](https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints) + +```{=html} + +``` +- [GCP Solutions + Architecture](https://uscensus.sharepoint.com/:w:/s/DITDATO/EaDaa4eHgJhPqnt-9FlGq2YBnV_F83HIhbree1TuW219Zg?e=jK1hQv) + +- [GCP DNS Detailed + Design](https://uscensus.sharepoint.com/:w:/s/DITDATO/ESnMqy6UOX5CttAk5vaYXL4BSn5JVRPWAwQ_1nRhXkepsw?e=XchWOq) + +- [GCP Identity Detailed + Design](https://uscensus.sharepoint.com/:w:/s/DITDATO/ESt6lFhB4RhPnLnHtZEX9nABfZLUyrf9Oezy1sRX_ewqAQ?e=sD9sxk) + +- [GCP Operations Suite Cloud Logging Detailed + Design](https://uscensus.sharepoint.com/:w:/s/DITDATO/EYSWYQAs28pLmu2I1T8Gt7ABC3zzy3ikv2XgB2lHLXJZJQ?e=QSDXv0) + +- [GCP Standards & + Configurations](https://uscensus.sharepoint.com/:f:/s/DITDATO/EpC2S1BFNCVNqU93LkYHXzsBYFzBXygEJZEDYHJjqmBFbA?e=kNdbl8) + +- [GCP CIDR + Allocation](https://uscensus.sharepoint.com/:x:/s/DITDATO/EeY54Ec_dyFKhW77iTdWlUsBfLUpkoRy_m6ZbwZ-tQit7Q?e=cuQSB4) + +- [GCP Operating + Plan](https://uscensus.sharepoint.com/:w:/s/DITDATO/ETXm6FlSxm5PrqipdIqa1aQB3uIXt8gBvf01KZcd4wl-ow?e=9Q2SKd) + +## *Appendix A* + +The Appendix below contains specific configuration information and +requirements for Private Service Connect and Private Google Access. + ++----------------------+--------------------------------+-------------+ +| **Domain and IP | Supported services | Example | +| address ranges** | | usage | ++======================+================================+=============+ +| Default domains. | Enables API access to most | The default | +| | Google APIs and services | domains are | +| All domain names for | regardless of whether they are | used when | +| Google APIs and | supported by VPC Service | you don\'t | +| services *except | Controls. Includes API access | configure | +| for* | to Google Maps, Google Ads, | DNS records | +| pr | and Google Cloud. Includes | for | +| ivate.googleapis.com | Google Workspace web | private.goo | +| and | applications such as Gmail and | gleapis.com | +| restri | Google Docs, and other web | and | +| cted.googleapis.com. | applications. | rest | +| | | ricted.goog | +| Various IP address | | leapis.com. | +| ranges---you can | | | +| determine a set of | | | +| IP ranges that | | | +| contains the | | | +| possible addresses | | | +| used by the default | | | +| domains by | | | +| referencing [IP | | | +| addresses for | | | +| default | | | +| domains](http | | | +| s://cloud.google.com | | | +| /vpc/docs/configure- | | | +| private-google-acces | | | +| s#ip-addr-defaults). | | | ++----------------------+--------------------------------+-------------+ +| pr | Enables API access to most | Use | +| ivate.googleapis.com | Google APIs and services | private.goo | +| | regardless of whether they are | gleapis.com | +| 199.36.153.8/30 | supported by VPC Service | to access | +| | Controls. Includes API access | Google APIs | +| 2600 | to Google Maps, Google Ads, | and | +| :2d00:0002:2000::/64 | Google Cloud, and most other | services by | +| | Google APIs, including the | using a set | +| | following list. Does not | of IP | +| | support Google Workspace web | addresses | +| | applications such as Gmail and | only | +| | Google Docs. Does not support | routable | +| | any interactive websites. | from within | +| | | Google | +| | Domain names that match: | Cloud. | +| | | | +| | - accounts.google.com (only | Choose | +| | the paths needed for OAuth | private.goo | +| | authentication) | gleapis.com | +| | | under these | +| | - \*.aiplat | cir | +| | form-notebook.cloud.google.com | cumstances: | +| | | | +| | - \*.aiplatform- | - You | +| | notebook.googleusercontent.com | don\'t | +| | | use VPC | +| | - appengine.google.com | Service | +| | | | +| | - \*.appspot.com | Controls. | +| | | | +| | - | - You do | +| | \*.backupdr.cloud.google.com | use VPC | +| | | Service | +| | - backupdr.cloud.google.com | | +| | | Controls, | +| | - \*. | but you | +| | backupdr.googleusercontent.com | also | +| | | need to | +| | - | access | +| | backupdr.googleusercontent.com | Google | +| | | APIs | +| | - \*.cloudfunctions.net | and | +| | | | +| | - \*.cloudproxy.app | services | +| | | that | +| | - | are not | +| | \*.composer.cloud.google.com | | +| | | supported | +| | - \*. | by VPC | +| | composer.googleusercontent.com | Service | +| | | Cont | +| | - | rols.[^1^]( | +| | \*.datafusion.cloud.google.com | https://clo | +| | | ud.google.c | +| | - \*.da | om/vpc/docs | +| | tafusion.googleusercontent.com | /configure- | +| | | private-goo | +| | - | gle-access# | +| | \*.dataproc.cloud.google.com | footnote-1) | +| | | | +| | - dataproc.cloud.google.com | | +| | | | +| | - \*. | | +| | dataproc.googleusercontent.com | | +| | | | +| | - | | +| | dataproc.googleusercontent.com | | +| | | | +| | - dl.google.com | | +| | | | +| | - gcr.io or \*.gcr.io | | +| | | | +| | - \*.googleapis.com | | +| | | | +| | - \*.gstatic.com | | +| | | | +| | - \*.ltsapis.goog | | +| | | | +| | - | | +| | \*.notebooks.cloud.google.com | | +| | | | +| | - \*.n | | +| | otebooks.googleusercontent.com | | +| | | | +| | - packages.cloud.google.com | | +| | | | +| | - pkg.dev or \*.pkg.dev | | +| | | | +| | - pki.goog or \*.pki.goog | | +| | | | +| | - \*.run.app | | +| | | | +| | - | | +| | source.developers.google.com | | ++----------------------+--------------------------------+-------------+