diff --git a/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/README.md b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/README.md index efc12a4a..fcab43f9 100644 --- a/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/README.md +++ b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/README.md @@ -3,7 +3,7 @@ # Conversion ```script -pandoc --extract-media=images/dns GCP\ Cloud\ DNS\ Service\ Architecture.docx -o dns.md -pandoc --extract-media=images/networking GCP\ Networking\ Solution\ Architecture.docx -o networking.md -pandoc --extract-media=images/private-service GCP\ Private\ Service\ Connect\ Service\ Design.docx -o private-service.md +pandoc --to gfm --extract-media=images/dns GCP\ Cloud\ DNS\ Service\ Architecture.docx -o dns.md +pandoc --to gfm --extract-media=images/networking GCP\ Networking\ Solution\ Architecture.docx -o networking.md +pandoc --to gfm --extract-media=images/private-service GCP\ Private\ Service\ Connect\ Service\ Design.docx -o private-service.md ``` diff --git a/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/convert.sh b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/convert.sh new file mode 100755 index 00000000..bb7fdcd3 --- /dev/null +++ b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/convert.sh @@ -0,0 +1,6 @@ +#!/bin/bash + +pandoc --to gfm --extract-media=images/dns GCP\ Cloud\ DNS\ Service\ Architecture.docx -o dns.md +pandoc --to gfm --extract-media=images/networking GCP\ Networking\ Solution\ Architecture.docx -o networking.md +pandoc --to gfm --extract-media=images/private-service GCP\ Private\ Service\ Connect\ Service\ Design.docx -o private-service.md + diff --git a/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/dns.md b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/dns.md index 80688c0b..6b61562e 100644 --- a/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/dns.md +++ b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/dns.md @@ -6,11 +6,11 @@ **Service Architecture** -**\ +** GCP Cloud DNS** -![](images/dns/media/image1.png){width="6.5in" -height="1.6979166666666667in"} + 1 Revision History [1](#revision-history) @@ -94,23 +94,15 @@ List of Table # Revision History -[]{#_Toc157520309 .anchor}Table 1 Revision History +Table 1 Revision History - ----------------------------------------------------------------------------- - Version Date Author Description - --------- ------------ ------------ ----------------------------------------- - 0.01 8/3/2023 Ethan Rowe Initial template draft - - 0.02 10/11/2023 Michael Draft content - Willetts - - 0.03 12/13/2023 Ethan Rowe Updated DNS Suffix to csp3 and best - practices link. - - 1.00 1/30/2024 Ethan Rowe Baseline inf/core #426 - - - ----------------------------------------------------------------------------- +| Version | Date | Author | Description | +|---------|------------|------------------|-----------------------------------------------------| +| 0.01 | 8/3/2023 | Ethan Rowe | Initial template draft | +| 0.02 | 10/11/2023 | Michael Willetts | Draft content | +| 0.03 | 12/13/2023 | Ethan Rowe | Updated DNS Suffix to csp3 and best practices link. | +| 1.00 | 1/30/2024 | Ethan Rowe | Baseline inf/core \#426 | +| | | | | # Overview @@ -180,7 +172,7 @@ the DNS server. ## Value -GCP's Cloud DNS provides highly available DNS services. Cloud DNS is a +GCP’s Cloud DNS provides highly available DNS services. Cloud DNS is a managed service and simplifies the cost and complexity of managing an equivalent server footprint. Cloud DNS permits name registration and resolution of census.gov assets from within GCP. @@ -194,71 +186,68 @@ web service. You can use Cloud DNS to perform three main functions in any combination: domain registration, DNS resolution & routing, and health checking. -- **Highly Reliable:** Cloud DNS is provided as a managed service from - GCP. There is no management of servers or activities other than - configuration necessary to utilize Cloud DNS. +- **Highly Reliable:** Cloud DNS is provided as a managed service from + GCP. There is no management of servers or activities other than + configuration necessary to utilize Cloud DNS. -- **Scalable:** Cloud DNS automatically scales to handle large traffic - spikes and can be configured to handle DNS for GCP assets, as well - as on-prem and even public if need. +- **Scalable:** Cloud DNS automatically scales to handle large traffic + spikes and can be configured to handle DNS for GCP assets, as well as + on-prem and even public if need. -- **Secure:** Cloud DNS is integrated with IAM, the access to Cloud - DNS is secured by giving its permissions to only the authorized - users. +- **Secure:** Cloud DNS is integrated with IAM, the access to Cloud DNS + is secured by giving its permissions to only the authorized users. -- **Integrated:** Cloud DNS can be used to map domain names to GCP - resources including Cloud Storage Buckets, load balancers, and - virtual machines. +- **Integrated:** Cloud DNS can be used to map domain names to GCP + resources including Cloud Storage Buckets, load balancers, and virtual + machines. -```{=html} -``` -- **Hybrid DNS:** Supports private DNS that spans on-prem and other - cloud environments. -- **Traffic Routing:** According to the geolocation, latency, health, - and other factors, directs traffic to the optimal endpoint - available. +- **Hybrid DNS:** Supports private DNS that spans on-prem and other + cloud environments. + +- **Traffic Routing:** According to the geolocation, latency, health, + and other factors, directs traffic to the optimal endpoint available. ### Requirements -- All Cloud DNS Zones forwards requests to Census Enterprise On-Prem - Infoblox for name resolution of resources in the applicable domain. +- All Cloud DNS Zones forwards requests to Census Enterprise On-Prem + Infoblox for name resolution of resources in the applicable domain. -- VPC Cloud DNS resolves the local VPC domain and the Shared Service - VPC Prod domain. +- VPC Cloud DNS resolves the local VPC domain and the Shared Service VPC + Prod domain. -- Cloud DNS resolves DNS queries of GCP domains from Census on-prem - and AWS networks. +- Cloud DNS resolves DNS queries of GCP domains from Census on-prem and + AWS networks. -- Route tables must advertise the Cloud DNS source range through the - VPN link for hybrid DNS to function correctly. +- Route tables must advertise the Cloud DNS source range through the VPN + link for hybrid DNS to function correctly. -- Cloud DNS endpoints must register in the Census Enterprise Infoblox. +- Cloud DNS endpoints must register in the Census Enterprise Infoblox. ## Assumptions -- DNS domain for GCP is approved. +- DNS domain for GCP is approved. -- DNS subdomain of GCP is approved for each VPC. +- DNS subdomain of GCP is approved for each VPC. ## Constraints -- Route tables must advertise the Cloud DNS source range through the - VPN link for hybrid DNS to function correctly. +- Route tables must advertise the Cloud DNS source range through the VPN + link for hybrid DNS to function correctly. -- Full DNS resolution among all Cloud DNS managed domains require a - full mesh implementation of peering zones. A design decision was - made to only peer with VPC's to provide routing from an originating - VPC. This design was reviewed by Google. +- Full DNS resolution among all Cloud DNS managed domains require a full + mesh implementation of peering zones. A design decision was made to + only peer with VPC’s to provide routing from an originating VPC. This + design was reviewed by Google. ## Hybrid DNS -![Diagram Description automatically -generated](images/dns/media/image2.png){width="6.5in" -height="3.077777777777778in"} + -[]{#_Toc157520308 .anchor}Figure DNS logical design +Figure DNS logical design The GCP Design leverages a custom domain name, \*.csp3.census.gov, with subdomains managed for each VPC. The GCP Cloud DNS design centralizes @@ -278,22 +267,20 @@ each VPC to handle outbound queries to on-prem. The following points cover the traffic flow for how DNS resolution is achieved for inbound and outbound queries to on-prem/ Infoblox. -- **Outbound DNS Resolution** --- GCP Cloud DNS utilizes forwarding - zones registered in each VPC to register the DNS servers used for - external resolution. +- **Outbound DNS Resolution** — GCP Cloud DNS utilizes forwarding zones + registered in each VPC to register the DNS servers used for external + resolution. -```{=html} -``` -- **Inbound DNS resolution** -- An inbound server policy created in - the Prod Hub VPC to receive inbound DNS resolution requests. Each - GCP VPC Cloud DNS zone has a peering zone created in the Prod Hub - account to allow for resolution of any GCP resource from external - requests. -- **GCP VPC DNS resolution** -- Services utilize the standard GCP - metadata DNS resolution order to query Cloud DNS. Forwarding Zones - are created to direct queries for zones external to Cloud DNS. +- **Inbound DNS resolution** – An inbound server policy created in the + Prod Hub VPC to receive inbound DNS resolution requests. Each GCP VPC + Cloud DNS zone has a peering zone created in the Prod Hub account to + allow for resolution of any GCP resource from external requests. + +- **GCP VPC DNS resolution** – Services utilize the standard GCP + metadata DNS resolution order to query Cloud DNS. Forwarding Zones are + created to direct queries for zones external to Cloud DNS. ## Interfaces @@ -333,12 +320,11 @@ customer. ## *Roles and Responsibilities* -- GCP (Cloud Service Provider) -- Maintain, patch, and update Cloud - DNS. +- GCP (Cloud Service Provider) – Maintain, patch, and update Cloud DNS. -- Cloud Engineering Team -- Develop and maintain the DNS Design and - IaC to leverage Cloud DNS. Utilize Infrastructure as Code to - provision DNS records for GCP resources. +- Cloud Engineering Team – Develop and maintain the DNS Design and IaC + to leverage Cloud DNS. Utilize Infrastructure as Code to provision DNS + records for GCP resources. ## Service Limits & Capacity Planning @@ -360,29 +346,21 @@ number of queries and the number of managed zones maintained. Pricing should always be checked against GCP documentation for the latest. As of this writing, pricing and examples are provided in the tables below. -[]{#_Toc157520310 .anchor}Table Query Pricing - - ----------------------------------------------------------------------- - Number of Queries Regular Queries Routing Policy Queries - ----------------------- ----------------------- ----------------------- - 0-1 Billion \$0.40/million per \$0.70/million per - month month - - Over 1 Billion \$0.20/million per \$0.35/million per - month month - ----------------------------------------------------------------------- - -[]{#_Toc157520311 .anchor}Table Managed Zone Pricing +Table Query Pricing - ----------------------------------------------------------------------- - Managed Zones Price - ----------------------------------- ----------------------------------- - 0-25 \$0.20/zone per month +| Number of Queries | Regular Queries | Routing Policy Queries | +|-------------------|--------------------------|--------------------------| +| 0-1 Billion | \$0.40/million per month | \$0.70/million per month | +| Over 1 Billion | \$0.20/million per month | \$0.35/million per month | - 26-10,000 \$0.10/zone per month +Table Managed Zone +Pricing - Over 10,000 \$0.03/zone per month - ----------------------------------------------------------------------- +| Managed Zones | Price | +|---------------|-----------------------| +| 0-25 | \$0.20/zone per month | +| 26-10,000 | \$0.10/zone per month | +| Over 10,000 | \$0.03/zone per month | # Backup and Recovery @@ -423,17 +401,16 @@ Standards](https://uscensus.sharepoint.com/:f:/s/DITDSIRS/EpbVeuUQbE1Ftjo-SJpUjj ## Links -- [Google Cloud DNS SLA](https://cloud.google.com/dns/sla) +- [Google Cloud DNS SLA](https://cloud.google.com/dns/sla) -- [Current Cloud DNS Pricing](https://cloud.google.com/dns/pricing) +- [Current Cloud DNS Pricing](https://cloud.google.com/dns/pricing) -- [Cloud DNS Best - Practices](https://cloud.google.com/dns/docs/best-practices#reference_architectures_for_hybrid_dns) +- [Cloud DNS Best + Practices](https://cloud.google.com/dns/docs/best-practices#reference_architectures_for_hybrid_dns) -- [GCP Cloud DNS Quotas and - Limits](https://cloud.google.com/dns/quotas) +- [GCP Cloud DNS Quotas and Limits](https://cloud.google.com/dns/quotas) -- [GCP Operations - Plan](https://uscensus.sharepoint.com/:w:/s/DITDSIRS/EfHiSwwk1mFCtyuXFnCKGoMBjV158-8VEfxVUCVI37D3lQ?e=qordeA) +- [GCP Operations + Plan](https://uscensus.sharepoint.com/:w:/s/DITDSIRS/EfHiSwwk1mFCtyuXFnCKGoMBjV158-8VEfxVUCVI37D3lQ?e=qordeA) [^1]: GCP Block from Cloud DNS. diff --git a/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/networking.md b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/networking.md index 80820c83..8708c99d 100644 --- a/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/networking.md +++ b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/networking.md @@ -6,8 +6,8 @@ **GCP Networking Design** -![](images/networking/media/image1.png){width="6.5in" -height="1.70625in"} + 1 Revision History [1](#revision-history) @@ -130,31 +130,19 @@ List of Table # Revision History -[]{#_Toc157695672 .anchor}Table 1 Revision History +Table 1 Revision History - -------------------------------------------------------------------------------- - **Version** **Date** **Description** - ------------- ------------ ----------------------------------------------------- - 0.01 8/3/2023 Initial template draft - - 0.02 9/27/2023 Draft content for Network Design - - 0.03 10/3/2023 Broad content update for Review and Comment - - 0.04 10/11/2023 Peer reviewed. Removed references to AWS. Content - rewrite to focus on design decisions. Comments added. - - 0.05 12/15/2023 Grammar Updates - - 1.0 12/18/2023 Updates from CSvD review - - 1.01 1/5/2024 Update Logical Design drawing and CIDR Table -- - Michael Jones - - 1.02 1/31/2024 Baseline core/inf #426 - - 1.03 3/28/2024 Update shared vpc subnet layout - -------------------------------------------------------------------------------- +| **Version** | **Date** | **Description** | +|-------------|------------|---------------------------------------------------------------------------------------------------------| +| 0.01 | 8/3/2023 | Initial template draft | +| 0.02 | 9/27/2023 | Draft content for Network Design | +| 0.03 | 10/3/2023 | Broad content update for Review and Comment | +| 0.04 | 10/11/2023 | Peer reviewed. Removed references to AWS. Content rewrite to focus on design decisions. Comments added. | +| 0.05 | 12/15/2023 | Grammar Updates | +| 1.0 | 12/18/2023 | Updates from CSvD review | +| 1.01 | 1/5/2024 | Update Logical Design drawing and CIDR Table – Michael Jones | +| 1.02 | 1/31/2024 | Baseline core/inf \#426 | +| 1.03 | 3/28/2024 | Update shared vpc subnet layout | # Overview @@ -187,15 +175,15 @@ Product Owner and/or Technical Lead and approved by the CGB. ## Terms and Definitions -**Host Project** --- Equivalent of an Account in Amazon Web Services -(AWS) where Shared VPCs, VPN connections, and core networking resources -are defined and provisioned. +**Host Project** — Equivalent of an Account in Amazon Web Services (AWS) +where Shared VPCs, VPN connections, and core networking resources are +defined and provisioned. -**Service Project** --- A Service project resides outside the host -project but subscribes to the host project's network services (i.e., -Shared VPC). +**Service Project** — A Service project resides outside the host project +but subscribes to the host project’s network services (i.e., Shared +VPC). -**Shared VPC** --- A Shared VPC allows an organization to connect +**Shared VPC** — A Shared VPC allows an organization to connect resources from multiple projects to a common Virtual Private Cloud (VPC) network to communicate with each other securely and efficiently by using internal IP addresses from that network. @@ -207,24 +195,24 @@ internal IP addresses from that network. The USCB GCP Network Design provides reliable, scalable, and secure communication services to GCP hosted resources. -- **Highly Secure Connectivity --** Traditional networking solutions - require complex configurations to establish security and access - controls. To meet privacy and security requirements, GCP VPCs are - configured with advanced security features including Network - Firewalls and hierarchical firewall policies for performing inbound - and outbound filtering. GCP Private Service Connect is used to - establish secure network connections between VPCs and GCP services - by delivering traffic on the GCP backbone network. - -- **Scalable Networking --** GCP Hub VPC is utilized for centralized - management, monitoring, and routing of all bi-directional traffic - for each VPC and VPN. GCP Hub VPCs connect VPCs and USCB on-premises - networks networking by peering connections to a Shared VPC and VPN - connections for hybrid connectivity. - -- **Monitored --** VPC and Host VPC traffic flows are monitored for - security and network anomalies and assist with troubleshooting using - a Palo Alto IDS managed by OIS. +- **Highly Secure Connectivity –** Traditional networking solutions + require complex configurations to establish security and access + controls. To meet privacy and security requirements, GCP VPCs are + configured with advanced security features including Network Firewalls + and hierarchical firewall policies for performing inbound and outbound + filtering. GCP Private Service Connect is used to establish secure + network connections between VPCs and GCP services by delivering + traffic on the GCP backbone network. + +- **Scalable Networking –** GCP Hub VPC is utilized for centralized + management, monitoring, and routing of all bi-directional traffic for + each VPC and VPN. GCP Hub VPCs connect VPCs and USCB on-premises + networks networking by peering connections to a Shared VPC and VPN + connections for hybrid connectivity. + +- **Monitored –** VPC and Host VPC traffic flows are monitored for + security and network anomalies and assist with troubleshooting using a + Palo Alto IDS managed by OIS. ## Capabilities, Features, and Requirements @@ -234,58 +222,56 @@ several GCP Network services including VPC, Cloud Router, Cloud HA VPN, Cloud DNS, and Private Service Connect. The requirements for this design include: -- Network Connectivity in the cloud among applications, systems, and - services. +- Network Connectivity in the cloud among applications, systems, and + services. -- Network Connectivity to the Census on-prem environment. +- Network Connectivity to the Census on-prem environment. -- Provide network segmentation among environments. +- Provide network segmentation among environments. -- Secure, encrypted connections over any public interfaces or - circuits. +- Secure, encrypted connections over any public interfaces or circuits. -- Ability to restrict and permit cloud communication with TCP/IP based - firewalls and access control lists. +- Ability to restrict and permit cloud communication with TCP/IP based + firewalls and access control lists. -- To the greatest extent possible, perform cost optimization and - complexity reduction by leveraging the capability to share cloud - resources. +- To the greatest extent possible, perform cost optimization and + complexity reduction by leveraging the capability to share cloud + resources. -- High availability, fault tolerance, and resilience through redundant - networking. +- High availability, fault tolerance, and resilience through redundant + networking. -- Integration with GCP IAM to control privileged access to cloud - networking resources and to support security compliance, auditing, - and logging. +- Integration with GCP IAM to control privileged access to cloud + networking resources and to support security compliance, auditing, and + logging. ## Assumptions -- Connectivity to the Census on-prem environment is required. +- Connectivity to the Census on-prem environment is required. -- Routing to AWS networks is outside the scope at the time of this - writing. +- Routing to AWS networks is outside the scope at the time of this + writing. -```{=html} -``` -- IPv6 integration is out of scope. IPv6 addressing is allocated and - reserved for future integration when required by TCO. + +- IPv6 integration is out of scope. IPv6 addressing is allocated and + reserved for future integration when required by TCO. ## Constraints -- Access to GCP Networking resources is provided by GCP Cloud - Identity. Cloud Identity provides support for permission sets, - users, roles, groups, and policies required to manage and administer - GCP Networking services. +- Access to GCP Networking resources is provided by GCP Cloud Identity. + Cloud Identity provides support for permission sets, users, roles, + groups, and policies required to manage and administer GCP Networking + services. -- Due to the VRF design of the on-premises networking, it does not - permit the use of a centralized hub for reducing cost and complexity - of GCP to Enterprise connectivity. +- Due to the VRF design of the on-premises networking, it does not + permit the use of a centralized hub for reducing cost and complexity + of GCP to Enterprise connectivity. -- Due to the lack of segmentation of test workloads on-prem, - end-to-end segmentation is not possible and a single VRF is utilized - for all test workloads including, but not limited to: QA, Project - Test, Integration Testing, and Test. +- Due to the lack of segmentation of test workloads on-prem, end-to-end + segmentation is not possible and a single VRF is utilized for all test + workloads including, but not limited to: QA, Project Test, Integration + Testing, and Test. ## Conceptual Design @@ -296,15 +282,19 @@ Connectivity to Census on-prem environments uses GCP HA VPN connections to redundant locations. VPC peering connections are used to interconnect the Shared VPCs in the host project to the service projects. -![](images/networking/media/image2.png){width="6.5in" height="2.9125in"} + -[]{#_Toc157695668 .anchor}Figure 1 GCP Conceptual Networking Design +Figure 1 GCP Conceptual +Networking Design ## Logical Design -![](images/networking/media/image3.png){width="5.0in" height="4.0in"} + -[]{#_Toc157695669 .anchor}Figure 2 Logical GCP Network Design +Figure 2 Logical GCP +Network Design ### GCP Cloud Router/ Cloud HA VPN Design @@ -323,15 +313,15 @@ environment that that terminate on Cisco Cloud routers via a connection over the Internet. Initially, the minimum number of tunnels are listed below but may be increased as required. -- Services -- 4 VPN tunnels +- Services – 4 VPN tunnels -- Dev --4 VPN tunnels +- Dev –4 VPN tunnels -- Test -- 4 VPN tunnels +- Test – 4 VPN tunnels -- Stage -- 4 VPN tunnels +- Stage – 4 VPN tunnels -- Prod -- 4 VPN tunnels +- Prod – 4 VPN tunnels The VPN tunnels terminate on routers at two Census locations: Bowie Computer Center (BCC) and Census Headquarters (HQ). The traffic flowing @@ -347,23 +337,23 @@ Traffic over the VPN tunnels utilizes HQ as the preferred route. Within the GCP organization, we designate projects as Shared VPC host projects for each of the environments shown below: -- Dev +- Dev -- Test +- Test -- ITE +- ITE -- Stage +- Stage -- Prod +- Prod -- Non-routable Lab +- Non-routable Lab -- Shared Services-Dev +- Shared Services-Dev -- Shared Services-Test +- Shared Services-Test -- Shared Services-Prod +- Shared Services-Prod The GCP organization uses Shared VPCs in common host projects. Application and systems teams from peered service projects in the @@ -379,15 +369,15 @@ services VPC, are peered to all environmental shared VPCs. Subnet routes for each peered VPC are automatically exchanged over the network peering connection. There are a few important caveats about VPC Network Peering: -- Resources in a peered VPC network cannot use DNS names created by a - local VPC network. A peered VPC network can\'t use Cloud DNS managed - private zones that are authorized for only a local VPC network, so - we deploy DNS cloud peering zones to address this issue. +- Resources in a peered VPC network cannot use DNS names created by a + local VPC network. A peered VPC network can't use Cloud DNS managed + private zones that are authorized for only a local VPC network, so we + deploy DNS cloud peering zones to address this issue. -- VPC Network Peering does not exchange any VPC firewall rules. VPC - firewall rules in one VPC network can\'t specify targets or sources - using network tags or service accounts from the other VPC network. - However, the same target network tag can be used in both networks. +- VPC Network Peering does not exchange any VPC firewall rules. VPC + firewall rules in one VPC network can't specify targets or sources + using network tags or service accounts from the other VPC network. + However, the same target network tag can be used in both networks. ### Shared VPC Design @@ -402,18 +392,19 @@ GCP, we will create new service projects, and all resources created will support connectivity via the subnets available from Shared VPC, as shown in Figure 3 Shared VPC Design. -![](images/networking/media/image4.png){width="4.210416666666666in" -height="5.375in"} + -[]{#_Ref153543324 .anchor}Figure 3 Shared VPC Design +Figure 3 Shared VPC +Design ### VPC and Subnet CIDR Allocation The GCP network is allocated a /16 CIDR block for the GCP network: -- IPv4: 10.36.0.0/16 GCP Cloud Internal +- IPv4: 10.36.0.0/16 GCP Cloud Internal -- IPv6: 2610:20:2061::/48 GCP Cloud +- IPv6: 2610:20:2061::/48 GCP Cloud **Note:** Cloud-native applications are typically dynamic and require sufficient IP space to support auto-scaling. Shared VPCs are intended @@ -433,27 +424,18 @@ allocation of addresses, see the GCP CIDR Applications link in the reference section of this document. At the time of this writing, a general VPC Subnet allocation for each environment is shown below: -[]{#_Toc157695673 .anchor}Table 2 VPC Subnets - - ----------------------------------------------------------------------- - Subnets Mask per AZ Total Addresses - --------------------------------- --------------------- --------------- - App /22 1024 +Table 2 VPC Subnets - Load Balancer /24 256 - - Load Balancer Regional Proxy /24 256 - - SPA-Functions-1 /28 16 - - SPA-CloudRun-1 /28 16 - - - - - - - ----------------------------------------------------------------------- +| Subnets | Mask per AZ | Total Addresses | +|------------------------------|-------------|-----------------| +| App | /22 | 1024 | +| Load Balancer | /24 | 256 | +| Load Balancer Regional Proxy | /24 | 256 | +| SPA-Functions-1 | /28 | 16 | +| SPA-CloudRun-1 | /28 | 16 | +| | | | +| | | | +| | | | ### Private Service Connect Design @@ -471,11 +453,11 @@ environment (Dev, Test, Prod, etc.). The following figure depicts resources accessing a VPC Private Service Connect to reach Google services over the Google backbone. -![](images/networking/media/image5.png){width="6.401388888888889in" -height="4.090972222222222in"} + -[]{#_Toc157695671 .anchor}Figure 4 Example VPC Private Service -Connection to GCP services +Figure 4 Example VPC +Private Service Connection to GCP services ## Interfaces @@ -498,28 +480,28 @@ using infrastructure as Code. ## Key Performance Indicators No KPIs are defined at this time. There are a variety of USCB standard, -GCP-native, and 3^rd^ party tools that will be used to measure +GCP-native, and 3rd party tools that will be used to measure performance on the GCP Network including Cloud Monitoring, Ping, iPerf3, and Solarwinds. The GCP Network key performance indicators for VPN, VPC and Shared VPC traffic on the network are: -- VPN Connection Status +- VPN Connection Status -- Inbound Packet per Second +- Inbound Packet per Second -- Outbound Packets per Second +- Outbound Packets per Second -- Inbound Bytes per Second +- Inbound Bytes per Second -- Outbound Bytes per Second +- Outbound Bytes per Second -- Throughput +- Throughput -- Round-Trip Time (RTT) +- Round-Trip Time (RTT) -- Retransmission Rate +- Retransmission Rate -- Window Size +- Window Size These metrics and associated monitoring activities are detailed in the USCB Enterprise GCP Operating Guide. @@ -536,20 +518,10 @@ responsibility of Google. ## Roles and Responsibilities - ----------------------------------------------------------------------------------- - Role - ---------------------- ----------------- ------------------------------------------ - NetworkAdministrator gr-net.admin Requires admin level permissions to - create, modify and delete GCP cloud - networking resources including VPCs, - Shared VPC, Cloud DNS, VPN Connections, - Cloud Logging, and similar services. - - NetworkSupport gr-net.operator Permissions to read, describe and list GCP - cloud networking resources including VPCs, - Shared VPC, Cloud DNS, VPN Connections, - Cloud Monitoring, and similar services. - ----------------------------------------------------------------------------------- +| Role | | | +|----------------------|-----------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| NetworkAdministrator | gr-net.admin | Requires admin level permissions to create, modify and delete GCP cloud networking resources including VPCs, Shared VPC, Cloud DNS, VPN Connections, Cloud Logging, and similar services. | +| NetworkSupport | gr-net.operator | Permissions to read, describe and list GCP cloud networking resources including VPCs, Shared VPC, Cloud DNS, VPN Connections, Cloud Monitoring, and similar services. | ## Service Limits & Capacity Planning @@ -557,104 +529,58 @@ At the time of this writing, the following tables describe the current service limits. Refer to official Google Cloud Platform documentation when planning any future use cases. -[]{#_Toc157695674 .anchor}Table 3 GCP VPC Service Limits - - ----------------------------------------------------------------------------------------------------- - **Shared VPC limits** - ---------------------------- -------------- --------------------------------------------------------- - The number of service - projects that can be - attached to a host project - is a configurable - per-project quota. - - **Item** **Limit** **Notes** - - Number of Shared VPC host 100 [To request an update to this limit, file a support - projects in a single case.](https://cloud.google.com/support-hub) - organization - - Number of host projects to 1 This limit cannot be increased. - which a service project can - attach - - - - **Per network** - - The following limits apply - to VPC networks. - - **Item** **Limit** **Notes** - - [Subnet IP - ranges]{.underline} - - Primary IP ranges per subnet 1 [Each subnet must have exactly one primary IP range (CIDR - block). This range is used for VM primary internal IP - addresses, VM alias IP ranges, and the IP addresses of - internal load balancers. This limit cannot be - increased.](https://cloud.google.com/vpc/docs/alias-ip) - - Maximum number of secondary 30 Optionally, you can define up to thirty secondary CIDR - IP ranges per subnet blocks per subnet. These secondary IP ranges can only be - used for alias IP ranges. This limit cannot be increased. - - [Routes]{.underline} - - Maximum number of network 256 The maximum number of network tags that you can associate - tags per route with a static route. This limit cannot be increased. - - - - **VPC Network Peering - limits** - - **Item** **Peering **Notes** - group limit** - - [Peering group]{.underline} - - Maximum number of 25 The maximum number of networks that can connect to a - connections to a single VPC given VPC network using VPC Network Peering. - network - ----------------------------------------------------------------------------------------------------- +Table 3 GCP VPC Service +Limits + +| **Shared VPC limits** | | | +|------------------------------------------------------------------------------------------------------------|-------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| The number of service projects that can be attached to a host project is a configurable per-project quota. | | | +| **Item** | **Limit** | **Notes** | +| Number of Shared VPC host projects in a single organization | 100 | [To request an update to this limit, file a support case.](https://cloud.google.com/support-hub) | +| Number of host projects to which a service project can attach | 1 | This limit cannot be increased. | +| | | | +| **Per network** | | | +| The following limits apply to VPC networks. | | | +| **Item** | **Limit** | **Notes** | +| Subnet IP ranges | | | +| Primary IP ranges per subnet | 1 | [Each subnet must have exactly one primary IP range (CIDR block). This range is used for VM primary internal IP addresses, VM alias IP ranges, and the IP addresses of internal load balancers. This limit cannot be increased.](https://cloud.google.com/vpc/docs/alias-ip) | +| Maximum number of secondary IP ranges per subnet | 30 | Optionally, you can define up to thirty secondary CIDR blocks per subnet. These secondary IP ranges can only be used for alias IP ranges. This limit cannot be increased. | +| Routes | | | +| Maximum number of network tags per route | 256 | The maximum number of network tags that you can associate with a static route. This limit cannot be increased. | +| | | | +| **VPC Network Peering limits** | | | +| **Item** | **Peering group limit** | **Notes** | +| Peering group | | | +| Maximum number of connections to a single VPC network | 25 | The maximum number of networks that can connect to a given VPC network using VPC Network Peering. | The following limits apply to Cloud VPN. In this table, *VPN tunnel* means either a Classic VPN tunnel or an HA VPN tunnel. Unless otherwise stated, these limits cannot be increased. -[]{#_Toc157695675 .anchor}Table 4 Cloud VPN Limits - - --------------------------------------------------------------------------------------------------------------------------------------------------------- - Item Limit Notes - ----------------------- ----------------------- --------------------------------------------------------------------------------------------------------- - Bandwidth per VPN 250,000 packets per 250,000 packets per second is roughly equivalent to 1 Gbps to 3 Gbps, depending on the average packet - tunnel second for the sum of size within the tunnel. - ingress and egress +Table 4 Cloud VPN Limits - Cloud VPN only throttles egress IPsec traffic. It does not throttle ingress traffic. - - [For more details, see Network - bandwidth.](https://cloud.google.com/network-connectivity/docs/vpn/concepts/overview#network-bandwidth) - --------------------------------------------------------------------------------------------------------------------------------------------------------- +| Item | Limit | Notes | +|--------------------------|--------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------| +| Bandwidth per VPN tunnel | 250,000 packets per second for the sum of ingress and egress | 250,000 packets per second is roughly equivalent to 1 Gbps to 3 Gbps, depending on the average packet size within the tunnel. | +| | | Cloud VPN only throttles egress IPsec traffic. It does not throttle ingress traffic. | +| | | [For more details, see Network bandwidth.](https://cloud.google.com/network-connectivity/docs/vpn/concepts/overview#network-bandwidth) | ## Consumption, Licensing, and Assets -- VPC hourly charge - Customers are charged for GCP VPC on an hourly - basis. +- VPC hourly charge - Customers are charged for GCP VPC on an hourly + basis. -- VPC data processing charge - Data Processing is charged to the VPC - owner who sends outbound traffic to a VPC. +- VPC data processing charge - Data Processing is charged to the VPC + owner who sends outbound traffic to a VPC. -- VPC data processing charge for outbound inter-Region peering - attachments - Inbound inter-Region data transfer charges are free. +- VPC data processing charge for outbound inter-Region peering + attachments - Inbound inter-Region data transfer charges are free. -- Private Service Connect +- Private Service Connect -- Cloud DNS +- Cloud DNS -- Cloud HA VPN +- Cloud HA VPN # Cost Consideration @@ -665,65 +591,84 @@ Google Published pricing data and confer with USCB FinOps administrators to forecast and model pricing for ongoing consumption of GCP resources. All assets are tagged and tracked in Apptio Cloudability. -[]{#_Ref153791171 .anchor}Table 5 Virtual Private Cloud Pricing - -![](images/networking/media/image6.png){width="6.178844050743657in" -height="4.054864391951006in"} - -[]{#_Ref153791180 .anchor}Table 6 Cloud VPN - -+---------------------------+------------------------------------------+ -| Item | Price per hour (USD) | -+===========================+==========================================+ -| Hourly charge for each | \$0.055 | -| tunnel attached to the | | -| gateway.\ | | -| \ | | -| HA VPN only: For 99.99% | | -| availability, you must | | -| configure two tunnels. | | -+---------------------------+------------------------------------------+ -| IPsec traffic | You are charged as follows: | -| | | -| | If the Cloud VPN tunnel connects to | -| | another Cloud VPN gateway, you are | -| | charged egress pricing as described in | -| | VM-VM egress pricing within Google | -| | Cloud. Egress pricing is based on the IP | -| | addresses of the destination VPN | -| | gateway---not the destination VM | -| | address.\ | -| | \ | -| | If the source and destination Cloud VPN | -| | gateways are in the same Google Cloud | -| | region, egress traffic is billed as | -| | traffic between zones in the same | -| | region. | -| | | -| | If the Cloud VPN tunnel connects to a | -| | VPN gateway outside of Google Cloud, you | -| | are charged as described in Internet | -| | egress rates. | -+---------------------------+------------------------------------------+ -| External IP address for | You are charged as described in IP | -| VPN gateway | address pricing.\ | -| | \ | -| | An external IP address is charged only | -| | if it is not being used by a VPN tunnel. | -+---------------------------+------------------------------------------+ - -[]{#_Ref153791215 .anchor}Table 7 Private Service Connect - - --------------------------------------------------------------------------------------------------------------------------------- - Item Price per hour (USD) Price per GiB - processed,\ - both egress and ingress - (USD) - --------------------------------------------------------------------------------- ----------------------- ----------------------- - Private Service Connect endpoint (forwarding rule) used to [access Google \$0.01 No data charge - APIs](https://cloud.google.com/vpc/docs/configure-private-service-connect-apis) - - --------------------------------------------------------------------------------------------------------------------------------- +Table 5 Virtual Private +Cloud Pricing + + + +Table 6 Cloud VPN + + ++++ + + + + + + + + + + + + + + + + + + + + +
ItemPrice per hour (USD)
Hourly charge for each tunnel attached to the gateway.
+
+HA VPN only: For 99.99% availability, you must configure two +tunnels.		$0.055
IPsec traffic

You are charged as follows:

+

If the Cloud VPN tunnel connects to another Cloud VPN gateway, you +are charged egress pricing as described in VM-VM egress pricing within +Google Cloud. Egress pricing is based on the IP addresses of the +destination VPN gateway—not the destination VM address.
+
+If the source and destination Cloud VPN gateways are in the same Google +Cloud region, egress traffic is billed as traffic between zones in the +same region.

+

If the Cloud VPN tunnel connects to a VPN gateway outside of Google +Cloud, you are charged as described in Internet egress rates.

External IP address for VPN gatewayYou are charged as described in IP address pricing.
+
+An external IP address is charged only if it is not being used by a VPN +tunnel.
+ +Table 7 Private Service +Connect + + +++++ + + + + + + + + + + + + + + +
ItemPrice per hour (USD)Price per GiB processed,
+both egress and ingress (USD)
Private Service Connect endpoint (forwarding rule) used to access +Google APIs$0.01No data charge
# Backup and Recovery @@ -733,17 +678,17 @@ communicate, shared data and exchange information. GCP Networking provides the infrastructure and services to support backup and recovery services in the cloud including: -- Global, Multi Region Network +- Global, Multi Region Network -- Multiple Availability Zones +- Multiple Availability Zones -- Autoscaling +- Autoscaling -- Serverless Architecture +- Serverless Architecture -- Load Balancers +- Load Balancers -- Cloud DNS +- Cloud DNS GCP networking configurations for USCB are deployed through Infrastructure as Code. If required, recovery of networking @@ -811,74 +756,73 @@ Standards](https://uscensus.sharepoint.com/:x:/s/DITDATO/EV12IGvaxTJLnkkB-vz7c3I ## Links -- [GCP FedRAMP Services in - Scope](https://aws.amazon.com/compliance/services-in-scope/FedRAMP/) +- [GCP FedRAMP Services in + Scope](https://aws.amazon.com/compliance/services-in-scope/FedRAMP/) -- [Compute Engine Service Level Agreement (SLA) \| Google - Cloud](https://cloud.google.com/compute/sla) +- [Compute Engine Service Level Agreement (SLA) \| Google + Cloud](https://cloud.google.com/compute/sla) -- GCP Best Practices +- GCP Best Practices - - [Shared VPC - Design](https://cloud.google.com/architecture/best-practices-vpc-design#shared-common-vpc) + - [Shared VPC + Design](https://cloud.google.com/architecture/best-practices-vpc-design#shared-common-vpc) - - [Liens](o%09https:/cloud.google.com/vpc/docs/provisioning-shared-vpc#protectsharedvpc) + - [Liens](o%09https:/cloud.google.com/vpc/docs/provisioning-shared-vpc#protectsharedvpc) - - [Accidental Shared VPC - Shutdowns](https://cloud.google.com/billing/docs/how-to/secure-project-billing-account-link) + - [Accidental Shared VPC + Shutdowns](https://cloud.google.com/billing/docs/how-to/secure-project-billing-account-link) - - [Constrain host project - attachments](https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints) + - [Constrain host project + attachments](https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints) - - [Constrain the subnets in the host - project](https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints) + - [Constrain the subnets in the host + project](https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints) -```{=html} -``` -- [GCP Solutions - Architecture](https://uscensus.sharepoint.com/:w:/s/DITDATO/EaDaa4eHgJhPqnt-9FlGq2YBnV_F83HIhbree1TuW219Zg?e=jK1hQv) -- [GCP DNS Detailed - Design](https://uscensus.sharepoint.com/:w:/s/DITDATO/ESnMqy6UOX5CttAk5vaYXL4BSn5JVRPWAwQ_1nRhXkepsw?e=XchWOq) +- [GCP Solutions + Architecture](https://uscensus.sharepoint.com/:w:/s/DITDATO/EaDaa4eHgJhPqnt-9FlGq2YBnV_F83HIhbree1TuW219Zg?e=jK1hQv) + +- [GCP DNS Detailed + Design](https://uscensus.sharepoint.com/:w:/s/DITDATO/ESnMqy6UOX5CttAk5vaYXL4BSn5JVRPWAwQ_1nRhXkepsw?e=XchWOq) -- [GCP Identity Detailed - Design](https://uscensus.sharepoint.com/:w:/s/DITDATO/ESt6lFhB4RhPnLnHtZEX9nABfZLUyrf9Oezy1sRX_ewqAQ?e=sD9sxk) +- [GCP Identity Detailed + Design](https://uscensus.sharepoint.com/:w:/s/DITDATO/ESt6lFhB4RhPnLnHtZEX9nABfZLUyrf9Oezy1sRX_ewqAQ?e=sD9sxk) -- [GCP Operations Suite Cloud Logging Detailed - Design](https://uscensus.sharepoint.com/:w:/s/DITDATO/EYSWYQAs28pLmu2I1T8Gt7ABC3zzy3ikv2XgB2lHLXJZJQ?e=QSDXv0) +- [GCP Operations Suite Cloud Logging Detailed + Design](https://uscensus.sharepoint.com/:w:/s/DITDATO/EYSWYQAs28pLmu2I1T8Gt7ABC3zzy3ikv2XgB2lHLXJZJQ?e=QSDXv0) -- [GCP Standards & - Configurations](https://uscensus.sharepoint.com/:f:/s/DITDATO/EpC2S1BFNCVNqU93LkYHXzsBYFzBXygEJZEDYHJjqmBFbA?e=kNdbl8) +- [GCP Standards & + Configurations](https://uscensus.sharepoint.com/:f:/s/DITDATO/EpC2S1BFNCVNqU93LkYHXzsBYFzBXygEJZEDYHJjqmBFbA?e=kNdbl8) -- [GCP CIDR - Allocation](https://uscensus.sharepoint.com/:x:/s/DITDATO/EeY54Ec_dyFKhW77iTdWlUsBfLUpkoRy_m6ZbwZ-tQit7Q?e=cuQSB4) +- [GCP CIDR + Allocation](https://uscensus.sharepoint.com/:x:/s/DITDATO/EeY54Ec_dyFKhW77iTdWlUsBfLUpkoRy_m6ZbwZ-tQit7Q?e=cuQSB4) -- [GCP Operating - Plan](https://uscensus.sharepoint.com/:w:/s/DITDATO/ETXm6FlSxm5PrqipdIqa1aQB3uIXt8gBvf01KZcd4wl-ow?e=9Q2SKd) +- [GCP Operating + Plan](https://uscensus.sharepoint.com/:w:/s/DITDATO/ETXm6FlSxm5PrqipdIqa1aQB3uIXt8gBvf01KZcd4wl-ow?e=9Q2SKd) ## Shared VPC Best Practices -- Create a VPC network for each autonomous team, with Services in a - common VPC network - - -- Automatically prevent accidental deletion of host projects with a - special lock---called a lien. Liens can also be placed upon a - project automatically. When Identity and Access Management (IAM) - service accounts are allowed from one project to be attached to - resources in other projects, a lien is placed upon the project where - the service accounts are located. - ( - ) - -- Prevent a possible occurrence of an accidental Shared VPC shutdown - due to inactive or disabled billing - () - -- Constrain host project attachments - () - -- Constrain the subnets in the host project that a service project can - use - () +- Create a VPC network for each autonomous team, with Services in a + common VPC network + + +- Automatically prevent accidental deletion of host projects with a + special lock—called a lien. Liens can also be placed upon a project + automatically. When Identity and Access Management (IAM) service + accounts are allowed from one project to be attached to resources in + other projects, a lien is placed upon the project where the service + accounts are located. + ( + ) + +- Prevent a possible occurrence of an accidental Shared VPC shutdown due + to inactive or disabled billing + () + +- Constrain host project attachments + () + +- Constrain the subnets in the host project that a service project can + use + () diff --git a/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/private-service.md b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/private-service.md index 9277173c..a80c7081 100644 --- a/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/private-service.md +++ b/gcp/documentation/02_Solutions_and_Service_Architecture/GCP Networking Designs/private-service.md @@ -1,4 +1,7 @@ -## ![](images/private-service/media/image1.jpeg){width="3.0in" height="2.1041666666666665in"}![](images/private-service/media/image2.png){width="2.7777777777777777in" height="2.0833333333333335in"}![](images/private-service/media/image3.png){width="2.7430555555555554in" height="2.1041666666666665in"} {#section .Alphabetical .unnumbered} +## [0](#_Toc161138597) @@ -80,26 +83,22 @@ List of Figures List of Table -[Table 1 -- Revision History [1](#_Toc125379086)](#_Toc125379086) +[Table 1 – Revision History [1](#_Toc125379086)](#_Toc125379086) -[Table 2 -- Acronyms [4](#_Toc125379087)](#_Toc125379087) +[Table 2 – Acronyms [4](#_Toc125379087)](#_Toc125379087) # Revision History -[]{#_Toc125379086 .anchor}Table 1 -- Revision History +Table 1 – Revision +History - ------------------------------------------------------------------------------- - Version Date Author Description - --------- ------------ ----------- -------------------------------------------- - 0.01 8/3/2023 Ethan Rowe Initial template draft +| Version | Date | Author | Description | +|---------|------------|---------------|-----------------------------------------| +| 0.01 | 8/3/2023 | Ethan Rowe | Initial template draft | +| 0.02 | 03/06/2023 | Nathaniel Ely | Private Service Connect initial content | +| | | | | - 0.02 03/06/2023 Nathaniel Private Service Connect initial content - Ely - - - ------------------------------------------------------------------------------- - - : Table 2 Private Google Access Zone Entry +Table 2 Private Google Access Zone Entry # Overview @@ -139,30 +138,29 @@ Owner and/or Technical Lead and approved by the EWG. ## Terms and Definitions -- **Private Google Access** --- Private Google provides the capability - to access [Google APIs and - services](https://developers.google.com/apis-explorer/#p/) in - Google\'s production infrastructure without traversing the public - internet. +- **Private Google Access** — Private Google provides the capability to + access [Google APIs and + services](https://developers.google.com/apis-explorer/#p/) in Google's + production infrastructure without traversing the public internet. -- **[Private Service Connect - Endpoints](https://cloud.google.com/vpc/docs/private-service-connect#endpoints)** - --- PSC Endpoints are IP addressable resources in GCP VPCs mapped to - the PSC Service, providing connectivity to the Google Cloud APIs. - These are equivalent to an Amazon Web Services (AWS) VPC endpoints. +- **[Private Service Connect + Endpoints](https://cloud.google.com/vpc/docs/private-service-connect#endpoints)** + — PSC Endpoints are IP addressable resources in GCP VPCs mapped to the + PSC Service, providing connectivity to the Google Cloud APIs. These + are equivalent to an Amazon Web Services (AWS) VPC endpoints. -- **Host Project** --- Equivalent of an Account AWS where Shared VPCs, - VPN connections, and core networking resources are defined and - provisioned. +- **Host Project** — Equivalent of an Account AWS where Shared VPCs, VPN + connections, and core networking resources are defined and + provisioned. -- **Service Project** --- A Service project resides outside the host - project but subscribes to the host project's network services (i.e., - Shared VPC). +- **Service Project** — A Service project resides outside the host + project but subscribes to the host project’s network services (i.e., + Shared VPC). -- **Shared VPC** --- A Shared VPC allows an organization to connect - resources from multiple projects to a common Virtual Private Cloud - (VPC) network to communicate with each other securely and - efficiently by using internal IP addresses from that network. +- **Shared VPC** — A Shared VPC allows an organization to connect + resources from multiple projects to a common Virtual Private Cloud + (VPC) network to communicate with each other securely and efficiently + by using internal IP addresses from that network. # Service Design @@ -170,63 +168,62 @@ Owner and/or Technical Lead and approved by the EWG. The USCB GCP Private Service Connect design provides reliable, scalable, and secure communication services to Google APIs from VPCs using -Google's internal network. +Google’s internal network. -- **Secure Connectivity --** Traffic destined to Google APIs are kept - securely within the internal USCB GCP network and do not traverse - the public internet. Traffic communication across the PSC endpoints - is secured using https. +- **Secure Connectivity –** Traffic destined to Google APIs are kept + securely within the internal USCB GCP network and do not traverse the + public internet. Traffic communication across the PSC endpoints is + secured using https. -- **Monitored --** VPC traffic flows are monitored for security and - network anomalies and assist with troubleshooting using a Palo Alto - IDS managed by OIS. +- **Monitored –** VPC traffic flows are monitored for security and + network anomalies and assist with troubleshooting using a Palo Alto + IDS managed by OIS. -- **Scalability --** GCP automatically scales its infrastructure to - meet demand for network traffic to Google APIs from the VPC - endpoint. Service limits apply per VPC, per region. +- **Scalability –** GCP automatically scales its infrastructure to meet + demand for network traffic to Google APIs from the VPC endpoint. + Service limits apply per VPC, per region. ## Capabilities, Features, and Requirements Private Service Connect is a component of the USCB GCP Network design and provides connectivity to Google APIs in the GCP cloud. -- Secure network connectivity to Google APIs using Private Service - Connect endpoints. +- Secure network connectivity to Google APIs using Private Service + Connect endpoints. -- To the greatest extent possible, perform cost optimization and - complexity reduction by leveraging the capability to share cloud - resources.  +- To the greatest extent possible, perform cost optimization and + complexity reduction by leveraging the capability to share cloud + resources.  -- High availability, fault tolerance, and resilience through redundant - networking.  +- High availability, fault tolerance, and resilience through redundant + networking.  -- Integration with GCP IAM to control privileged access to cloud - networking resources and to support security compliance, auditing, - and logging.  +- Integration with GCP IAM to control privileged access to cloud + networking resources and to support security compliance, auditing, and + logging.  ## Assumptions -- Routing to GCP Private Service Connect endpoints from on-premise - networks is outside the scope at the time of this writing. +- Routing to GCP Private Service Connect endpoints from on-premise + networks is outside the scope at the time of this writing. -```{=html} -``` -- Support for IPv6 integration with Private Service Connect is out of - scope. IPv6 addressing and its support with PSC is reserved for - future integration when required by TCO. -- Private Service Connect endpoints are required within each GCP VPC. +- Support for IPv6 integration with Private Service Connect is out of + scope. IPv6 addressing and its support with PSC is reserved for future + integration when required by TCO. + +- Private Service Connect endpoints are required within each GCP VPC. ## Constraints -- At the time of this writing, Private Service Connect is only - configured to access Google APIs and services using the **ALL APIs** - bundle. +- At the time of this writing, Private Service Connect is only + configured to access Google APIs and services using the **ALL APIs** + bundle. -- The design and usage of PSC is not intended to provide private - networking connections between VPCs and associated services in a - consumer / producer configuration. +- The design and usage of PSC is not intended to provide private + networking connections between VPCs and associated services in a + consumer / producer configuration. ## Logical Design @@ -234,8 +231,8 @@ Private Service Connect is configured within GCP for access to Google API without traversing the public internet, known as Private Google Access. Private Google Access is enabled on a subnet-by-subnet basis. -![](images/private-service/media/image7.png){width="6.401388888888889in" -height="4.090972222222222in"} + Figure 1 PSC Private Google Access @@ -249,15 +246,11 @@ internal IP addresses. Endpoints within the VPC are then able to communicate via IPv4 and forward traffic to the Google Services Network and corresponding APIs. - ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- - Zone name DNS name Description Zone type - ----------------------------------------------------- ------------------- ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- ----------- - *goog-psc-vpcs-services-ssvcpr-7131329659510185153* p.googleapis.com. \[DO NOT DELETE\]\[Private Service Connect for Google APIs\] The private managed DNS zone for the Service Directory namespace Service - https://servicedirectory.googleapis.com/v1/projects/gcp-inf-vpcs-ssvcprod-x2vdlm4/locations/us-central1/namespaces/goog-psc-vpcs-services-ssvcpr-7131329659510185153. Directory - - ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +| Zone name | DNS name | Description | Zone type | +|-----------------------------------------------------|-------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------| +| *goog-psc-vpcs-services-ssvcpr-7131329659510185153* | p.googleapis.com. | \[DO NOT DELETE\]\[Private Service Connect for Google APIs\] The private managed DNS zone for the Service Directory namespace https://servicedirectory.googleapis.com/v1/projects/gcp-inf-vpcs-ssvcprod-x2vdlm4/locations/us-central1/namespaces/goog-psc-vpcs-services-ssvcpr-7131329659510185153. | Service Directory | - : Table 4 Private Google Access Configuration Requirements +Table 4 Private Google Access Configuration Requirements ## Interfaces @@ -274,15 +267,15 @@ Systems within a VPC send packets to the external IP addresses of Google APIs and services using Private Google Access if all these conditions are met: -- The subnet where traffic is originating has Private Google Access is - enabled. +- The subnet where traffic is originating has Private Google Access is + enabled. -- The VPC network that contains the subnet meets the [network - requirements for Google APIs and - services](https://cloud.google.com/vpc/docs/configure-private-google-access#requirements). +- The VPC network that contains the subnet meets the [network + requirements for Google APIs and + services](https://cloud.google.com/vpc/docs/configure-private-google-access#requirements). -- The source IP address of packets on the subnet uses an internal IPv4 - address from an alias IP range +- The source IP address of packets on the subnet uses an internal IPv4 + address from an alias IP range ## Consumption @@ -300,7 +293,7 @@ Code. ## Key Performance Indicators No KPIs are defined at this time. There are a variety of USCB standard, -GCP-native, and 3^rd^ party tools that will be used to measure +GCP-native, and 3rd party tools that will be used to measure performance on the GCP Network including Cloud Monitoring, Ping, iPerf3, and Solarwinds. @@ -315,58 +308,26 @@ GCP Private Service Connect and Private Google Access are features of the Google Cloud Platform provided by Google. As this is a managed service, all patches or upgrades are solely the responsibility of Cloud Service Provider. However, any new VPCs and/or subnets created within -USCB's GCP boundary requires associated configurations to enable Private +USCB’s GCP boundary requires associated configurations to enable Private Service Connect, Private Google Access, and associated DNS Private Hosted Zones (PHZs). ## *Roles and Responsibilities* - ----------------------------------------------------------------------------------- - Role - ---------------------- ----------------- ------------------------------------------ - NetworkAdministrator gr-net.admin Requires admin level permissions to - create, modify and delete GCP cloud - networking resources including VPCs, - Shared VPC, Cloud DNS, Private Service - Connect, and Private Google Access. - - NetworkSupport gr-net.operator Permissions to read, describe and list GCP - cloud networking resources including VPCs, - Shared VPC, Cloud DNS, Private Service - Connect, and Private Google Access. - ----------------------------------------------------------------------------------- +| Role | | | +|----------------------|-----------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| NetworkAdministrator | gr-net.admin | Requires admin level permissions to create, modify and delete GCP cloud networking resources including VPCs, Shared VPC, Cloud DNS, Private Service Connect, and Private Google Access. | +| NetworkSupport | gr-net.operator | Permissions to read, describe and list GCP cloud networking resources including VPCs, Shared VPC, Cloud DNS, Private Service Connect, and Private Google Access. | ## Service Limits & Capacity Planning The following Quotas apply with the Private Service Connect service: - ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- - Quota Type Quota Description Quota name: Quota Value - --------------------------------------------------------------------------------------------------------------------------------------------------------- ------------------------- ----------------------------------------------------- ----------- - [PSC internal LB forwarding The maximum number of PSC-INTERNAL-LB-FORWARDING-RULES-per-project-region 1000 - rules](https://console.cloud.google.com/iam-admin/quotas?service=compute.googleapis.com&metric=compute.googleapis.com/psc_internal_lb_forwarding_rules) Private Service Connect - forwarding rules - (endpoints) that a - service consumer can - create to connect to - producer services. This - quota is per region, per - project. - - [Service attachments](https://console.cloud.google.com/iam-admin/quotas?service=compute.googleapis.com&metric=compute.googleapis.com/service_attachments) The maximum number of SERVICE-ATTACHMENTS-per-project-region 1000 - Private Service Connect - service attachments that - a service producer can - create. This quota is per - region, per project. - - [Network attachments](https://console.cloud.google.com/iam-admin/quotas?service=compute.googleapis.com&metric=compute.googleapis.com/service_attachments) The maximum number of NETWORK-ATTACHMENTS-per-project-region 1000 - network attachments that - a Private Service Connect - consumer can create. This - quota is per region, per - project. - ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +| Quota Type | Quota Description | Quota name: | Quota Value | +|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------|-------------| +| [PSC internal LB forwarding rules](https://console.cloud.google.com/iam-admin/quotas?service=compute.googleapis.com&metric=compute.googleapis.com/psc_internal_lb_forwarding_rules) | The maximum number of Private Service Connect forwarding rules (endpoints) that a service consumer can create to connect to producer services. This quota is per region, per project. | PSC-INTERNAL-LB-FORWARDING-RULES-per-project-region | 1000 | +| [Service attachments](https://console.cloud.google.com/iam-admin/quotas?service=compute.googleapis.com&metric=compute.googleapis.com/service_attachments) | The maximum number of Private Service Connect service attachments that a service producer can create. This quota is per region, per project. | SERVICE-ATTACHMENTS-per-project-region | 1000 | +| [Network attachments](https://console.cloud.google.com/iam-admin/quotas?service=compute.googleapis.com&metric=compute.googleapis.com/service_attachments) | The maximum number of network attachments that a Private Service Connect consumer can create. This quota is per region, per project. | NETWORK-ATTACHMENTS-per-project-region | 1000 | ## Consumption, Licensing, and Assets @@ -385,17 +346,10 @@ with USCB FinOps administrators to forecast and model pricing for ongoing consumption of GCP resources. All assets are tagged and tracked in Apptio Cloudability. - ------------------------------------------------------------------------------------------------------------------------------ - Item Price per hour (USD) Price per GiB - processed, Data - In/Out charges - --------------------------------------------------------------------------------- ------------------------- ------------------ - Private Service Connect endpoint (forwarding rule) to access the regional \$0.01 No data charge - endpoints of Google APIs - - [Private Service Connect endpoint (forwarding rule) used to access Google \$0.01 No data charge - APIs](https://cloud.google.com/vpc/docs/configure-private-service-connect-apis) - ------------------------------------------------------------------------------------------------------------------------------ +| Item | Price per hour (USD) | Price per GiB processed, Data In/Out charges | +|-----------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------|----------------------------------------------| +| Private Service Connect endpoint (forwarding rule) to access the regional endpoints of Google APIs | \$0.01 | No data charge | +| [Private Service Connect endpoint (forwarding rule) used to access Google APIs](https://cloud.google.com/vpc/docs/configure-private-service-connect-apis) | \$0.01 | No data charge | # Backup and Recovery @@ -405,11 +359,11 @@ communicate, shared data and exchange information. GCP Networking provides the infrastructure and services to support backup and recovery services for Private Service Connect, including: -- Global, Multi Region Network +- Global, Multi Region Network -- Cloud DNS +- Cloud DNS -- Serverless Architecture +- Serverless Architecture GCP networking configurations for USCB are deployed through Infrastructure as Code. If required, recovery of networking @@ -461,17 +415,13 @@ Connect and the VPC endpoints, to GCP Cloud Monitoring. References ## Acronyms -[]{#_Toc125379087 .anchor}Table 3 -- Acronyms - - ----------------------------------------------------------------------- - Acronym Definition - ------------- --------------------------------------------------------- - DITD Decennial Information Technology Directorate +Table 3 – Acronyms - - - - ----------------------------------------------------------------------- +| Acronym | Definition | +|---------|----------------------------------------------| +| DITD | Decennial Information Technology Directorate | +| | | +| | | ## Tagging @@ -484,182 +434,142 @@ Standards](https://uscensus.sharepoint.com/:f:/s/DITDSIRS/EpbVeuUQbE1Ftjo-SJpUjj ## *Links* -- [GCP FedRAMP Services in - Scope](https://aws.amazon.com/compliance/services-in-scope/FedRAMP/) +- [GCP FedRAMP Services in + Scope](https://aws.amazon.com/compliance/services-in-scope/FedRAMP/) -- [Compute Engine Service Level Agreement (SLA) \| Google - Cloud](https://cloud.google.com/compute/sla) +- [Compute Engine Service Level Agreement (SLA) \| Google + Cloud](https://cloud.google.com/compute/sla) -- GCP Best Practices +- GCP Best Practices - - [Shared VPC - Design](https://cloud.google.com/architecture/best-practices-vpc-design#shared-common-vpc) + - [Shared VPC + Design](https://cloud.google.com/architecture/best-practices-vpc-design#shared-common-vpc) - - [Liens](https://uscensus.sharepoint.com/sites/DITDSIRS/Shared%20Documents/General/C4.1.4%20IT%20Integration/Infrastructure/04_Solutions%20and%20Service%20Architecture/Google/GCP%20Networking%20Designs/o%09https:/cloud.google.com/vpc/docs/provisioning-shared-vpc#protectsharedvpc) + - [Liens](https://uscensus.sharepoint.com/sites/DITDSIRS/Shared%20Documents/General/C4.1.4%20IT%20Integration/Infrastructure/04_Solutions%20and%20Service%20Architecture/Google/GCP%20Networking%20Designs/o%09https:/cloud.google.com/vpc/docs/provisioning-shared-vpc#protectsharedvpc) - - [Accidental Shared VPC - Shutdowns](https://cloud.google.com/billing/docs/how-to/secure-project-billing-account-link) + - [Accidental Shared VPC + Shutdowns](https://cloud.google.com/billing/docs/how-to/secure-project-billing-account-link) - - [Constrain host project - attachments](https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints) + - [Constrain host project + attachments](https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints) - - [Constrain the subnets in the host - project](https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints) + - [Constrain the subnets in the host + project](https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints) -```{=html} -``` -- [GCP Solutions - Architecture](https://uscensus.sharepoint.com/:w:/s/DITDATO/EaDaa4eHgJhPqnt-9FlGq2YBnV_F83HIhbree1TuW219Zg?e=jK1hQv) -- [GCP DNS Detailed - Design](https://uscensus.sharepoint.com/:w:/s/DITDATO/ESnMqy6UOX5CttAk5vaYXL4BSn5JVRPWAwQ_1nRhXkepsw?e=XchWOq) +- [GCP Solutions + Architecture](https://uscensus.sharepoint.com/:w:/s/DITDATO/EaDaa4eHgJhPqnt-9FlGq2YBnV_F83HIhbree1TuW219Zg?e=jK1hQv) + +- [GCP DNS Detailed + Design](https://uscensus.sharepoint.com/:w:/s/DITDATO/ESnMqy6UOX5CttAk5vaYXL4BSn5JVRPWAwQ_1nRhXkepsw?e=XchWOq) -- [GCP Identity Detailed - Design](https://uscensus.sharepoint.com/:w:/s/DITDATO/ESt6lFhB4RhPnLnHtZEX9nABfZLUyrf9Oezy1sRX_ewqAQ?e=sD9sxk) +- [GCP Identity Detailed + Design](https://uscensus.sharepoint.com/:w:/s/DITDATO/ESt6lFhB4RhPnLnHtZEX9nABfZLUyrf9Oezy1sRX_ewqAQ?e=sD9sxk) -- [GCP Operations Suite Cloud Logging Detailed - Design](https://uscensus.sharepoint.com/:w:/s/DITDATO/EYSWYQAs28pLmu2I1T8Gt7ABC3zzy3ikv2XgB2lHLXJZJQ?e=QSDXv0) +- [GCP Operations Suite Cloud Logging Detailed + Design](https://uscensus.sharepoint.com/:w:/s/DITDATO/EYSWYQAs28pLmu2I1T8Gt7ABC3zzy3ikv2XgB2lHLXJZJQ?e=QSDXv0) -- [GCP Standards & - Configurations](https://uscensus.sharepoint.com/:f:/s/DITDATO/EpC2S1BFNCVNqU93LkYHXzsBYFzBXygEJZEDYHJjqmBFbA?e=kNdbl8) +- [GCP Standards & + Configurations](https://uscensus.sharepoint.com/:f:/s/DITDATO/EpC2S1BFNCVNqU93LkYHXzsBYFzBXygEJZEDYHJjqmBFbA?e=kNdbl8) -- [GCP CIDR - Allocation](https://uscensus.sharepoint.com/:x:/s/DITDATO/EeY54Ec_dyFKhW77iTdWlUsBfLUpkoRy_m6ZbwZ-tQit7Q?e=cuQSB4) +- [GCP CIDR + Allocation](https://uscensus.sharepoint.com/:x:/s/DITDATO/EeY54Ec_dyFKhW77iTdWlUsBfLUpkoRy_m6ZbwZ-tQit7Q?e=cuQSB4) -- [GCP Operating - Plan](https://uscensus.sharepoint.com/:w:/s/DITDATO/ETXm6FlSxm5PrqipdIqa1aQB3uIXt8gBvf01KZcd4wl-ow?e=9Q2SKd) +- [GCP Operating + Plan](https://uscensus.sharepoint.com/:w:/s/DITDATO/ETXm6FlSxm5PrqipdIqa1aQB3uIXt8gBvf01KZcd4wl-ow?e=9Q2SKd) ## *Appendix A* The Appendix below contains specific configuration information and requirements for Private Service Connect and Private Google Access. -+----------------------+--------------------------------+-------------+ -| **Domain and IP | Supported services | Example | -| address ranges** | | usage | -+======================+================================+=============+ -| Default domains. | Enables API access to most | The default | -| | Google APIs and services | domains are | -| All domain names for | regardless of whether they are | used when | -| Google APIs and | supported by VPC Service | you don\'t | -| services *except | Controls. Includes API access | configure | -| for* | to Google Maps, Google Ads, | DNS records | -| pr | and Google Cloud. Includes | for | -| ivate.googleapis.com | Google Workspace web | private.goo | -| and | applications such as Gmail and | gleapis.com | -| restri | Google Docs, and other web | and | -| cted.googleapis.com. | applications. | rest | -| | | ricted.goog | -| Various IP address | | leapis.com. | -| ranges---you can | | | -| determine a set of | | | -| IP ranges that | | | -| contains the | | | -| possible addresses | | | -| used by the default | | | -| domains by | | | -| referencing [IP | | | -| addresses for | | | -| default | | | -| domains](http | | | -| s://cloud.google.com | | | -| /vpc/docs/configure- | | | -| private-google-acces | | | -| s#ip-addr-defaults). | | | -+----------------------+--------------------------------+-------------+ -| pr | Enables API access to most | Use | -| ivate.googleapis.com | Google APIs and services | private.goo | -| | regardless of whether they are | gleapis.com | -| 199.36.153.8/30 | supported by VPC Service | to access | -| | Controls. Includes API access | Google APIs | -| 2600 | to Google Maps, Google Ads, | and | -| :2d00:0002:2000::/64 | Google Cloud, and most other | services by | -| | Google APIs, including the | using a set | -| | following list. Does not | of IP | -| | support Google Workspace web | addresses | -| | applications such as Gmail and | only | -| | Google Docs. Does not support | routable | -| | any interactive websites. | from within | -| | | Google | -| | Domain names that match: | Cloud. | -| | | | -| | - accounts.google.com (only | Choose | -| | the paths needed for OAuth | private.goo | -| | authentication) | gleapis.com | -| | | under these | -| | - \*.aiplat | cir | -| | form-notebook.cloud.google.com | cumstances: | -| | | | -| | - \*.aiplatform- | - You | -| | notebook.googleusercontent.com | don\'t | -| | | use VPC | -| | - appengine.google.com | Service | -| | | | -| | - \*.appspot.com | Controls. | -| | | | -| | - | - You do | -| | \*.backupdr.cloud.google.com | use VPC | -| | | Service | -| | - backupdr.cloud.google.com | | -| | | Controls, | -| | - \*. | but you | -| | backupdr.googleusercontent.com | also | -| | | need to | -| | - | access | -| | backupdr.googleusercontent.com | Google | -| | | APIs | -| | - \*.cloudfunctions.net | and | -| | | | -| | - \*.cloudproxy.app | services | -| | | that | -| | - | are not | -| | \*.composer.cloud.google.com | | -| | | supported | -| | - \*. | by VPC | -| | composer.googleusercontent.com | Service | -| | | Cont | -| | - | rols.[^1^]( | -| | \*.datafusion.cloud.google.com | https://clo | -| | | ud.google.c | -| | - \*.da | om/vpc/docs | -| | tafusion.googleusercontent.com | /configure- | -| | | private-goo | -| | - | gle-access# | -| | \*.dataproc.cloud.google.com | footnote-1) | -| | | | -| | - dataproc.cloud.google.com | | -| | | | -| | - \*. | | -| | dataproc.googleusercontent.com | | -| | | | -| | - | | -| | dataproc.googleusercontent.com | | -| | | | -| | - dl.google.com | | -| | | | -| | - gcr.io or \*.gcr.io | | -| | | | -| | - \*.googleapis.com | | -| | | | -| | - \*.gstatic.com | | -| | | | -| | - \*.ltsapis.goog | | -| | | | -| | - | | -| | \*.notebooks.cloud.google.com | | -| | | | -| | - \*.n | | -| | otebooks.googleusercontent.com | | -| | | | -| | - packages.cloud.google.com | | -| | | | -| | - pkg.dev or \*.pkg.dev | | -| | | | -| | - pki.goog or \*.pki.goog | | -| | | | -| | - \*.run.app | | -| | | | -| | - | | -| | source.developers.google.com | | -+----------------------+--------------------------------+-------------+ + +++++ + + + + + + + + + + + + + + + + + + + +
Domain and IP address rangesSupported servicesExample usage

Default domains.

+

All domain names for Google APIs and services except for +private.googleapis.com and restricted.googleapis.com.

+

Various IP address ranges—you can determine a set of IP ranges that +contains the possible addresses used by the default domains by +referencing IP +addresses for default domains.

Enables API access to most Google APIs and services regardless of +whether they are supported by VPC Service Controls. Includes API access +to Google Maps, Google Ads, and Google Cloud. Includes Google Workspace +web applications such as Gmail and Google Docs, and other web +applications.The default domains are used when you don't configure DNS records +for private.googleapis.com and restricted.googleapis.com.

private.googleapis.com

+

199.36.153.8/30

+

2600:2d00:0002:2000::/64

Enables API access to most Google APIs and services regardless of +whether they are supported by VPC Service Controls. Includes API access +to Google Maps, Google Ads, Google Cloud, and most other Google APIs, +including the following list. Does not support Google Workspace web +applications such as Gmail and Google Docs. Does not support any +interactive websites.

+

Domain names that match:

+
    +

  • accounts.google.com (only the paths needed for OAuth +authentication)

    • +

  • *.aiplatform-notebook.cloud.google.com

    • +

  • *.aiplatform-notebook.googleusercontent.com

    • +

  • appengine.google.com

    • +

  • *.appspot.com

    • +

  • *.backupdr.cloud.google.com

    • +

  • backupdr.cloud.google.com

    • +

  • *.backupdr.googleusercontent.com

    • +

  • backupdr.googleusercontent.com

    • +

  • *.cloudfunctions.net

    • +

  • *.cloudproxy.app

    • +

  • *.composer.cloud.google.com

    • +

  • *.composer.googleusercontent.com

    • +

  • *.datafusion.cloud.google.com

    • +

  • *.datafusion.googleusercontent.com

    • +

  • *.dataproc.cloud.google.com

    • +

  • dataproc.cloud.google.com

    • +

  • *.dataproc.googleusercontent.com

    • +

  • dataproc.googleusercontent.com

    • +

  • dl.google.com

    • +

  • gcr.io or *.gcr.io

    • +

  • *.googleapis.com

    • +

  • *.gstatic.com

    • +

  • *.ltsapis.goog

    • +

  • *.notebooks.cloud.google.com

    • +

  • *.notebooks.googleusercontent.com

    • +

  • packages.cloud.google.com

    • +

  • pkg.dev or *.pkg.dev

    • +

  • pki.goog or *.pki.goog

    • +

  • *.run.app

    • +

  • source.developers.google.com

    • +

Use private.googleapis.com to access Google APIs and services by +using a set of IP addresses only routable from within Google Cloud.

+

Choose private.googleapis.com under these circumstances:

+
    +

  • You don't use VPC Service Controls.

    • +

  • You do use VPC Service Controls, but you also need to access +Google APIs and services that are not supported by VPC Service +Controls.1

    • +