GCP Landing Zone

Problem

A cloud organization decays in a predictable direction. Projects get created outside any hierarchy, each with a default VPC nobody chose. Service account keys accumulate. Someone holds owner in prod because they needed it once. Audit logs exist per project and nowhere centrally, and data can leave through any API a stolen credential can call.

None of this is caused by bad engineers. It is caused by the defaults being wrong and by every correct decision needing to be remade by each person who creates a project. A landing zone moves those decisions into the hierarchy, where inheritance does the enforcing, and a paved road proves a workload can land on it without hand-wiring the controls back in.

Four roots, one identity

• bootstrap creates the seed project, the state bucket, sa-terraform, and a GitHub Workload Identity Federation pool. It is the only layer that runs as a human, because it creates the identity everything else runs as, and it sits outside the hierarchy it manages.
• governance, network, and workload each apply by impersonating sa-terraform. A person holds exactly one grant that matters, token creation on that account, which is short-lived, logged per token, and revoked by removing one binding.
• network and workload are the hourly layers and apply separately, so the governance layer can stay up for nearly nothing while the expensive tiers come and go.

Governance: the API refuses, nobody reviews

• 15 constraints at the organization root, not a folder, because a folder policy is bypassed by creating a project somewhere else. Three are custom CEL rules: GKE clusters must use private nodes and carry a cost_center label, and Cloud SQL instances must not have a public IP. Prod additionally rejects anything without a customer-managed key.
• No standing write in prod. A platform engineer requests a Privileged Access Manager entitlement with a justification, a security approver grants it, and the roles exist on the prod folder for at most an hour. Humans federate in through a workforce identity pool, so the external IdP stays authoritative and removing someone from a group removes their access on the next token.
• Two org sinks with include_children feed a CMEK BigQuery dataset and a Log Analytics bucket. 13 alert policies count against the bucket: org IAM, org policy, VPC-SC, firewall policy, break-glass use, and CIS 2.4 to 2.11.
• Google pre-applies six secure-by-default policies to a new organization. They are left unmanaged on purpose; the first version imported them, and destroy deleted them.

Network: a perimeter, not just a firewall

IAM answers whether an identity may call an API. A VPC Service Controls perimeter answers a question IAM cannot: whether the call may cross this boundary. The restricted Shared VPC and its service project sit inside one, so a stolen credential with storage read cannot read a bucket in the perimeter from a laptop. The operator's own identity is deliberately left out, and the test suite proves the denial.

Egress is default-deny with an FQDN allowlist in a Cloud NGFW network firewall policy, beneath a hierarchical org policy that drops traffic to known-malicious addresses in both directions. It is distributed rather than an AWS-style inspection hub, which is the deliberate difference. Google APIs resolve to a Private Service Connect endpoint through private DNS, so storage.googleapis.com answers on 10.3.255.254 instead of the internet.

The paved road on top

• A private GKE Standard cluster with private nodes and a private endpoint, reached only through its IAM-authenticated DNS endpoint, so there is no IP allowlist to maintain. Dataplane V2 enforces NetworkPolicy natively and etcd secrets are envelope-encrypted with Cloud KMS.
• Binary Authorization admits only digests signed by a KMS-backed attestor, and images pull through an Artifact Registry virtual repository over a Docker Hub remote, so nothing is fetched from the internet at runtime.
• Pods read their database password from Secret Manager through Workload Identity, granted directly to the Kubernetes service account's principal with no Google service account and no key in between.
• Cloud SQL PostgreSQL is regional HA on private IP with CMEK, replicating to a CMEK-encrypted replica in a second region, and a Backup and DR vault with enforced retention is the isolation layer that even the project owner cannot purge early.

What the first live apply found

• The perimeter denied Terraform's own calls. Every root bills quota to the seed project, which sits outside the perimeter, so VPC-SC read each call as leaving it. The denial log named serviceusage.services.use against the seed, and the fix was an egress rule scoped to sa-terraform and the seed project alone.
• Privileged Access Manager's org agent needs a folder role as well as an org role. The org-scoped entitlement created and the folder-scoped one kept failing until the agent could edit folder IAM.
• Cloud SQL does not expose its labels to custom constraints at all, so the cost_center rule moved one step earlier into a conftest policy, and the API-level constraint became the private-IP rule it can actually enforce.
• Firewall policy priorities are unique across both directions, log metrics scoped to a bucket report against the bucket's own resource type, and the Workload Identity pool does not exist until the first cluster creates it. 15 defects in all, none visible to a static gate, each fixed in code rather than by hand.

Verified, then destroyed

Two suites ran against the live org. The guardrail suite proved controls deny, and requires each denial to name the control, so a refusal for an unrelated reason cannot pass: a service account key, an out-of-region bucket, and an unencrypted prod bucket refused; a non-allowlisted site unreachable from inside the VPC while an allowlisted one answered; the operator blocked by the perimeter while sa-terraform was admitted; and the org sink delivering over 13,000 audit rows to BigQuery.

The workload suite proved the paved road end to end: an unsigned image rejected at admission, a signed one admitted, an app pod reading its secret with no key and connecting over TLS, an unlabelled pod and a VM in the same VPC both unable to reach the database, and a zonal failover from us-central1-f to us-central1-c in 37 seconds on the same private IP. Then all four roots were destroyed and a teardown check confirmed nothing hourly survived and Google's defaults were intact.

Compute baseline, detection and response

The private bake reads Ubuntu through Artifact Registry remote repositories, because the bake VPC has no internet route. A golden image carrying the shared cis_baseline role boots the management VM, which is reached only through IAP and OS Login and has no external IP. compute.trustedImageProjects rejects stock images everywhere else. make test-compute passed 20 of 20, including the live hardening check and a SUCCEEDED OS Config patch job through the restricted Private Service Connect path.

On top sit Cloud Asset Inventory feeds with a BigQuery export (an IAM change was found in both, then reverted: 9 of 9), 11 ops alert policies (a forced egress-deny alert opened in about three minutes: 15 of 15), and a private Cloud Run handler. The handler quarantined the management VM from a sample Security Command Center finding: tagged, snapshotted, stopped and its service account detached, then restored. It also resized the GKE pool from 2 to 3 and back (24 of 24 live). The database password was moved out of Terraform, and no secret value appears in any of 7 state objects (30 of 30).

Gated CI on federated identities

The federation pool existed from the first build, but its environment bindings pointed at sa-terraform, which holds organizationAdmin and can rewrite the pool's own trust. On 2026-10-26 those bindings moved to a new sa-gha-apply that holds networking roles in the seed project and nothing at the org. It is reachable only from a job bound to the prod-apply or destroy environment, because its binding matches the token's environment claim, and both environments carry a required reviewer and accept main only. sa-terraform is no longer reachable from CI at all.

State got the same treatment. The plan identity had object admin on the whole state bucket, so a plan could have edited state. It now reads every root's state but writes only .tflock objects, and the apply identity reads and writes only the ci-proof prefix, both through IAM conditions. The pool also checks the repository's numeric ID and owner ID alongside its name, because a name can be freed and claimed by someone else. The AWS and Azure pipelines each failed a first run on a name-based OIDC subject, so this one pinned IDs from the start.

The first run failed anyway, at init: a 403 writing the ci-proof root's state. The GCS backend writes an empty state object the first time a root is initialized, and the plan identity may now write only lock files. That was the new condition working on CI itself, and a one-time operator init seeds a new root. After that, a gated apply built three resources after approval, a gated destroy removed all three with the hourly check clean, and the TTL guard read network, workload, compute, and ci-proof green.

Scope, stated plainly: CI applies only a free proof root, a VPC, subnet, and deny-all firewall rule in the seed project. Governance, network, workload, and compute stay operator-applied through impersonation, because they need org roles CI should never hold.

Honest limits

Security Command Center is not activated for the organization (console-only), so that path was proven by publishing a sample finding to the topic it would publish to. The first live incident run failed Cloud SQL over by mistake, because the script claimed a dry-run gate the handler did not have; failover is now a separate, off-by-default switch. Cloud SQL failover through the handler, a real secret rotation, and deleting a backup vault that holds backups are not proven. An earlier run in August measured a zonal failover in 37 seconds on the same private IP; the final session skipped that step.