Designing a Secure Multi-AZ VPC Architecture

Good to continue from requirements into the network boundary that makes those requirements enforceable. In the previous lesson, you separated the exchange’s critical order and cancellation path from lower-criticality reporting and defined targets for availability, recovery, security, and latency. This lesson turns that brief into a VPC design that can survive an Availability Zone failure, limits public exposure, and makes outbound connectivity deliberate rather than accidental.

The target is not “a VPC with public and private subnets.” It is a network design you can defend in an interview: clear trust boundaries, routes that match each workload’s purpose, private access to AWS services, layered controls, and known failure behavior.

---

Begin with failure domains and trust zones

A VPC is regional; a subnet belongs to exactly one Availability Zone (AZ). High availability therefore comes from placing equivalent capacity in multiple AZs and ensuring that routing does not silently create a dependency on one AZ.

For the exchange scenario, assume one primary AWS Region with three AZs. Three AZs provide better capacity and maintenance resilience, though a two-AZ design remains acceptable where the service, cost profile, or Region makes it appropriate. The essential requirement is that the critical service can continue when one AZ is unavailable.

A practical layout has four subnet roles in each AZ:

Subnet role	Typical contents	Internet reachability	Design purpose
Public ingress/egress subnet	Internet-facing load balancer nodes, NAT Gateway, optionally egress firewall components	Direct route to Internet Gateway	The only subnet tier that needs a route to the public internet
Private application subnet	ECS tasks, EKS nodes and Pods, internal services, worker processes	Outbound-only through controlled egress; no direct Internet Gateway route	Runs the application and order-processing services without public IPs
Isolated data subnet	RDS, cache nodes, internal stateful services	No default internet route	Limits network paths to high-value data systems
Private endpoint subnet (optional but useful at scale)	Interface VPC endpoint network interfaces	No public route	Makes endpoint capacity, IP use, and security policy easier to manage

For a senior answer, say explicitly that a public subnet does not mean every resource in it is publicly reachable. A subnet is called public because its route table has a route to an Internet Gateway. A workload is reachable only when several conditions align: it has a public address where applicable, its security group permits traffic, its network ACL permits traffic, and a route exists.

Likewise, a private application subnet can use a NAT Gateway for outbound connections without being publicly reachable. NAT permits connections initiated from the private side; it is not a path for unsolicited inbound connections.

The AWS reference architecture below shows the core two-AZ version of this pattern: an internet-facing Application Load Balancer and NAT Gateways in public subnets, with application servers in private subnets. We will strengthen that baseline by adding an isolated data tier, endpoint strategy, and explicit egress policy.

Before continuing, read the AWS VPC reference. It is a concise, authoritative baseline for the public/private routing and security-group relationship.

Establish address space that will not become a constraint

CIDR allocation is easy to overlook until an EKS rollout, a VPC peering connection, an acquisition, or a second Region reveals that the initial address space is too small or overlaps another network.

For a production platform, reserve a non-overlapping VPC range after confirming enterprise IP address management constraints. A sample primary Region allocation could be $10.42.0.0/16$:

Tier	AZ A example	AZ B example	AZ C example	Why this size
Public ingress/egress	$10.42.0.0/22$	$10.42.4.0/22$	$10.42.8.0/22$	Space for ALB scaling, NAT, firewall endpoints, and future ingress components
Private application	$10.42.16.0/20$	$10.42.32.0/20$	$10.42.48.0/20$	Headroom for ECS tasks or Kubernetes Pod IP consumption
Isolated data	$10.42.64.0/22$	$10.42.68.0/22$	$10.42.72.0/22$	Database, cache, and internal stateful capacity
Endpoint/shared services	$10.42.80.0/24$	$10.42.81.0/24$	$10.42.82.0/24$	Interface endpoint ENIs and private shared services

These are examples, not universal subnet sizes. The justification matters more than the numbers:

• EKS using the AWS VPC CNI consumes VPC addresses for Pods, which can exhaust casually sized application subnets well before CPU or memory is exhausted.
• AWS reserves IP addresses in every subnet, so nominal CIDR capacity is not fully available.
• Future VPC peering, Transit Gateway, on-premises connectivity, and disaster-recovery networks require non-overlapping ranges.
• A secondary Region should have a separate, pre-reserved CIDR range. Do not discover overlap during a regional-recovery event.

For a lead-level design, document the AZ IDs rather than assuming an AZ name such as us-east-1a maps to the same physical AZ in every AWS account. This matters when coordinating network topology across accounts.

---

Route tables express intent and failure behavior

The route table associated with a subnet is the clearest expression of its allowed network paths. Keep route tables specific to a purpose and, where required, specific to an AZ. A single shared “private route table” is often the beginning of hidden cross-AZ dependencies.

Public subnet routing

Each public subnet needs the VPC-local route and a default route to the Internet Gateway:

Destination	Target	Meaning
VPC CIDR	local	Communication inside the VPC
$0.0.0.0/0$	Internet Gateway	IPv4 internet path for public-tier resources
$::/0$, if dual stack	Internet Gateway	IPv6 internet path for public-tier resources

The NAT Gateway lives in this subnet and has an Elastic IP. Internet-facing load balancer nodes also use public subnets, but the application tasks behind them do not need public addresses.

Private application subnet routing

Each private application subnet should have a route table dedicated to its AZ:

Destination	Target	Meaning
VPC CIDR	local	Internal VPC communication
$0.0.0.0/0$	NAT Gateway in the same AZ	Controlled IPv4 egress for approved external dependencies
S3 managed prefix list	S3 Gateway Endpoint	Private, direct route to S3
DynamoDB managed prefix list	DynamoDB Gateway Endpoint	Private, direct route to DynamoDB, if needed
$::/0$, if dual stack	Egress-only Internet Gateway	Outbound-only IPv6 internet connectivity

The important design point is same-AZ NAT routing. If workloads in AZ A use a NAT Gateway in AZ B, then an AZ B failure can remove egress for otherwise healthy AZ A workloads. It also creates cross-AZ data transfer charges and unnecessary latency.

An AZ-local NAT design does not mean every service must depend on NAT. Critical workloads should use VPC endpoints for AWS dependencies such as secrets retrieval, logging, image pulls, and artifact access. This reduces both NAT cost and the blast radius of an egress failure.

Isolated data subnet routing

The data tier should normally have only:

Destination	Target	Meaning
VPC CIDR	local	Connectivity to approved services within the VPC
Specific endpoint prefix list, only if required	Gateway Endpoint	Narrow access to a necessary AWS service

There is deliberately no default route to a NAT Gateway or Internet Gateway. An RDS instance should not download packages, call arbitrary external APIs, or need general internet access. Operational access should be performed through managed controls and application-level processes, not by turning the data subnet into an application subnet.

Multi-AZ NAT: resilience before apparent savings

A common interview trap is to create one NAT Gateway because it is expensive, then route every private subnet through it. That architecture may work, but it makes a single AZ and its NAT path a dependency for all workloads’ external access.

AWS recommends at least one NAT Gateway in each AZ where workloads run. The following AWS Networking blog explains both the distributed per-AZ approach and the alternative centralized egress model.

There are two defensible egress patterns:

1. Distributed egress within the workload VPC. Each AZ routes to its local NAT Gateway. This is usually the straightforward choice for a single product VPC or a high-availability exchange service with independent teams.

2. Centralized egress through a dedicated egress VPC. Multiple workload VPCs reach an egress VPC through Transit Gateway, where firewall inspection, proxying, logging, and NAT are centrally managed. This can fit a regulated multi-account environment, but it introduces additional routing, inspection, failure, and capacity dependencies. It must still preserve AZ locality.

For the exchange platform, begin with distributed NAT in the production workload VPC unless a validated enterprise networking pattern already mandates centralized inspection. Do not introduce Transit Gateway merely because it sounds more sophisticated.

---

Use VPC endpoints to reduce egress dependency

A NAT Gateway is appropriate for approved third-party APIs, software repositories, certificate services, or vendor endpoints. It is usually the wrong default for calls to AWS services available through VPC endpoints.

There are two endpoint types with different network behavior.

Endpoint type	Common services	Network mechanism	Key controls
Gateway Endpoint	S3, DynamoDB	Route-table entry using an AWS-managed prefix list	Endpoint policy, bucket/table policy, workload IAM policy
Interface Endpoint	Secrets Manager, STS, ECR APIs, CloudWatch Logs, KMS, SSM, many others	Private Elastic Network Interfaces in selected subnets	Endpoint security group, endpoint policy, workload IAM policy, private DNS

For an exchange workload, a realistic initial endpoint set might include:

• S3 Gateway Endpoint for artifacts, immutable audit exports, backups, and controlled data retrieval.
• DynamoDB Gateway Endpoint if the platform legitimately uses DynamoDB.
• Secrets Manager Interface Endpoint so workloads retrieve credentials without requiring NAT.
• CloudWatch Logs Interface Endpoint for private log export.
• ECR API and ECR Docker Interface Endpoints, together with the S3 Gateway Endpoint used for image layers.
• STS Interface Endpoint where workloads use AWS STS without internet egress.
• KMS Interface Endpoint if the private workload needs KMS API access.
• SSM, EC2 Messages, and SSM Messages Interface Endpoints for managed operational access to EC2 nodes, where relevant.

Interface endpoints should be deployed in at least the AZs where the consuming workload runs. Enable private DNS so standard AWS service hostnames resolve to endpoint-private IP addresses inside the VPC. Otherwise, a workload may continue resolving the public service hostname and unexpectedly use NAT.

An endpoint is not a blanket authorization bypass. A successful request still requires:

1. A route or local network path to the endpoint.
2. Security-group permission for interface endpoints.
3. An endpoint policy, where supported, that permits the intended call.
4. IAM authorization for the workload role.
5. A resource policy, where relevant, such as an S3 bucket policy or KMS key policy.

That layered model is valuable in regulated environments. For example, an S3 bucket policy can require access through a named VPC endpoint, while the task role is limited to a particular bucket prefix. A stolen credential used outside the approved endpoint path is then less useful.

Controlled egress means more than “put a NAT Gateway there”

A NAT Gateway translates addresses; it does not provide domain-aware allowlisting, TLS inspection, request inspection, or a full audit decision for each destination. If application security groups allow outbound TCP port $443$ to all destinations, the NAT Gateway provides a stable public source IP but not strict control of where applications can connect.

Choose the control level based on risk:

Requirement	Appropriate approach
Stable source IP for an external vendor allowlist	NAT Gateway Elastic IPs
Prevent direct internet access for AWS service calls	VPC endpoints and endpoint policies
Restrict outbound access to specific external destinations	Explicit proxy, DNS-aware controls, or firewall controls with a maintained policy
Inspect and log egress flows centrally	Egress VPC with Network Firewall or an approved third-party firewall/proxy
Prevent unapproved package downloads from production workloads	No general egress by default; use a controlled artifact repository and allowlisted paths

For the order-processing core, minimize external calls entirely. The matching and risk paths should not synchronously depend on a public SaaS endpoint. Third-party calls belong in asynchronous, retriable integration components with explicit timeout, retry, and circuit-breaker behavior.

---

Build microsegmentation with security groups

Security groups are stateful, apply to ENIs and resources, and should be the primary mechanism for application-level segmentation. A stateful rule means return traffic for an allowed connection is automatically permitted; you do not need a matching rule for every ephemeral response port.

The most maintainable pattern is to reference security groups as sources, not broad CIDR ranges. This makes the policy track workload identity rather than an address allocation that changes during deployments.

A minimal service-chain model could be:

Security group	Inbound rules	Outbound rules
sg-alb-public	TCP $443$ from approved public CIDRs; in most public API cases, $0.0.0.0/0$ with AWS WAF policy at the load balancer	TCP $8443$ only to sg-api-service
sg-api-service	TCP $8443$ only from sg-alb-public; health-check path uses the same controlled source	TCP $5432$ only to sg-orders-db; TCP $443$ to required endpoint groups and approved egress path
sg-order-worker	No inbound rule unless a known internal caller requires one	Only required ports to message infrastructure, database, endpoint groups, and observability services
sg-orders-db	TCP $5432$ only from sg-api-service and sg-order-worker	Normally no broad outbound rule; add only justified dependencies
sg-vpce-private	TCP $443$ from specific workload security groups	Return traffic is statefully allowed

A few operational implications matter in interviews:

• Do not permit database access from the entire VPC CIDR merely because it is “internal.”
• Do not permit SSH from the internet. Prefer federated access, short-lived roles, Session Manager, and a recorded break-glass process. Identity design is covered in a later lesson.
• A load balancer security group is the correct source for application-service inbound rules. The client IP address is not a reliable enforcement identity at the application target.
• Restrict outbound rules where operationally feasible, but test dependencies carefully. Overly broad inbound rules are the greater immediate risk; overly restrictive egress rules can create hard-to-diagnose production failures if endpoint, DNS, certificate, and telemetry dependencies are not mapped.

Security groups give the required policy for the direct data path:

Intended connection	Network policy statement
Internet client to public API	Client reaches the public load balancer on TCP $443$ only
Public load balancer to API tasks	sg-alb-public reaches sg-api-service on the application listener port only
API tasks to order database	sg-api-service reaches sg-orders-db on TCP $5432$ only
Worker to managed secret service	sg-order-worker reaches sg-vpce-private on TCP $443$, with IAM authorization required as well
Database to internet	Not permitted because no default route exists

This is an interview-quality distinction: routing determines whether a path can exist; security groups determine which identified workloads may use that path.

---

Use network ACLs as coarse subnet guardrails, not application firewalls

Network ACLs (NACLs) are stateless, subnet-level filters. Every permitted request path needs matching inbound and outbound rules because response traffic is not automatically allowed. Rules are evaluated in numbered order, with the first matching rule taking effect.

That makes NACLs useful for:

• Coarse deny rules for a known malicious CIDR.
• Organization-mandated subnet guardrails.
• Additional control at a public subnet boundary.
• Isolating a subnet class from obviously prohibited network ranges.

They are generally not the best primary mechanism for tier-to-tier access control because:

• They cannot reference security groups.
• Dynamic task and Pod IPs make CIDR rules brittle.
• Load balancers, NAT, and operating systems use ephemeral ports.
• A small mistake can block return traffic and resemble an application timeout.

A pragmatic policy is:

Subnet tier	NACL posture	Main protection mechanism
Public ingress/egress	Explicitly associated NACL; optionally deny known hostile ranges; carefully allow required listener and return traffic	WAF, ALB security group, workload security groups
Private application	Broad internal and required return-path allowances; do not encode workload identity here	Service security groups and egress architecture
Isolated data	Explicit internal-only guardrails if required by policy, tested thoroughly	No default route plus database security group rules
Endpoint subnet	Permit required TCP $443$ paths and return traffic	Endpoint security group, endpoint and IAM policies

If you use restrictive NACLs, account for ephemeral ports in both directions. This is especially important for ALB-to-target traffic, NAT return traffic, and outbound HTTPS connections. In a production change review, a NACL modification that lacks an explicit return-path analysis should be treated as high risk.

---

Failure analysis: what happens when an AZ fails?

A multi-AZ diagram is not sufficient. State what continues to work and what must be tested.

Failure	Expected behavior in this design	Required verification
One application AZ fails	Load balancer sends new requests to healthy targets in remaining AZs; ECS or EKS replaces capacity according to service configuration	Load test remaining AZ capacity; validate health-check convergence and autoscaling limits
One NAT Gateway or NAT AZ path fails	Workloads in that AZ may lose non-endpoint internet egress; workloads in other AZs retain their local NAT paths	Ensure each app subnet points to same-AZ NAT; prove critical AWS calls use endpoints rather than NAT
S3 or Secrets Manager call needed during an internet-egress incident	Endpoint-enabled workload continues through the AWS private path, assuming endpoint capacity and policy are healthy	Test with NAT route disabled; validate private DNS resolution
Database subnet route table is modified accidentally	No internet path exists unless a default route is explicitly introduced	Terraform policy checks, route-table drift detection, change approval
A compromised API task attempts database access outside its role	Security group permits only its defined database port and endpoint path; IAM and database credentials further constrain access	Test deny cases, not only successful connectivity

The central operational lesson is that a healthy service in AZ A should not depend on a NAT Gateway, endpoint ENI, database endpoint, or route table path located only in AZ B.

For critical exchange services, design so that loss of general internet egress does not stop safe order cancellation, internal risk checks, or reconciliation. If external dependencies are unavailable, move to a defined degraded mode rather than allowing an uncontrolled accumulation of retries and connection exhaustion.

---

A concise architecture-defense answer

If asked, “How would you design the VPC?” a strong answer can sound like this:

“I would create a Region-level VPC spanning three Availability Zones, with public ingress and egress, private application, and isolated data subnets in each AZ. The internet-facing load balancer and one NAT Gateway per AZ reside in public subnets. ECS tasks or Kubernetes workloads run without public IPs in private subnets, and databases have no default route to NAT or the Internet Gateway.

“Each application subnet uses a route table pointing only to its same-AZ NAT Gateway for approved external egress. I would add Gateway Endpoints for S3 and DynamoDB where used, and Interface Endpoints in multiple AZs for Secrets Manager, ECR, CloudWatch Logs, STS, KMS, and SSM as required. That reduces NAT dependency for critical AWS calls.

“Security groups provide service-level segmentation: load balancer to API service, API and worker services to database, and workloads to endpoints, each on specific ports. I would use NACLs only as coarse subnet guardrails because they are stateless and unsuitable for expressing dynamic service identity. Finally, I would test AZ failure, endpoint-only operations, route-table changes, DNS behavior, and egress-policy failure as part of production readiness.”

That response states the topology, explains why it is resilient, and makes the security and cost trade-offs explicit.

---

Key takeaways

A defensible multi-AZ VPC design is organized by both trust boundary and failure domain:

• Keep public ingress and NAT in public subnets; keep workloads private; keep databases and high-value state isolated without default internet routes.
• Use one NAT Gateway per workload AZ and route each private subnet to its local NAT Gateway. Avoid hidden cross-AZ egress dependencies.
• Use Gateway Endpoints for S3 and DynamoDB, and Interface Endpoints for supported AWS APIs that critical private workloads need.
• Treat NAT as address translation and controlled source addressing, not as a complete egress-security control.
• Use security groups as the stateful, identity-oriented microsegmentation layer. Use NACLs sparingly as stateless subnet guardrails.
• Validate the design through failure scenarios, not only through a successful deployment.

Next, you will decide how traffic should enter and be protected at the edge: when to use Route 53, CloudFront, WAF, API Gateway, an Application Load Balancer, or a Network Load Balancer for differing latency, availability, and security requirements.