Balancing CAP and Scalability

Hello! Welcome back to your system design course.

In our last lesson, we explored the high-level blueprints for applications: monolithic, microservices, and serverless architectures. We saw that choosing an architecture is a game of trade-offs, especially when you need a system that can scale. The move from a simple monolith to a distributed architecture like microservices solves scaling problems but introduces a new set of fundamental challenges.

Today's Goal

Today, we'll dive into the "physics" of distributed systems. The learning outcome for this lesson is to: Identify and articulate trade-offs between consistency, availability, latency, and scalability for given requirements.

These four properties are at the heart of every large-scale system. You can't maximize all of them at once. As a designer, your job is to understand the constraints and make deliberate choices that align with the product's goals. This is much like product design, where you might have to trade off between a product's durability, weight, and cost. You can't have a feather-light, indestructible product that costs a dollar; you must choose what matters most for the user.

---

1. The Four Pillars of System Performance

Before we get into the trade-offs, let's clearly define the terms we'll be working with.

• Scalability: We've touched on this before. It's the system's ability to handle increasing load (more users, more data, more requests) efficiently. The microservices architecture we discussed is a strategy for achieving scalability.
• Availability: This means the system is operational and ready to respond to requests. If a user opens your app, does it work? An available system says "yes." Availability is often measured in "nines" (e.g., 99.9% uptime, or "three nines").
• Consistency: In a distributed system, this means that anyone who reads data from the system gets the most recent, correct version of that data, regardless of where they access it from. Imagine you update your profile picture; consistency guarantees that all your friends see the new picture immediately, not the old one.
• Latency: This is simply the delay between a user's action and the system's response. From a user's perspective, it's the time you wait for a page to load or a photo to post. Low latency is crucial for a good user experience.

These four properties are interconnected. Improving one often comes at the expense of another. The rest of this lesson is about understanding these relationships.

---

2. The CAP Theorem: A Fundamental Choice

The most famous trade-off in distributed systems is described by the CAP Theorem. It states that a distributed data store can only provide two of the following three guarantees simultaneously:

• Consistency
• Availability
• Partition Tolerance

Let's break this down. We already know Consistency and Availability. The new term is Partition Tolerance.

Partition Tolerance (P): This means the system continues to operate even if communication is lost between some of its servers (a "network partition"). In any large, real-world distributed system that communicates over a network, failures are inevitable. A server can crash, or a network link can break. Therefore, any serious distributed system must be partition tolerant.

Because partitions are a fact of life, the CAP theorem forces a difficult choice: when a partition happens, do you sacrifice Consistency or Availability?

To get a clear, intuitive understanding of this, let's watch a video that explains it with a great analogy.

Making the Choice: CP vs. AP Systems

As the video explains, since you can't avoid partitions (P), your real choice is between C and A. This leads to two main types of systems:

• CP (Consistency + Partition Tolerance): When a partition occurs, the system chooses to preserve consistency by refusing to respond to some requests. It might return an error or time out. This makes it temporarily unavailable.

  • Example: A banking system. If you deposit a check, the system cannot risk showing you an old balance. If it can't confirm the deposit was successfully recorded across its databases due to a network issue, it would rather give you an error message (become unavailable) than show inconsistent data.

• AP (Availability + Partition Tolerance): When a partition occurs, the system chooses to remain available, but it might return stale (out-of-date) data. It prioritizes responding over being perfectly correct.

  • Example: A social media feed. If you're scrolling through posts, it's better to see a slightly outdated feed than a loading spinner or an error page. The system prioritizes keeping the user engaged (availability) even if it means you don't see your friend's latest post for a few seconds (temporary inconsistency).

To see more examples of this trade-off in action, please read the following short sections from GeeksforGeeks.

---

3. PACELC: Adding Latency to the Equation

The CAP theorem is powerful, but it only describes what happens during a network failure. What about when the system is running normally? This is where Latency enters the picture, leading to an extension of CAP called the PACELC theorem.

PACELC states:

• If there is a Partition, a system must choose between Availability and Consistency (this is the CAP part).
• Else (if the system is running normally), it must choose between Latency and Consistency.

Let's explore that "Else" part. Even without failures, there's a trade-off between how fast your system is (latency) and how consistent it is.

• To prioritize Consistency (EC): When data is written, the system must wait for that data to be copied to all its replicas before telling the user "Success!" This coordination takes time and increases latency.
• To prioritize Latency (EL): The system can respond "Success!" immediately after writing to just one server. The data is then copied to other replicas in the background. This is very fast (low latency), but it means for a short period, different users might see different data, leading to eventual consistency.

The same video you watched earlier has a great segment explaining this.

Real-World Systems and PACELC

This PACELC framework is incredibly useful for classifying real-world databases and understanding their behavior. You don't need to memorize this, but seeing how popular systems fit into this model makes the concept very concrete.

System Type	Description	Examples
PC/EC	Prefers Consistency over Availability during a partition, and Consistency over Latency otherwise.	Traditional SQL databases (PostgreSQL, MySQL), Google's Bigtable
PA/EL	Prefers Availability over Consistency during a partition, and Latency over Consistency otherwise.	Amazon's Dynamo, Cassandra, Couchbase
PA/EC	Prefers Availability during a partition, but Consistency when running normally.	MongoDB, Hazelcast
PC/EL	Prefers Consistency during a partition, but Latency when running normally.	PNUTS (Yahoo's database)

This table, derived from the "PACELC design principle" Wikipedia article, shows that system designers make these trade-offs explicit when building their products. When you choose a database for your application, you are also choosing where you want to be in this trade-off space.

---

Conclusion

Today we've unpacked the fundamental constraints that govern all large-scale systems. Understanding these trade-offs is non-negotiable for a system designer. It's about making informed, deliberate choices based on what the user and the business truly need.

Key Takeaways:

• The Four Pillars: System design is a balancing act between Scalability, Availability, Consistency, and Latency.
• CAP Theorem: During a network failure (Partition), you are forced to choose between Consistency (CP) and Availability (AP). You cannot have both.
• PACELC Theorem: This extends CAP by stating that even when the system is running normally (Else), you must trade off between Latency (L) and Consistency (C).
• It's All About Requirements: The "right" choice is never technical; it's driven by the product. A banking app needs consistency above all else (CP/EC), while a social media feed prioritizes being available and fast (AP/EL).

Preview of the Next Lesson:
We've now covered high-level architectural patterns (monoliths vs. microservices) and the fundamental trade-offs of distributed systems. The next logical step is to define how the components of these systems communicate. In our next lesson, we will learn how to design API contracts for system components using REST or RPC patterns. This is how we define the "language" that our UserService and ImageUploadService will use to speak to each other.