Building a Scalable Chat App with WebSockets & Pub/Sub

Welcome back. In our previous lesson, we established a strong foundation by comparing real-time communication protocols, concluding that WebSockets are the superior choice for applications demanding low-latency, bidirectional communication. You learned that while WebSockets solve the problem of pushing data from server to client, their stateful nature introduces new challenges for scalability and fault tolerance.

Today, we will build directly on that knowledge to tackle a cornerstone of system design interviews: designing a scalable chat application utilizing WebSockets and a Pub/Sub backend. This problem is a favorite among interviewers because it elegantly bundles many distributed systems challenges—real-time communication, state management, and data persistence at scale—into a single exercise. For you, mastering this design is a crucial step toward bridging the gap between small-business architectures and the highly scalable systems you aim to build. Our primary focus will be on the core architectural problem: how to efficiently route messages between millions of users when they are connected to thousands of different servers.

Step 1: Clarifying the Requirements

As you know from your goal of preparing for senior-level interviews, jumping straight into a design without a clear scope is a common pitfall. A robust design starts with well-defined requirements. Let's establish what we're building, which will act as our guide for the rest of the lesson.

From this reading, we can crystallize our core requirements:

• Functional:
  • One-to-one and group messaging.
  • Message history / persistence.
  • Presence indicators (online, typing).
• Non-functional:
  • Low latency: Messages should feel instantaneous.
  • High availability & Fault tolerance: The system must work even if servers fail.
  • Scalability: The architecture must support millions of concurrent users.

The core challenge highlighted—managing millions of concurrent, stateful connections—is what we will now address.

Step 2: The Challenge of Scaling WebSocket Servers

In the last lesson, you learned that WebSocket connections are persistent. This is a fundamental shift from the stateless request-response model of HTTP. Let's consider a naive approach to scaling: place multiple WebSocket servers behind a load balancer.

Now, imagine User A connects and their connection is routed to WebSocket Server 1. User B connects and is routed to WebSocket Server 2. User A sends a message intended for User B.

The message arrives at WebSocket Server 1. But how does it get to User B, whose connection lives on WebSocket Server 2? WebSocket Server 1 doesn't know about Server 2 or the connections it holds. This is the central routing problem in any large-scale, real-time system.

Step 3: The Publish/Subscribe (Pub/Sub) Pattern

The solution to this routing problem is to decouple the servers from each other using an intermediary message broker that implements the Publish/Subscribe (Pub/Sub) pattern.

Instead of servers communicating directly, they all communicate through the Pub/Sub system. Here’s how it works:

• Publish: When a server receives a message, it doesn't try to find the recipient server. It simply publishes the message to a specific "topic" or "channel" in the Pub/Sub system.
• Subscribe: When a user connects to a server, that server subscribes to the topic(s) relevant to that user.
• Routing: The Pub/Sub system handles the routing, instantly forwarding any message published to a topic to all servers subscribed to that topic.

This pattern elegantly solves the routing problem. Your WebSocket servers no longer need to know about each other; they only need to know how to talk to the Pub/Sub system. This allows you to add or remove WebSocket servers seamlessly, enabling true horizontal scaling.

Step 4: Choosing and Using a Pub/Sub System

Many technologies can act as a Pub/Sub backend, including Kafka, RabbitMQ, and Redis. For a chat system, the choice has significant implications. We need something extremely fast for real-time notifications.

As the video explains, Redis Pub/Sub is an excellent fit for the real-time notification part of our chat system. It's incredibly fast but provides "at-most-once" delivery, meaning a notification might get dropped under certain failure conditions.

This is acceptable because we will design a two-pronged delivery strategy:

1. The "Fast Path" (Online Users): Use Redis Pub/Sub to instantly notify the recipient's connected server, which then pushes the message down the WebSocket. This covers the vast majority of cases and provides low latency.
2. The "Reliable Path" (Offline/Missed Messages): Before publishing, we first write every message to a durable database (like Cassandra or DynamoDB). If the fast path fails, or if the user is offline, they can retrieve the message from the database when they next connect.

This hybrid approach gives us both speed and reliability, a common and effective pattern in distributed systems.

Step 5: Deep Dive into the Architecture

Now let's flesh out the key components of our design.

Message Flow and Storage

The end-to-end flow for a one-to-one message from User A to User B would be:

1. User A's client sends the message via WebSocket to its connected server, WS Server A.
2. WS Server A generates a unique message ID (e.g., using a service like Snowflake) and writes the message to a durable NoSQL database. A good data model, as you'd know from your database experience, would be to partition by conversation_id and cluster by message_id or timestamp for efficient retrieval of a conversation's history.
3. WS Server A publishes a notification to a Redis topic specific to User B, e.g., user_B_messages. The notification contains the message itself or just its ID.
4. User B is connected to WS Server B, which is subscribed to the user_B_messages topic.
5. Redis pushes the notification to WS Server B.
6. WS Server B pushes the message down the WebSocket to User B's client.
7. User B's client sends an acknowledgment (ack) back to the server.

For handling offline users and ensuring delivery, we use an inbox pattern.

Group Chat and Fan-Out

Group chat is a natural extension of this model. Instead of a user-specific topic, a group chat corresponds to a shared topic.

• When a user sends a message to a group, their server publishes it to the group_XYZ topic.
• All servers with users currently in that group will be subscribed to the group_XYZ topic.
• The Pub/Sub system "fans out" the message to all subscribed servers, which then push it to their connected clients.

This fan-out on a Pub/Sub topic is a highly efficient way to manage group messaging.

Presence Indicators

Presence (the green dot) is another real-time feature that fits perfectly into our Pub/Sub architecture.

When a user's status changes (e.g., they come online, go offline, or a heartbeat times out), their connected WebSocket server publishes this status change to a presence topic. Their friends' servers, subscribed to this topic, receive the update and forward it to the clients, updating the UI.

Putting It All Together

Our final architecture looks something like this:

Conclusion

In this lesson, we designed a scalable chat application, moving from initial requirements to a robust, distributed architecture. You saw how to progress from a simple, non-scalable model to one that can handle millions of users by identifying and solving the core routing bottleneck.

Key Takeaways:

• The Core Problem: The stateful nature of WebSockets creates a routing challenge when scaling horizontally.
• The Pub/Sub Solution: Decoupling WebSocket servers with a Pub/Sub message broker is the canonical solution. Servers publish messages to topics, and other servers subscribe to receive them.
• Hybrid Reliability: For a chat system, combining a fast but less reliable notification layer (like Redis Pub/Sub) for online users with a durable persistence layer (like a NoSQL database) for guaranteed delivery is a powerful and efficient pattern.
• Unified Pattern: The Pub/Sub model is versatile and can be used to power message delivery, group chats, and presence notifications within the same architectural framework.

In our next lesson, we will expand on the concept of fan-out that we touched on in group chats. We'll apply it to a different problem domain: designing a news feed system, where the trade-offs between fanning out on write versus on read become even more critical.