Saga Pattern for Distributed Transactions

Hello! Welcome back to our module on Distributed Transactions & Data Management.

In our previous lesson, we contrasted the choreography and orchestration patterns, establishing them as the two primary ways to coordinate workflows across microservices. We learned that while choreography offers loose coupling, orchestration provides centralized control and visibility.

Today, we're moving from the "how" of coordination to the "what" of the workflow itself. Your learning outcome is to design a saga flow for a distributed transaction, outlining events and compensation logic. This is a fundamental skill tested in system design interviews, as it demonstrates your ability to think through failure scenarios and ensure data consistency in a distributed environment. We will break down a business process into a series of steps, define the logic for both success and failure, and structure it in a way that is robust and understandable.

1. The Anatomy of a Saga: Forward and Compensating Transactions

As we discussed, a saga is a sequence of local transactions. The key to designing a good saga is to meticulously plan for two paths:

1. The "Happy Path": The sequence of forward transactions (T1, T2, ..., Tn) that execute when everything goes right.
2. The Failure Path: The sequence of compensating transactions (C1, C2, ..., Cn-1) that undo the work of the forward transactions when something goes wrong.

Let's design a saga for a common e-commerce scenario: creating a new order.

Step 1: Defining the Forward Transactions (The Happy Path)

First, we need to identify all the individual service interactions required to successfully create an order. Each of these will be a local, atomic transaction within its own service.

Based on the resource, the happy path for our Create Order SAGA involves the following sequence of local transactions:

1. T₁: Order Service - Creates an Order and sets its initial state to APPROVAL_PENDING.
2. T₂: Consumer Service - Verifies that the consumer exists and is eligible to place orders.
3. T₃: Kitchen Service - Validates the order details (e.g., menu items are available) and creates a Ticket in a CREATE_PENDING state.
4. T₄: Accounting Service - Authorizes the consumer's credit card for the order amount.
5. T₅: Kitchen Service - Changes the state of the Ticket to AWAITING_ACCEPTANCE.
6. T₆: Order Service - Changes the state of the Order to APPROVED.

This sequence represents the ideal flow. But in distributed systems, we must design for failure.

Step 2: Designing for Failure with Compensating Transactions

A compensating transaction (C) is an operation that semantically reverses the effect of a corresponding forward transaction (T). If T₃ fails, the saga must execute C₂ and C₁ to roll back the changes made by T₂ and T₁.

It's crucial that compensating transactions are idempotent, meaning they can be executed multiple times without changing the result beyond the initial application. This is important because retry logic in a message broker or orchestrator might trigger the compensation more than once. For example, RejectOrder is idempotent; calling it twice on an already REJECTED order has no further effect.

The following image provides a great high-level visualization of how happy paths and compensation paths fit together in a saga.

Now, let's apply this concept to our specific Create Order SAGA.

Let's formalize the design by creating a table that maps each forward transaction to its compensation. This is an excellent format to use in a design document or a whiteboard interview.

#	Forward Transaction (T)	Compensation (C)	Description of Compensation
T₁	Order Service: Create Order (state: APPROVAL_PENDING)	Order Service: Reject Order	Changes the order state to REJECTED.
T₂	Consumer Service: Verify Consumer	(None)	This is a read-only step. No data was changed, so no compensation is needed.
T₃	Kitchen Service: Create Ticket (state: CREATE_PENDING)	Kitchen Service: Reject Ticket	Changes the ticket state to CREATE_REJECTED.
T₄	Accounting Service: Authorize Card	(None needed if T₅ always succeeds)	See discussion below on Pivot Transactions. If a refund were needed, the compensation would be Void Authorization.
T₅	Kitchen Service: Approve Ticket	(Assumed to always succeed)	---
T₆	Order Service: Approve Order	(Assumed to always succeed)	---

If transaction T₄ (Authorize Card) fails, the saga must execute compensations in reverse order:

1. C₃ is executed: The Kitchen Service rejects the pending ticket.
2. C₂ is skipped: Verify Consumer was read-only.
3. C₁ is executed: The Order Service rejects the pending order.

The system is now back in a consistent state.

2. Refining the Design: The Pivot Transaction

In a senior-level interview, you can elevate your design by structuring the saga more formally. A powerful concept is the pivot transaction.

• Compensatable Transactions: Transactions that occur before the pivot. They must have corresponding compensating transactions.
• Pivot Transaction (Tₚ): The "point of no return." Once the pivot transaction commits, the saga is guaranteed to complete successfully. It is the go/no-go point. A failure at the pivot transaction still triggers a full rollback.
• Retriable Transactions: Transactions that occur after the pivot. They must be designed to be idempotent and guaranteed to eventually succeed, perhaps through retries. They do not have compensating transactions because the saga is committed to finishing.

Applying this to our Create Order SAGA:

• Compensatable: T₁, T₂, T₃ (Create Order, Verify Consumer, Create Ticket). If any of these fail, we can safely roll back.
• Pivot: T₄ (Authorize Card). This is the critical moment. If we can secure the payment, we are committing to fulfilling the order. Failure here triggers a rollback of T₃ and T₁.
• Retriable: T₅, T₆ (Approve Ticket, Approve Order). Once payment is secured, these steps must succeed. The service might be down temporarily, but the orchestrator or a message listener will keep retrying until it works. We've committed to the customer, so we can't just "undo" this part of the flow.

This structure clarifies the design: you only need to worry about compensation logic for the first part of the saga.

The image below shows how an orchestrator would manage such a flow, keeping track of state and issuing commands for forward or compensating transactions.

Test your understanding!

You are designing a travel booking system that allows users to book a vacation package consisting of a flight, a hotel, and a car rental. The services are FlightService, HotelService, and CarRentalService. A booking is only confirmed if all three can be reserved successfully.

Design the saga flow for this process. Outline:

1. The sequence of forward transactions (the happy path).
2. The corresponding compensating transaction for each forward step.
3. A logical choice for the pivot transaction and justify why.

Show answer

Here is a possible saga design for the travel booking system.

1. Forward Transactions (Happy Path):

• T₁: Flight Service - ReserveFlight(user, flightDetails): Puts a temporary hold on the flight seat. The flight is in a RESERVED state.
• T₂: Hotel Service - ReserveHotel(user, hotelDetails): Puts a temporary hold on the hotel room. The room is in a RESERVED state.
• T₃: Car Rental Service - ReserveCar(user, carDetails): Puts a temporary hold on the car. The car is in a RESERVED state.
• T₄: Payment Service - ProcessPayment(user, totalCost): Charges the user's credit card for the full package amount.
• T₅: Flight Service - ConfirmFlight(bookingId): Changes the flight status from RESERVED to CONFIRMED.
• T₆: Hotel Service - ConfirmHotel(bookingId): Changes the hotel status from RESERVED to CONFIRMED.
• T₇: Car Rental Service - ConfirmCar(bookingId): Changes the car status from RESERVED to CONFIRMED.

2. Compensating Transactions:

Forward Transaction (T)	Compensating Transaction (C)
T₁: ReserveFlight	C₁: CancelFlightReservation
T₂: ReserveHotel	C₂: CancelHotelReservation
T₃: ReserveCar	C₃: CancelCarReservation
T₄: ProcessPayment	C₄: RefundPayment
T₅, T₆, T₇: Confirm...	(None - These are retriable)

3. Pivot Transaction:

The pivot transaction is T₄: ProcessPayment.

Justification:

• Go/No-Go Point: Taking the customer's money is the most significant commitment. Before this point, we have only placed temporary, low-cost holds. If any of ReserveFlight, ReserveHotel, or ReserveCar fails, we can simply cancel the other reservations without any financial impact.
• Point of No Return: Once the payment is successfully processed, the business is obligated to provide the service. We cannot simply "undo" the booking. The subsequent steps (ConfirmFlight, ConfirmHotel, ConfirmCar) become retriable. If ConfirmHotel fails because the service is down, we don't refund the customer; we keep retrying to confirm the hotel booking until it succeeds, as we have already taken their money for it.

Conclusion

In this lesson, we moved from abstract patterns to the concrete steps of designing a distributed transaction. You learned how to map out a business process into a resilient saga that can handle failures gracefully.

Key Takeaways:

• Saga design involves defining a sequence of forward transactions for the happy path and a corresponding set of compensating transactions to handle failures.
• Compensating transactions must be idempotent to handle message retries safely.
• Structuring a saga with a pivot transaction simplifies design. Steps before the pivot are compensatable; steps after are retriable and must be guaranteed to succeed.
• This design blueprint is the essential first step before deciding on an implementation strategy (choreography or orchestration).

Next Up

Having designed a complete saga flow on paper, our next step is to bring it to life. In the next lesson, we will begin the practical implementation. We will start with a choreographed saga, using events and a message broker to drive the workflow we designed today. This will show you how the design translates directly into Spring Boot services communicating via events.