Capacity Estimation and Constraint Analysis

Hello again. In the previous lesson, you defined the promise: users track concrete merchant listings, receive one email per downward threshold crossing, and expect recent observations for actively watched listings. Now we test whether that promise is plausible at scale.

For a senior-level interview, capacity estimation is not an accounting exercise. It is how you discover what actually shapes the architecture. A price tracker may have modest user-facing API traffic while generating a continuous, expensive stream of third-party price checks and years of time-series data. By the end of this lesson, you will be able to state assumptions, derive the important rates and volumes, and identify the bottlenecks worth designing around.

---

Estimate for decisions, not false precision

Back-of-the-envelope estimates should be transparent and directionally correct. State assumptions, round aggressively, and use the result to choose where to spend design effort. If an interviewer changes an input, you should be able to adjust the model rather than defend a fabricated exact number.

Use this compact estimation worksheet throughout an interview:

The worksheet is useful, but the price-tracking domain needs one important refinement: distinguish the entities and events that can be deduplicated.

• A watch is a user’s threshold rule for a listing.
• A distinct watched listing is the merchant listing that must be fetched.
• A price check is one attempted fetch from an external merchant source.
• A validated observation is a successful, usable result of that fetch.
• A price-change event is an observation whose price differs from the current stored price.
• A notification request is created only when a changed price produces a qualifying threshold crossing.

Five users watching the same listing should create five watch records but normally just one price check. Failing to make that distinction multiplies external requests, bandwidth, and merchant-rate-limit usage for no user benefit.

---

Establish a coherent working scale

An interviewer may give you numbers. If not, propose a set that is internally consistent with the scope from the previous lesson, then say which assumptions are most sensitive.

For this lesson, assume a service that has gained meaningful traction:

Assumption	Working value	Why it matters
Registered users	$10$ million	Establishes plausible product scale, but does not directly determine capacity.
Daily active users	$1$ million	Drives user-facing API traffic.
Active watches	$5$ million	Drives watch storage and alert evaluation.
Distinct listings with active watches	$2$ million	Drives collection work.
Average watches per active listing	$2.5$	Captures deduplication benefit.
Freshness policy for planning	Check each active watched listing once per hour	A conservative way to meet the $95\%$ under-one-hour objective.
Successful validation rate	$95\%$	Separates attempted checks from usable data.
Changed-price fraction	$10\%$ of valid observations	Limits downstream change and alert work.
Peak multiplier	$3$	Covers scheduled bursts, retries, and uneven source traffic.

The source of the $2.5$ average is:

$$\frac{5\text{ million active watches}}{2\text{ million distinct watched listings}}=2.5\text{ watches per listing}$$

That ratio is strategically important. A naïve design polling each watch hourly performs $120$ million fetches per day. A deduplicated design polls the listing once and evaluates all watches attached to it, requiring only $48$ million fetches per day.

A one-hour interval for every listing is stricter than the stated objective, which permits a small fraction to be older than one hour. That is appropriate for an initial capacity plan: it provides headroom for failures and makes the reasoning simple. Later, a scheduler can allocate shorter intervals to high-priority watches and allow a controlled tail of lower-priority listings to approach the four-hour objective.

---

Derive user-request QPS separately from collection QPS

User-facing traffic is driven by daily active users and their behavior. Suppose the typical daily active user performs:

• $8$ current-price or watch-list reads;
• $2$ price-history page reads; and
• $0.2$ watch-management writes, such as creating, updating, pausing, or deleting a watch.

That gives:

$$\text{Current-price reads per day}=1\text{ million}\times 8=8\text{ million}$$ $$\text{History reads per day}=1\text{ million}\times 2=2\text{ million}$$ $$\text{Watch writes per day}=1\text{ million}\times 0.2=200,000$$

Using approximately $86,400$ seconds per day:

User-facing operation	Daily volume	Average rate	Peak rate at $10$ times average
Current-price and watch-list reads	$8$ million	about $93$ QPS	about $930$ QPS
Paginated history reads	$2$ million	about $23$ QPS	about $230$ QPS
Watch-management writes	$200,000$	about $2.3$ QPS	about $23$ QPS
All reads	$10$ million	about $116$ QPS	about $1,160$ QPS

The read/write ratio is strongly read-heavy. Yet a peak of roughly $1,200$ API reads per second is not intrinsically extreme for a horizontally scaled stateless API tier, especially when current-price reads are cacheable. Do not make “millions of users” alone justify an elaborate serving architecture.

The more consequential rate comes from the freshness promise.

---

Convert freshness into price-check and event rates

With $2$ million distinct actively watched listings checked once per hour:

$$\text{Checks per hour}=2\text{ million}$$ $$\text{Checks per day}=2\text{ million}\times 24=48\text{ million}$$ $$\text{Average check rate}=\frac{48\text{ million}}{86,400}\approx 556\text{ checks per second}$$

At a peak multiplier of $3$, plan the collection plane to dispatch roughly:

$$556\times 3\approx 1,700\text{ checks per second}$$

This is already substantially greater than user-facing write QPS. More importantly, each check crosses a system boundary into a merchant API, feed, or web endpoint, where your own service cannot simply add servers to remove a rate limit.

Now carry the calculation through the pipeline.

Stage	Calculation	Daily volume	Average rate	Peak planning rate
Attempted price checks	$2\text{ million}\times 24$	$48$ million	$556$ per second	$1,700$ per second
Valid observations	$48\text{ million}\times 95\%$	$45.6$ million	$528$ per second	$1,600$ per second
Changed-price events	$45.6\text{ million}\times 10\%$	$4.56$ million	$53$ per second	$160$ per second
Alert evaluations	$4.56\text{ million}\times 2.5$	$11.4$ million	$132$ per second	$400$ per second

For the alert-evaluation row, assume that an unchanged price cannot alter a watch’s threshold state, so alert evaluation is triggered only by a price-change event. Each changed listing must be matched against the watches for that listing.

This rate table gives you a clean event model to say aloud:

The ingestion pipeline receives roughly $1,600$ validated-observation events per second at peak. Only about $160$ of those per second are changed-price events, but each changed event fans out to the watches for that listing, producing about $400$ threshold evaluations per second on average peak planning assumptions.

The average is not the whole story. The distribution of watches matters. A popular game console or phone might have $100,000$ watches even though the global average is $2.5$. A single price change then creates a hot-key fan-out burst. This is why later designs will partition and queue alert work by listing identity rather than scan a global watch table.

---

Estimate notification volume without confusing it with alert evaluation

Notifications are typically much rarer than observations or even threshold evaluations. We need one further explicit assumption.

Assume:

• Half of price changes are downward changes.
• $2\%$ of watch evaluations caused by downward changes cross an armed user threshold.

First find the downward evaluation volume:

$$11.4\text{ million evaluations per day}\times 50\%=5.7\text{ million downward evaluations per day}$$

Then estimate notification requests:

$$5.7\text{ million}\times 2\%=114,000\text{ notification requests per day}$$ $$\frac{114,000}{86,400}\approx 1.3\text{ notifications per second}$$

Even a $3$-times burst is only around $4$ notification requests per second under this model. Email sending throughput is therefore unlikely to be the steady-state capacity bottleneck.

However, notification correctness remains critical. A low average rate does not excuse duplicate sends, and a promotional event can create correlated price drops across thousands of listings. The alert system must tolerate bursts, provider slowdowns, and retries without turning them into duplicate user-visible emails. We will address that in Module 4.

A useful interview distinction is:

Workload	Main scaling concern
Price checks	Merchant quotas, fetch concurrency, response latency, compliance
Observation ingestion	Durable throughput and ordered handling per listing
Historical storage	Long-term write volume, indexes, retention cost
Alert evaluation	Fan-out and hot listings
Notification delivery	Idempotency, retry control, provider reliability

The notification count may be small, but the evaluation path cannot be ignored.

---

Convert rates into bandwidth and worker concurrency

Bandwidth calculations should reflect what crosses each boundary. For price collection, assume an average compressed response payload of $20\text{ KB}$, blending smaller API responses with larger fetched pages. Assume each outgoing request is roughly $1\text{ KB}$.

External collection bandwidth

$$48\text{ million fetch responses per day}\times 20\text{ KB}=960\text{ GB per day}$$ $$\frac{960\text{ GB}}{86,400\text{ seconds}}\approx 11\text{ MB per second average}$$

At three times average, the collector must sustain about:

$$33\text{ MB per second}$$

of inbound third-party response traffic, plus protocol overhead. The outgoing request bytes are much smaller:

$$48\text{ million}\times 1\text{ KB}=48\text{ GB per day}$$

These estimates are highly sensitive to acquisition strategy. A compact merchant API response may be only a few kilobytes. Full HTML pages, redirects, scripts, and embedded metadata can easily make the effective payload much larger. This is another reason APIs and bulk feeds are preferable when permitted: they improve reliability and reduce bandwidth, parser cost, and compliance risk.

Internal event bandwidth

Suppose a serialized observation event, including identifiers, timestamp, price, status, and metadata, averages $200$ bytes:

$$45.6\text{ million events per day}\times 200\text{ bytes}=9.12\text{ GB per day}$$

That is only about $0.1\text{ MB}$ per second on average before broker replication. The internal event payload is far smaller than fetched merchant content; persistence volume and event durability are still meaningful, but raw network bandwidth is not the primary concern.

User-facing egress bandwidth

Assume a current-price response is $2\text{ KB}$, while a page of compact history points is $10\text{ KB}$:

$$8\text{ million}\times 2\text{ KB}=16\text{ GB per day}$$ $$2\text{ million}\times 10\text{ KB}=20\text{ GB per day}$$

The total is around $36\text{ GB}$ per day, or about $0.4\text{ MB}$ per second average. At a $10$-times user-traffic peak, it is roughly $4\text{ MB}$ per second. This is manageable, but it reinforces two API choices for a later lesson: return a compact current-price record and paginate or downsample price history rather than return an unbounded series.

Translate check rate into worker concurrency

Request rate alone hides a collector’s real resource requirement: external latency. If a typical fetch holds an I/O connection for $2$ seconds, then peak in-flight fetches are approximately:

$$1,700\text{ checks per second}\times 2\text{ seconds}=3,400\text{ concurrent fetches}$$

If one asynchronous collector worker safely manages $100$ in-flight requests, the raw concurrency estimate is:

$$\frac{3,400}{100}=34\text{ workers}$$

You would provision more than $34$ workers for failures, uneven merchant latency, retries, and deployments. But do not present “fifty workers” as the solution to collection capacity. If the largest merchant owns $30\%$ of watched listings, it receives:

$$1,700\times 30\%\approx 510\text{ peak requests per second}$$

If that merchant permits only $100$ requests per second, your global fleet capacity is irrelevant. The schedule must respect the quota, use an authorized feed or API where available, reduce the eligible collection population, or renegotiate the freshness requirement.

---

Project price-history storage over time

Price-history storage grows with observations, not DAU. Suppose a compact stored observation contains a listing identifier, timestamp, normalized price, currency, source metadata, and status. Use $100$ bytes as a deliberately simple logical estimate, excluding substantial database-specific overhead.

$$45.6\text{ million observations per day}\times 100\text{ bytes}=4.56\text{ GB per day}$$

At this rate:

Retention period	Logical observation data	With $3$ replicas
$90$ days of granular history	about $410\text{ GB}$	about $1.2\text{ TB}$
$1$ year of granular history	about $1.7\text{ TB}$	about $5\text{ TB}$
$3$ years of granular history	about $5\text{ TB}$	about $15\text{ TB}$

The replicated number is still not a complete physical-storage estimate. Indexes, compaction space, write amplification, operational headroom, and backups all consume additional capacity. A prudent early planning envelope might be roughly twice the replicated data size, making three years of fully granular retained observations a system on the order of tens of terabytes rather than a few terabytes.

Compare this with state records:

Dataset	Approximate logical size	Capacity significance
$5$ million watches at $200$ bytes each	about $1\text{ GB}$	Small in bytes; important for lookup and alert fan-out.
$2$ million current-price records at $200$ bytes each	about $400\text{ MB}$	Small and highly cacheable.
$2$ million listing metadata records at $1\text{ KB}$ each	about $2\text{ GB}$	Small relative to history.
Three years of observations	about $5\text{ TB}$ logical	Dominates durable data volume.

The implication is clear: the latest-price serving dataset is small; the append-oriented history is large and continuously growing. They should not automatically receive the same storage treatment.

The curated CamelCamelCamel-style design makes the same workload distinction: price history is a time-series workload with sustained writes, time-range reads, and linear long-term growth. It also notes that compression, retention tiers, and aggregation can materially change the storage curve. We will turn that insight into concrete datastore and retention decisions in the next module.

One final sensitivity check makes the storage risk vivid. If the interval changes from one hour to fifteen minutes, every collection-derived number is multiplied by four:

• $192$ million attempted checks per day;
• roughly $182$ million valid observations per day;
• about $18\text{ GB}$ logical raw history growth per day; and
• four times the merchant API load and fetch bandwidth.

Freshness is therefore not merely an SLO. It is the principal capacity dial.

---

Identify the dominant constraints and state the design consequences

A senior answer should end with interpretation, not a pile of arithmetic. For this scenario, the dominant constraints are:

1. Third-party collection capacity and merchant-specific rate limits
   The system needs roughly $1,700$ peak checks per second to pursue hourly freshness for two million listings. A large merchant can exceed its quota even while the global rate appears manageable. The scheduler must later enforce per-source limits, prioritize watched listings, spread work with jitter, and apply backpressure.

2. Growing time-series storage and write path
   The service writes about $45.6$ million valid observations per day. At three-year retention, fully granular history reaches roughly $5\text{ TB}$ logical before replication and operational overhead. This drives a separation between small current-price state and append-heavy history, plus explicit retention, compression, and aggregation policies.

3. Alert-evaluation fan-out and hot listings
   The global estimate of $400$ peak evaluations per second looks comfortable, but it hides skew. One viral product can have orders of magnitude more watches than average. Alert state must be retrieved by listing, processed asynchronously, and protected from a single hot partition.

4. User-facing reads are comparatively tractable
   Roughly $1,160$ peak read QPS and a few megabytes per second of API response traffic justify stateless API scaling, caching of current prices, and pagination of history. They do not, by themselves, justify treating the API tier as the hard problem.

A concise interview summary might sound like this:

I assume one million daily active users, five million active watches, and two million distinct watched listings. Deduplicating watches gives $48$ million hourly price checks per day, about $556$ checks per second average and $1,700$ at peak. With a $95\%$ validation success rate, ingestion handles about $46$ million observations daily. At $100$ bytes each, that is about $1.7$ terabytes of logical history per year before replication. The main risks are merchant quotas and collection freshness, then time-series retention and alert hot spots; the approximately $1,200$ peak API read QPS is relatively straightforward with caching and horizontal API capacity.

---

Key takeaways

Capacity estimates for a price tracker begin with explicit behavioral assumptions, then separate user actions from collection work:

• Deduplicate watches into distinct listings to check; this model reduces external work by the average watches per listing.
• A freshness interval directly determines price-check QPS, observation writes, bandwidth, worker concurrency, and historical storage growth.
• In this working model, collection reaches about $1,700$ peak checks per second, while user reads reach about $1,160$ peak QPS.
• Third-party rate limits and source latency are more constraining than raw collector server count.
• Price history, not current-price state or watch records, dominates durable storage over time.
• Alert notifications may be rare, while alert evaluation can still suffer from fan-out and hot listings.

Next, you will use these access patterns and volumes to define service contracts and the identity model for canonical products, merchant listings, watches, current prices, and paginated price history.