Clarifying ML System Design Requirements

Hello, and welcome to this MLOps system-design interview sprint. Over this module, you will practice turning an ambiguous ML product prompt into a credible production design—then defending its trade-offs as an engineering manager.

This first lesson focuses on the opening five minutes of the interview. Before drawing architecture, establish a design contract: what product outcome matters, what the model must predict or generate, how success will be measured, what scale the system faces, and which constraints are non-negotiable. A strong design built for the wrong problem is still a weak interview answer.

By the end, you should be able to lead this clarification phase concisely, state assumptions when information is unavailable, and summarize the agreed problem before proceeding to sizing and architecture.

---

The goal: turn a vague prompt into a design contract

Consider a prompt such as: “Design a personalized content-ranking system.” It sounds precise, but it omits almost every decision that determines the system:

• Which surface: home feed, search results, notifications, or all three?
• What does “personalized” mean for new, logged-out, or privacy-restricted users?
• Is the system maximizing clicks, meaningful consumption, retention, creator health, or revenue?
• Does it rank a few hundred candidates in 100 ms, or score millions of items overnight?
• Is a stale but available feed acceptable during an outage?

Do not silently decide these questions for the interviewer. Instead, use questions to expose the decisions, then either confirm answers or state explicit assumptions. Your goal is not to exhaustively gather requirements; it is to identify the few requirements that would materially change the design.

A useful opening sounds like this:

“Before I propose an architecture, I’d like to align on the product goal, the model’s decision and target, how we measure success, expected workload, and the key reliability or latency constraints. I’ll state assumptions where needed, then summarize the scope before moving into the design.”

This signals structured thinking and prevents the common failure mode of jumping directly to a feature store, model-serving stack, or neural architecture.

---

1. Establish the product objective and scope

Start with the ideal product outcome, stated in human or business terms. It should describe the change the product seeks, not the model technique.

Compare these statements:

Too vague	Product objective
“Build an engagement model.”	“Help listeners quickly find music they will choose to play and return for.”
“Detect fraud.”	“Reduce fraudulent payment losses without blocking legitimate customers.”
“Make a support chatbot.”	“Resolve common customer issues accurately while escalating unsafe or uncertain cases to human agents.”

The product objective gives you a basis for rejecting inappropriate technical choices later. For example, maximizing clicks may conflict with helping users find valuable content; a fraud model that blocks too many valid purchases may harm the business even if its offline classification accuracy is high.

Clarify scope immediately after the objective:

1. User and surface: Who uses the feature, and where does it appear?
2. Primary use case: Which decision is in scope for this interview?
3. Exclusions: What adjacent functions are explicitly out of scope?
4. Fallback behavior: What should a user receive if personalization, data, or the model is unavailable?

For a feed-ranking prompt, you might ask:

“Should we focus on the signed-in home feed, or also notifications and search? Do we need to support new users? If personalized ranking is unavailable, is a globally popular or editorial feed an acceptable fallback?”

This is managerial as well as technical reasoning: it converts an expansive roadmap into a deliverable system boundary.

---

2. Translate the product outcome into an ML decision and prediction target

An ML model should not be described merely as “a recommender” or “a classifier.” State exactly what decision the system supports, the unit it operates on, its input, and its output.

For example:

Layer	Feed-ranking example
Product objective	Help users find content worth consuming
System decision	Order candidate posts for a particular feed request
Prediction unit	A user–post pair at a particular time and context
Model output	A score estimating the chance of a valuable interaction
Product action	Display the highest-ranked eligible posts

This distinction matters because the ideal outcome is often not directly observable. “Useful,” “satisfying,” “safe,” and “healthy” are product concepts, not clean labels in a training table. You will frequently use a proxy label: an observable behavior that imperfectly stands in for the desired outcome.

For a video or social feed, possible proxies include:

• click or play;
• watch time or completion;
• like, save, share, or comment;
• an explicit “not interested” signal;
• return visits or longer-term retention.

Each proxy has incentives embedded in it. Clicks can reward clickbait; watch time can favor long content; shares can mean admiration or outrage. In an interview, name the risk rather than pretending the label is neutral:

“I would begin with a composite engagement target, but I would validate it against downstream retention and negative-feedback rates. Optimizing clicks alone creates a clear clickbait risk.”

Choose the right output form

A prediction target should follow the action the product needs to take:

• Classification when the decision is a category, such as spam versus not spam, approve versus review, or likely-to-churn versus not likely-to-churn.
• Regression when the output is a continuous quantity, such as expected delivery time or expected demand.
• Ranking or scoring when the system must order many candidates for a user or context.
• Generative output when the product needs newly produced text, images, audio, or structured content.

Do not overcommit to model architecture during clarification. Saying “we need a transformer” before identifying the prediction and constraints is premature. A stronger statement is:

“The core ML task is to estimate the probability of a valuable interaction for each eligible item, then rank items using that score subject to policy and diversity constraints.”

The phrase subject to constraints matters. In production, the highest-scoring item may still be ineligible because of safety policies, inventory rules, legal restrictions, duplicate suppression, or user settings.

---

3. Define success as a stack of metrics, not one number

A good clarification phase distinguishes business success, model quality, operational quality, and guardrails. These metrics answer different questions.

Metric layer	Question it answers	Feed-ranking example
Product metric	Did the product create value?	Meaningful sessions per active user; 30-day retention
Model metric	Does the model discriminate well offline?	Ranking quality, calibration, precision at a cutoff
Online experiment metric	Did the new system improve behavior causally?	Increase in meaningful plays versus a control group
Guardrail	What harm is unacceptable?	Negative feedback, content-policy violations, creator concentration
Service metric	Does the system meet its operational promise?	p95 feed-ranking latency, error rate, availability
Cost metric	Is the gain economically viable?	Compute cost per thousand ranking requests

A model can improve an offline metric while harming the product. It may learn historical bias, overfit to short-term clicks, or make the experience less diverse. Conversely, a modest offline improvement can be commercially valuable at large scale if it moves the product metric and costs little.

In an interview, propose:

• one primary product metric;
• one or two online or model diagnostics;
• two guardrails that prevent obvious harm.

For instance:

“I would use meaningful consumption per daily active user as the primary success metric, validated through an A/B test. I would monitor ranking quality and score calibration as diagnostics. Guardrails would include negative-feedback rate and engagement distribution across relevant user segments, so gains do not come from degrading experience for new or low-activity users.”

Also clarify the measurement horizon. A click is measurable immediately; retention may take weeks. The system can optimize a near-term proxy but should be judged against a longer-term outcome when the product risk warrants it.

---

4. Elicit workload before estimating it

The next lesson will turn assumptions into order-of-magnitude estimates. At this stage, your job is to collect the variables that drive those estimates.

For an online ML service, ask about:

• Audience size: daily and monthly active users; geographic distribution.
• Request rate: requests per active user per day and peak-to-average traffic ratio.
• Serving shape: one prediction per request, or hundreds or thousands of candidates scored per request.
• Payload: input features, candidate count, output size, and model size.
• Freshness: how rapidly user behavior, inventory, or item information must affect predictions.
• Write volume: events, labels, and feature updates generated by the product.
• Growth: expected growth over the next year and any predictable events that produce spikes.

The key point is that “10 million users” does not specify serving load. A system used once per day has a fundamentally different request profile from a feed refreshed repeatedly during a session. Likewise, a ranking request that scores 500 candidates is different from a single fraud score, even if both receive the same request rate.

Phrase assumptions in a way that makes them easy to revise:

“For the initial design, I’ll assume 10 million daily active users, an average of five feed loads per user per day, and a tenfold traffic peak. I’ll also assume each request ranks roughly 500 pre-filtered candidates. I’ll quantify the serving implications next.”

This is much stronger than choosing arbitrary infrastructure capacity, because every later estimate remains traceable to a stated assumption.

---

5. Convert non-functional requirements into priorities

Non-functional requirements define the operating envelope of the system. They are often where ML designs differ most sharply.

Ask for a number, a measurement convention, and a priority whenever possible.

Latency

Do not ask only, “What is the latency requirement?” Clarify:

• Is the target end-to-end user latency or model-serving latency alone?
• Which percentile matters: p50, p95, or p99?
• Does the system rank a complete result set, or can results stream progressively?
• Is a slower first request acceptable for a new user?

A reasonable requirement is: “The feed must return within 300 ms at p95 end-to-end; model scoring receives no more than 80 ms of that budget.”

Availability and graceful degradation

Availability is a product decision. Ask what a user should experience when dependencies or the model fail:

• Is a cached prediction acceptable?
• Can the system serve popular, rule-based, or chronological content?
• Is a stale recommendation acceptable, and for how long?
• Is it ever safer to abstain or require human review?

For a recommendation feed, a non-personalized fallback can preserve availability. For high-risk fraud approval, an unavailable model may require a conservative review path instead.

Freshness and consistency

ML systems often trade freshness against cost and reliability. Clarify:

• How old may user or item features be?
• Must a user’s action affect the next recommendation immediately?
• Are slightly stale predictions acceptable?
• Must training and serving use exactly the same feature definitions?

A news-ranking system may need minute-level freshness. A weekly churn model likely does not. The answers influence whether later designs favor batch, streaming, asynchronous, or synchronous inference.

Other constraints worth surfacing

Depending on the product, ask about:

• Cost: Are specialized accelerators acceptable? Is there a per-request budget?
• Privacy and access: Does the system process personal, sensitive, or regulated data? Are certain features prohibited?
• Safety and policy: What content, decisions, or outputs must never be produced?
• Fairness: Which user or provider segments require comparable service quality or error rates?
• Explainability and audit: Must decisions be reviewable by users, operations staff, or regulators?
• Regional deployment: Are there data-residency or language requirements?

Avoid treating all of these as generic boxes to tick. Select the constraints implied by the product. A healthcare triage system needs a very different discussion from a music feed.

---

A five-minute interview sequence

Use this sequence as a compact rehearsal checklist:

1. State the intended clarification categories. Signal that you will cover objective, target, metrics, workload, and constraints.
2. Establish product outcome and scope. Identify users, surface, primary use case, exclusions, and fallback expectations.
3. Specify the ML decision. State the unit of prediction, output form, decision horizon, and how the product consumes the output.
4. Challenge the label. Identify the observable proxy for the desired outcome and one plausible misalignment risk.
5. Agree on success. Name a business metric, an online evaluation approach, and meaningful guardrails.
6. Collect workload drivers. Ask about active users, request patterns, candidates per request, freshness, and growth.
7. Prioritize non-functional constraints. Clarify latency percentiles, availability and fallback, freshness, cost, privacy, and safety as relevant.
8. Summarize assumptions aloud. Invite correction before beginning estimates and architecture.

For the running feed example, your summary might be:

“To confirm: we are designing the signed-in home-feed ranking service, not search or notifications. The product goal is meaningful consumption and longer-term retention. For each feed request, the model ranks eligible posts using predicted valuable engagement, while monitoring negative feedback and segment-level outcomes. The service must handle a large, bursty global workload, return within the agreed p95 latency budget, and fall back to a non-personalized feed if real-time scoring is unavailable. I’ll now estimate the workload and use those assumptions to choose a serving approach.”

That single summary creates a stable foundation for everything that follows.

---

Key takeaways

A strong ML system-design answer starts by framing the problem, not by choosing infrastructure or a model.

• Separate the product objective from the model prediction.
• Define the model’s decision unit, output, label or proxy label, and decision horizon.
• Use a metric stack: product success, model diagnostics, online validation, operational metrics, and guardrails.
• Gather workload drivers before performing capacity calculations.
• Make latency, availability, freshness, cost, privacy, and safety requirements concrete and prioritized.
• State assumptions explicitly, then summarize the agreed scope before designing.

Next, you will use these clarified assumptions to produce fast, defensible estimates for throughput, latency, availability, data volume, and compute cost.