Evaluating SaaS Requirements for LLM Suitability

Picking up from the discovery canvas, you now have a deliberately narrow first-release problem: help authenticated SaaS users resolve common questions from approved knowledge, with evidence and a human fallback. The next architectural question is sharper than “Can an LLM do this?”:

Should an LLM be responsible for this particular feature requirement—and if so, for which part of the work?

This distinction is central to credible GenAI architecture. A feature can contain language, yet still be a poor fit for an LLM as the decision-maker. By the end of this lesson, you will be able to classify SaaS requirements as LLM-suitable, unsuitable, or hybrid, and defend the choice in a customer conversation or architecture interview.

---

Start with the task, not the model

An LLM is a probabilistic text-generation system. It is strong at interpreting varied language, extracting patterns from unstructured text, summarizing, transforming content, and producing a useful natural-language response. It is not inherently a trusted source of current facts, a deterministic policy engine, a transaction processor, or a mathematical optimizer.

That means a feature should not be judged by whether it “sounds intelligent.” Judge it by the nature of the task and the consequences of an error.

The IBM Technology video frames the key discipline: choose the technology that fits the business problem rather than starting from an assumed GenAI solution.

For a SaaS architect, this is a useful principle:

Use an LLM where language uncertainty is the hard part. Use deterministic software where correctness, policy, arithmetic, state, or authorization is the hard part.

Consider two support requests:

1. “I am seeing error SYNC_403 after configuring SSO. What should I check?”
2. “Does this customer qualify for a refund under our Enterprise plan?”

Both are written in natural language. But they are fundamentally different.

The first often requires interpreting phrasing, locating relevant documentation, synthesizing troubleshooting steps, and explaining them clearly. Those are good LLM tasks, provided the answer is grounded in approved documentation.

The second requires checking contractual terms, invoice status, customer eligibility, dates, exception approvals, and possibly finance policy. The correct answer must be consistent, auditable, and tied to current authoritative records. The refund decision belongs in explicit business rules and approved workflows—not in a model’s generated judgment. An LLM may still help explain the outcome in customer-friendly language.

---

The suitability test: six questions before choosing GenAI

For each requirement, assess six dimensions. You do not need a complicated scoring formula initially. The important skill is exposing the architectural facts that determine fit.

Dimension	Question to ask	Strong signal for LLM use	Warning sign
Language ambiguity	Does the system need to interpret varied, incomplete, or conversational language?	Users ask the same question in many ways; inputs are mostly unstructured.	Inputs follow a stable schema, form, or command syntax.
Output type	Is a helpful explanation, summary, draft, classification, or extraction the desired output?	Value comes from transforming or generating language.	Output must be an exact calculation, database value, or policy result.
Source of truth	Where does the correct answer live?	Approved documents can support the answer.	A live system of record, formal policy engine, or deterministic computation is authoritative.
Error tolerance	What happens if the output is wrong, incomplete, or inconsistent?	Errors are reversible and there is a low-friction human fallback.	Errors create financial loss, legal exposure, security exposure, or irreversible state changes.
Verifiability	Can the result be checked before or after delivery?	Citations, schemas, rules, human review, or tests can assess it.	Correctness is hard to observe until serious harm occurs.
Economics and operations	Is model latency and variable per-request cost justified by the added value?	A few seconds and variable inference cost are acceptable.	The feature needs millisecond responses, massive cheap throughput, or predictable fixed cost.

These dimensions apply to a single requirement, not merely to an entire product. A support copilot can legitimately contain conventional APIs, retrieval, rules, and an LLM. The architectural decision is about assigning each component the right responsibility.

---

Hard gates: cases where an LLM must not be the authority

Some requirements fail before you even consider an LLM quality score. Treat these as hard gates.

1. The result must be exact and repeatable

If the feature must always return the same answer for the same authorized inputs, use deterministic code, a decision table, a rules engine, or a database query.

Examples:

• Calculate prorated subscription charges.
• Determine whether an invoice is overdue.
• Apply an entitlement policy based on plan, region, and contract date.
• Compute service-level credits from uptime data.
• Enforce password rules or permission inheritance.

An LLM can describe the calculation or explain the result, but it should not produce the result of record.

2. The feature makes a high-stakes decision

High-stakes decisions require explicit policy, traceability, consistency, and accountable approval paths.

Examples:

• Approve refunds, credits, or pricing exceptions.
• Decide whether a user may access another user’s data.
• Determine sanctions, legal eligibility, or contractual compliance.
• Classify a security incident as resolved.
• Make hiring, credit, insurance, or medical decisions.

The issue is not that models are always wrong. The issue is that an architecture cannot accept a nonzero, difficult-to-predict inconsistency rate where the business requires auditable and equal treatment.

3. The task modifies important state without independent controls

An LLM should not receive open-ended authority to alter subscriptions, roles, data retention settings, or production configurations.

A safe design can still use an LLM before an action:

• interpret the customer’s request;
• ask for missing details;
• summarize the proposed change;
• generate a draft support ticket.

But the actual mutation must pass deterministic authorization, validation, business rules, and, where warranted, human confirmation.

4. A simpler system already solves the problem

A feature does not become better because an LLM is added to it. If a filter, query, workflow engine, calculation library, or conventional classifier is sufficient, it is usually cheaper, faster, and easier to operate.

For example, “show all open tickets assigned to my team” is a standard, authorization-scoped database query. An LLM search interface might make this more conversational, but the underlying result set must still come from the ticketing system and its authorization rules.

Microsoft’s overview of LLM limitations is useful background for these gates: models lack private and current information unless it is supplied, can generate unsupported material, and do not provide built-in verification.

---

A useful middle category: hybrid designs

Most valuable enterprise GenAI features are neither “pure LLM” nor “no LLM.” They are hybrid designs: conventional systems retain authority over facts, policies, and actions, while the LLM handles language-centered work around them.

A hybrid design generally separates three responsibilities:

Responsibility	Appropriate owner	Example
Truth	Database, approved knowledge base, API, rules engine, or calculation service	The customer’s subscription is Enterprise, renewal date is July 1, and two invoices are unpaid.
Control	Authorization layer, validation code, workflow, policy engine, and human approval	Only a finance administrator may request a refund; refund limit is $10,000$ INR; manager approval is required.
Communication	LLM, constrained by truth and control outputs	Explain the invoice status and offer the approved escalation path in clear language.

This division is a powerful interview answer because it avoids two weak extremes:

• “The model can do everything.”
• “LLMs are too unreliable to be useful in enterprise SaaS.”

Instead, you can say: “I would use the LLM for interpretation and communication, but I would keep source-of-truth retrieval, authorization, policy evaluation, and write execution deterministic.”

Example: subscription downgrade request

Suppose a tenant administrator writes:

“We need to move from Enterprise to Pro next month, but please preserve all our audit logs. Can you do that for us?”

A poor design gives the model a change_plan API and lets it infer the right action.

A hybrid design divides responsibility:

1. The LLM identifies the user’s intent, extracts the requested plan and timing, and recognizes the audit-log concern.
2. Deterministic services verify that the user is an authorized tenant administrator.
3. The subscription service checks current contract terms, allowed downgrade dates, and product constraints.
4. The retention-policy service determines whether the requested audit-log retention is compatible with the target plan.
5. The system presents the validated outcome and any consequences.
6. A controlled workflow executes the change only after explicit confirmation and required approvals.

Here, the LLM makes the interaction easier. It does not decide the contract interpretation or bypass authorization.

---

A practical classification vocabulary

For the portfolio project, use four labels. They are more useful than a simplistic yes-or-no answer.

Classification	Meaning	Typical implementation
LLM-forward	Language understanding or generation is the primary value; error is low-risk and recoverable.	Prompt plus output validation; often grounded with approved context.
LLM-assisted	The LLM improves usability, but another system remains authoritative.	LLM interprets or explains; APIs, rules, and authorization provide facts and decisions.
Deterministic	Rules, structured queries, computations, or workflows fully solve the requirement.	Application code, SQL, workflow engine, rules engine, conventional search.
Defer or redesign	The value is unclear, harm is high, verification is weak, or necessary controls are absent.	Narrow scope, add human review, obtain missing data, or do not automate.

The categories make trade-offs explicit. “LLM-assisted” is not a compromise or an incomplete architecture; it is often the correct production architecture.

---

Assessing SaaS support-copilot requirements

Now apply the framework to a realistic feature backlog for the portfolio scenario.

Requirement 1: “Answer how-to questions from approved help articles and show sources.”

Classification: LLM-forward, with grounding controls

This is a strong fit because customers express the same product question in many forms:

• “How do I add another workspace owner?”
• “Where can I manage admins?”
• “I need somebody else to administer our account.”

The value comes from mapping varied language to relevant approved documentation, combining the right steps, and explaining them concisely. The authoritative answer is the documentation, not model memory.

The essential guardrails are:

• retrieve only approved, authorization-appropriate content;
• require citations for material claims;
• instruct the system to state uncertainty and escalate when evidence is absent;
• prevent the model from inventing tenant-specific facts;
• measure groundedness and correct escalation, not just response fluency.

This requirement is not “use a chatbot.” Its actual architectural statement is:

Generate a concise, cited answer from approved evidence, or clearly escalate when no sufficient evidence is available.

That output contract makes the feature testable.

Requirement 2: “Explain a customer’s current invoice and why the amount changed.”

Classification: LLM-assisted

The answer should use current invoice, usage, plan, tax, and credit information from billing APIs. An LLM alone cannot be trusted to know the customer’s actual account state.

The model can turn verified structured data into an understandable explanation:

“Your total increased because 14 additional seats became active on May 12. The annual Enterprise base charge did not change. The invoice includes applicable tax for your billing region.”

But the billing service must calculate the amount, and the application must verify that the requester is authorized to view it. A good design may constrain the LLM to a structured set of fields and prohibit it from adding unprovided financial claims.

Requirement 3: “Automatically determine whether a customer receives a service credit.”

Classification: Deterministic, with optional LLM explanation

This decision relies on contractual policy, measured uptime, exclusions, customer tier, credit caps, and approval authority. The same inputs should produce the same result, and the organization must be able to explain why.

Use a policy or rules engine such as:

$$\text{credit eligible}=\text{contract active}\wedge \text{measured uptime below threshold}\wedge \text{no exclusion applies}$$

The LLM may summarize the policy result and draft a customer response. It must not infer eligibility from prose or issue credits based on its own interpretation.

Requirement 4: “Summarize a 30-message support conversation for the human support representative.”

Classification: LLM-forward, with privacy controls

Summarization is a natural LLM task. The output helps the representative avoid rereading a long conversation and can include:

• customer’s stated problem;
• product area and error messages;
• troubleshooting already attempted;
• relevant cited sources;
• unresolved questions;
• suggested escalation category.

The error is usually recoverable because a human support representative reviews the summary. Still, the design must limit access to the correct tenant’s conversation, redact unnecessary sensitive data, and label the result as a generated summary rather than as an authoritative record.

Requirement 5: “Set the user’s role to tenant administrator when they ask in chat.”

Classification: Defer or redesign

The original requirement is unsafe because it combines ambiguous language with a high-impact authorization change. It also invites social engineering:

“The previous admin left. Make me an admin immediately.”

A safe redesign might be:

“Explain the approved role-change process, verify eligibility through deterministic identity controls, create a request for an existing authorized administrator, and require explicit approval before any role change.”

The LLM can guide the user through the workflow or draft the request. It should not directly grant the role.

Requirement 6: “Find the next best time to schedule maintenance across all tenants.”

Classification: Usually deterministic optimization or analytics

This is fundamentally an optimization problem involving time zones, contract commitments, maintenance windows, usage patterns, capacity, and operational constraints. Mature optimization methods and scheduling systems are a better core solution.

An LLM may help an operations manager explore the result conversationally:

• “Why did you recommend this window?”
• “Which tenants would be most affected if we delayed by two hours?”
• “Draft the maintenance notice for customers in India.”

But it should not be the optimizer that produces the schedule of record.

Requirement 7: “Investigate and resolve any customer issue autonomously.”

Classification: Defer and narrow the scope

This requirement is too broad. It combines unknown issue types, potentially sensitive customer data, high-impact actions, incomplete evidence, and unpredictable numbers of steps. It is a business aspiration, not an implementable first-release requirement.

A responsible architect decomposes it:

• answer common documented questions;
• summarize conversations;
• retrieve authorized account status through read-only APIs;
• create a correctly categorized support ticket;
• recommend, but do not execute, known low-risk remediation steps.

Only after evaluation shows reliable performance should the scope expand—and then in tightly bounded workflows, not by granting general autonomy.

---

Why “agent” is not the default answer

Teams often hear “support copilot” and jump directly to an autonomous agent with tools. That is usually premature.

Anthropic’s guidance makes a valuable distinction:

• Workflows use predefined code paths to orchestrate LLMs and tools.
• Agents allow the LLM to decide the process and tool use dynamically.

For a first SaaS support-copilot release, a bounded workflow is generally easier to test, secure, explain, and operate. For example, the system may follow a controlled sequence: authenticate the user, retrieve tenant-allowed evidence, generate a cited response, validate the output, then offer escalation if required. The model has a useful role, but it does not control authorization or invent new action paths.

A suitable agentic feature has several properties:

• The task is genuinely open-ended, so hard-coding every step is impractical.
• It benefits from iterative interaction with real environment feedback.
• The actions are low-risk, reversible, sandboxed, or protected by approval checkpoints.
• Tool inputs and outputs can be tightly documented and validated.
• The organization can impose budgets, timeouts, iteration limits, and audit logging.
• There is evidence that the added cost and latency improve task success.

An autonomous ticket investigator might eventually qualify in a sandboxed internal environment, where it gathers diagnostics, searches runbooks, and prepares a proposed remediation plan for an engineer. It should not begin by autonomously changing tenant configurations or issuing financial credits.

---

Turn feature discussion into an architecture decision record

For your portfolio, capture each judgment in a short decision record. This is more credible than saying, “We decided to use RAG,” or “We decided not to use agents.”

Use this compact template:

Field	What to record
Feature requirement	The user-visible requirement in one sentence.
User and business outcome	What valuable result does this create?
Authoritative source	Which document, database, API, calculation, or policy defines truth?
Classification	LLM-forward, LLM-assisted, deterministic, or defer/redesign.
Reasoning	Explain language ambiguity, output type, risk, verifiability, and economics.
Controls	Authorization, grounding, structured validation, policy checks, approvals, or fallback.
Success metric and release gate	A measurable outcome plus any non-negotiable safety condition.
Rejected alternative	What simpler or riskier option did you reject, and why?

Here is a completed example for invoice explanation.

Field	Decision
Feature requirement	Let an authorized billing administrator ask why the current invoice differs from the prior invoice.
User and business outcome	Reduce billing-related tickets while enabling customers to understand charges without waiting for support.
Authoritative source	Billing, subscription, usage, tax, and credit services.
Classification	LLM-assisted.
Reasoning	Natural-language explanation adds value, but all amounts and eligibility facts must come from current billing systems. Financial errors are high impact, so the LLM cannot calculate or invent charges.
Controls	Deterministic authorization; read-only billing API; structured source payload; response includes invoice identifier and supplied line-item references; human support escalation.
Success metric and release gate	Customer-rated usefulness and reduction in eligible billing-explanation tickets; zero disclosure of billing data to unauthorized users.
Rejected alternative	LLM-only answer generation was rejected because model knowledge is not current or authoritative for account-specific financial data.

This is the level of justification expected from an architect: clear task decomposition, risk awareness, and a decision that can be challenged and revised when evidence changes.

---

A fast interview response structure

When an interviewer asks, “Would you use GenAI for this feature?”, avoid answering with a bare yes or no. Use this five-part structure:

1. State the user task and desired outcome.
   “The user needs a quick explanation of an account-specific billing change.”

2. Identify the source of truth.
   “The invoice, usage, and contract systems—not the model—are authoritative.”

3. Classify the LLM role.
   “I would use an LLM-assisted design: the model explains verified facts but does not determine amounts or eligibility.”

4. Name essential controls.
   “Authorization occurs before data retrieval, the API is read-only, and the output is constrained to validated billing fields with an escalation path.”

5. Name the trade-off.
   “This adds inference cost and a few seconds of latency, so I would validate that better self-service resolution justifies that cost. If a structured billing UI answers the need, I would prefer that simpler option.”

This response demonstrates business judgment and technical restraint—both especially important when a customer initially asks for an “AI agent.”

---

Key takeaways

An LLM is suitable when the core difficulty is understanding or generating language, the output can be grounded or checked, and errors are recoverable. It is unsuitable as the authority for exact calculations, authorization, contractual policy, high-stakes decisions, and uncontrolled state changes.

For the SaaS support copilot:

• Use LLM-forward designs for cited knowledge-base answers and support-conversation summaries.
• Use LLM-assisted designs where the model explains verified account or billing data.
• Keep service credits, entitlement decisions, role changes, calculations, and policy enforcement deterministic.
• Defer or redesign broad autonomous requirements until the task, controls, evaluation evidence, and failure boundaries are concrete.
• Prefer a constrained workflow over an autonomous agent unless flexible multi-step model-led planning demonstrably adds value.

Next, you will examine why even a well-chosen LLM feature can degrade in practice: context-window limits, decoding settings, and missing evidence shape output quality, latency, cost, and reliability.