Choosing the Right AI Approach for SaaS Scenarios

Welcome back. Last lesson treated reliable generation as an evidence and context-management problem: a support copilot should work from a bounded set of authorized evidence, use low-variation settings for operational guidance, and explicitly abstain or escalate when evidence is missing.

That prepares the central architecture decision for this lesson. When a SaaS requirement calls for “AI,” you need to identify what is actually missing: clearer task instructions, access to documents, access to live systems, or more consistent learned behavior. You will select among prompt-only generation, retrieval-augmented generation (RAG), tool use, and fine-tuning, and justify the decision in a way that would stand up in a customer conversation or system-design interview.

---

Start with the failure, not the technique

A common but expensive misconception is that GenAI techniques form a mandatory maturity ladder: prompt engineering first, then RAG, then fine-tuning, then agents. In practice, these techniques address different failure modes. Adding the wrong one can increase latency, cost, security exposure, and operational burden without improving the result.

The first question is therefore not, “Which GenAI capability should we add?” It is:

What must be true for this task to produce a correct and safe outcome?

For the support copilot, consider four distinct needs:

Need	Best initial intervention
The model needs clearer instructions, examples, or an output format	Prompt-only generation
The answer depends on private, changing, or extensive unstructured documentation	RAG
The answer requires current structured data or an action in an external system	Tool use
The model repeatedly fails to follow a stable task pattern despite prompting and examples	Fine-tuning, after evaluation

All four approaches still use prompts. Prompting is the control layer around any model call; RAG, tools, and fine-tuning change what information or capabilities the model has available.

Before deciding, estimate the consequence of a bad answer. A weakly worded draft response can be reviewed by a human. An incorrect entitlement statement, an exposed tenant record, or an unintended account change has a far higher cost. That difference determines how much retrieval, validation, authorization, human approval, and evaluation the architecture needs.

The central diagnostic distinction is:

• An in-context information problem occurs when the model lacks the current, private, tenant-specific, or otherwise relevant material needed for this particular request.
• A learned behavior problem occurs when the model has, or can be supplied with, the needed information but does not reliably perform the task in the required style, format, or decision pattern.

RAG addresses the first category. Fine-tuning can address the second. Tool use addresses a different category again: the application needs a trustworthy interaction with another system.

---

The four choices, precisely distinguished

The decision tree below is a useful starting heuristic. It separates tasks that require external information from tasks that can be handled with the model’s existing knowledge and a well-designed prompt. It also highlights two important signals: whether data is real-time and whether a specific output behavior is needed.

The image’s “domain-specific output” branch needs careful interpretation. Domain-specific facts and domain-specific behavior are not the same thing.

• “What does our current SSO certificate-rotation guide say?” is a knowledge problem, usually solved by RAG.
• “Classify every support ticket using our internal taxonomy in exactly this JSON structure” is potentially a behavior problem. It may justify fine-tuning only after a prompt-and-examples baseline is evaluated.

1. Prompt-only generation: improve the use of existing capability

Use prompt-only generation when the model can perform the task using:

• the user’s supplied input;
• stable, broadly available knowledge that does not require a guarantee of freshness;
• a small amount of directly supplied reference text;
• clear instructions, examples, constraints, and an output contract.

For a support copilot, prompt-only generation is appropriate for requests such as:

“Rewrite this draft response in a concise, empathetic tone. Preserve all technical facts and do not add new troubleshooting steps.”

The source of truth is the user-provided draft. The model is transforming content, not discovering private product policy or making a live account decision.

Prompt-only generation is attractive because it has the smallest architecture footprint: one model call, no indexing pipeline, no tool executor, and no training run. It is also the right baseline for almost every task because it makes expected behavior testable.

It is not sufficient when the answer must be grounded in facts that the model was not trained on, facts that may have changed, or private tenant data. No amount of clever prompt wording can reliably inject fresh facts that are absent from the prompt.

2. RAG: bring the right documents into this request

RAG retrieves relevant passages from an approved knowledge source, adds those passages to the prompt as grounding context, and asks the model to answer from that evidence.

Use it when the task needs answers from:

• internal product documentation;
• release notes and current troubleshooting guides;
• private policy documents;
• tenant-specific knowledge-base content;
• a document collection too large to include in every prompt;
• sources that need citations and traceability.

For example:

“Why does SSO setup fail with SYNC_403 after we rotate a certificate?”

The answer should come from current product troubleshooting material, not a model’s general recollection of SAML or SSO. RAG can retrieve the relevant certificate-rotation and error-code passages, then the model can synthesize a concise, cited answer.

RAG is not a guarantee of accuracy. As the previous lesson established, retrieval can return irrelevant, incomplete, unauthorized, or stale passages. It also adds embedding, indexing, retrieval, and prompt-token costs. For this reason, RAG is justified by an actual need for grounded document knowledge, not by a vague desire to make the model “smarter.”

For the portfolio copilot, the expected baseline choice for knowledge-base questions is:

Prompt engineering plus tenant-aware RAG, with citations and an insufficient-evidence fallback.

The later retrieval module will implement that design locally.

3. Tool use: obtain live facts or perform bounded work

A tool is an application-controlled function that the model may request but cannot execute by itself. The tool can query a read-only API, retrieve a record, calculate a value, create a ticket, or invoke another constrained business operation.

Use tools when the answer depends on a live, structured system of record, such as:

• current subscription plan and entitlement;
• current invoice, usage, account, or ticket status;
• current feature-flag configuration;
• an approved action such as creating a support ticket;
• a deterministic calculation or workflow step.

For example:

“Which invoices are overdue for this customer, and what is their total outstanding balance?”

This should not be answered from a RAG index of invoices. The data may change every minute, must be scoped to the authenticated tenant, and may require precise numerical handling. A model can decide that it needs account data, but your application must execute a server-side get_overdue_invoices tool with authorization and argument validation.

Tool use therefore has two layers:

1. Model-mediated decision: based on the request and tool description, the model proposes a structured tool call.
2. Application enforcement: the server validates arguments, enforces tenant authorization, invokes the API, and returns a constrained result.

The model is not the authority. It must never be allowed to choose a tenant ID, bypass authorization, or directly call an internal service merely because it generated plausible arguments.

For a write-capable tool, add a third control: explicit confirmation or approval before the side effect. For example, a copilot may draft the action “create a P1 support case with this summary,” but the user or an authorized workflow should confirm before the system creates it.

Tool use is not automatically an agent

An agent is a broader orchestration pattern in which the model can plan, choose among tools, inspect results, and sometimes repeat this cycle. The decision tree labels real-time needs as “Agent,” but a single bounded tool call does not require a fully autonomous agent.

Start with a direct tool workflow when the sequence is known:

• The user asks for current plan limits.
• The system calls an entitlement API.
• The model presents the validated result.

An agentic approach may be warranted for a more open-ended task such as:

“Investigate why this customer’s onboarding is blocked, check their current configuration and recent error events, then recommend whether an escalation is needed.”

Even then, constrain the allowed tools, limit the number of calls, validate every argument, apply timeouts, and preserve an audit trail. In the initial portfolio prototype, a read-only tool call is the appropriately scoped design; a general-purpose autonomous agent would add risk before it adds evidence.

4. Fine-tuning: teach repeatable behavior, not fresh facts

Fine-tuning adjusts model parameters using a curated training set of input-output examples. It is most useful when evaluation shows that a prompt, examples, and relevant context still cannot make the model perform a stable task consistently enough.

Potential reasons include:

• a highly consistent output style or taxonomy;
• repeated extraction or classification patterns;
• reliable adherence to a specialized response format;
• a need to use a smaller adapted model for a well-bounded task;
• latency or cost constraints that make long prompts impractical.

A plausible SaaS example is high-volume ticket routing:

“Given an incoming support request, assign one of 35 internal issue categories, set the urgency band, identify missing fields, and produce a stable structured response.”

Start with a strong prompt, a clear output contract, and a representative evaluation set. If the model still systematically misclassifies categories or fails the format despite well-chosen examples, and you possess a sufficiently large, high-quality labeled dataset, fine-tuning becomes a defensible option.

Fine-tuning is a poor first choice for:

• current documentation, pricing, policies, or release notes;
• customer-specific facts;
• information requiring citations;
• fast-changing data;
• cases where training examples are sparse, low-quality, or not representative;
• a problem that can be solved by a better prompt, retrieval, or a bounded tool.

A fine-tuned model may memorize or generalize patterns from its examples, but it is not a trustworthy database. It does not provide an auditable source for a claim such as an entitlement rule, nor can it enforce who is authorized to see a record.

---

Make the decision through an architecture lens

For each requirement, use this sequence. It prevents both premature fine-tuning and the equally common mistake of building RAG for every feature.

1. Identify the source of truth.
   Is the required information already in the user’s input? Is it in a private document collection? Is it in a live API or database? Or is the requirement principally a repeatable behavioral pattern?

2. Assess freshness and volatility.
   A two-year-old style guide may be suitable for a prompt or a retrieved document. Current billing status, entitlements, and feature configuration require a tool backed by the authoritative service.

3. Determine whether the task reads or acts.
   Reading unstructured documentation suggests RAG. Querying structured operational data suggests a tool. Any action such as changing configuration, issuing a refund, or creating a ticket requires explicit business controls beyond model reasoning.

4. Separate information failure from behavior failure.
   If the model lacks the answer for this request, retrieve or call a tool. If the model sees correct evidence but repeatedly applies the wrong transformation, classification, or format, investigate prompting and then consider fine-tuning.

5. Choose the minimum sufficient architecture.
   Each added component creates operational responsibilities: document ingestion, indexes, API permissions, prompt-injection defenses, training-data lifecycle, observability, evaluation, and incident response.

6. Validate the hypothesis with examples.
   A decision is not complete until it has expected outputs and failure tests. Later modules will formalize the evaluation dataset and metrics, but start recording representative examples now.

The following examples apply that process to the support-copilot portfolio scenario.

SaaS requirement	Primary choice	Why
Rewrite a customer’s supplied draft in a concise, professional tone	Prompt-only	The model transforms supplied content; no private or current external facts are required
Explain the supported procedure for rotating an SSO certificate	Prompt plus RAG	The answer depends on product documentation that can change and should be cited
State the customer’s current plan, enabled features, and usage limit	Prompt plus read-only tool	The answer requires current, structured, tenant-scoped system-of-record data
Create a support ticket after collecting diagnostics	Tool use with confirmation	The request creates a side effect and requires authorization, validation, and an audit trail
Apply a 35-class internal ticket taxonomy at high volume	Prompt baseline, then possible fine-tuning	This is a stable behavior problem only if evaluation shows prompting and examples do not meet the target
Investigate a vague onboarding failure across docs, configuration, and current error events	RAG plus tools; agentic orchestration only if justified	The task needs both document knowledge and live data; an agent is optional orchestration, not the starting requirement

Notice that combinations are normal:

• Prompt plus RAG is the standard grounded-question-answering pattern.
• Prompt plus tools is appropriate for live account facts and bounded actions.
• Prompt plus RAG plus tools supports a copilot that must explain policy while checking a customer’s actual configuration.
• Fine-tuning plus RAG can be appropriate when the application needs both a learned response behavior and current private knowledge.
• Fine-tuning plus tools may support a narrowly specialized workflow, but tool authorization remains entirely outside model weights.

---

An interview-ready justification for the support copilot

For the current portfolio scenario, avoid proposing fine-tuning as the default solution. You do not yet have the most important prerequisite: a substantial, representative, high-quality dataset showing the expected input-output behavior and an evaluation baseline proving that prompt engineering is insufficient.

A sensible phased recommendation is:

Capability	Initial architecture choice	Reason
Drafting, summarizing, and rewriting user-provided support content	Prompt-only	Fast to prototype, low operational overhead, and readily evaluated
Product how-to and troubleshooting answers	Tenant-aware RAG with citations	Documentation is private or changing; answers need grounding and traceability
Current account, entitlement, usage, invoice, and ticket information	Constrained read-only tools	Structured live data must come from its authoritative system with server-side authorization
Account-changing actions	Tools with confirmation, approval, and audit logging	Side effects require deterministic enforcement and human or workflow controls
Specialized classification or formatting at scale	Fine-tuning only after evaluation	Fine-tuning is an optimization for demonstrated stable behavior gaps, not a knowledge store

A concise customer-facing explanation could be:

“We will use the base model with explicit prompts for drafting and summarization. For product guidance, we will retrieve only authorized, current knowledge-base passages and require citations. For live customer facts, the model will request a constrained API call, while the application enforces tenant authorization and validates every parameter. We will not fine-tune initially because the main need is current evidence, not a demonstrated behavior gap. After collecting evaluation data from the pilot, we can assess whether ticket classification or another high-volume pattern merits model customization.”

This recommendation is technically grounded and commercially sensible. It avoids paying for training and maintaining a customized model before proving that a simpler architecture cannot meet the success criteria.

Record the decision, not just the diagram

Add a short architecture decision record to your portfolio repository for each major GenAI feature. Use these fields:

• Requirement: the user-facing task and its measurable success condition.
• Source of truth: user input, knowledge base, system API, or labeled examples.
• Selected pattern: prompt-only, RAG, tool use, fine-tuning, or a justified combination.
• Why this pattern: freshness, structure of data, action requirement, behavior consistency, and failure cost.
• Controls: tenant isolation, authorization, citation policy, confirmation, output validation, or escalation.
• Rejected alternatives: for example, “Fine-tuning rejected because policies change frequently and citations are required.”
• Evaluation evidence: representative queries, expected outcomes, and the metrics that will prove the decision.

This is valuable interview material because it demonstrates that your architecture decisions are evidence-driven rather than provider-feature-driven.

---

Key takeaways

Prompt-only generation improves how the model applies its existing capabilities to supplied input. It is the right baseline and often sufficient for drafting, transformation, summarization, and clearly specified tasks that do not require fresh private facts.

RAG supplies relevant, authorized document evidence at runtime. Use it for private, changing, or extensive unstructured knowledge where grounded answers and citations matter. It does not replace access control, retrieval evaluation, or a safe missing-evidence response.

Tool use connects the model to live structured data and bounded operations. The model may request a tool call, but application code must validate arguments, enforce authorization, control side effects, and preserve an audit trail. A single tool call is not necessarily an agent.

Fine-tuning changes model behavior through training examples. Consider it only when evaluation demonstrates a stable behavior gap that prompting, examples, retrieval, and tools cannot solve economically. It is not a reliable substitute for current data, citations, or authorization.

Next, you will move from architecture choice to implementation: building a Python interaction with a local or free-tier model that uses system and user messages, supplied grounding context, and an explicit output contract.