From Ambiguous Request to Actionable Copilot Requirements

Hello, and welcome to the first lesson in your GenAI Architect portfolio course. This first module establishes the habit that separates architecture from technology selection: begin with the customer’s workflow, value, constraints, and failure costs—not with a preferred model or framework.

Your portfolio scenario is a multi-tenant SaaS support copilot. By the end of this lesson, you will be able to take a vague request such as “Build an AI chatbot to reduce support tickets” and turn it into a compact discovery canvas containing measurable success criteria, technical constraints, open questions, and a prioritized risk register. This is both a practical artifact for the project and a strong structure for customer-discovery interviews.

---

Start with the problem, not “the chatbot”

A customer request is usually a compressed statement of intent, not a requirements specification:

“Our support team gets too many repetitive questions. Can you add a GenAI chatbot that gives customers instant answers?”

This statement contains a business pain, but it leaves critical issues unresolved:

• Which customers should use it: end users, tenant admins, or support agents?
• Which repetitive questions are actually safe to answer automatically?
• Does “reduce tickets” mean fewer tickets created, faster resolution, lower cost, or better customer satisfaction?
• Is the assistant allowed to access tenant-specific configurations, billing information, or account data?
• What happens when the system is unsure, has no evidence, or sees a malicious prompt?

An architect’s first move is to decompose the request into claims that can be validated. Do not silently turn assumptions into requirements.

Use three labels throughout discovery:

Label	Meaning	Example
Confirmed fact	A stakeholder has verified it or evidence supports it.	Support receives a high volume of password-reset and configuration questions.
Working assumption	Plausible for the prototype, but must be validated.	Most repetitive questions can be answered from approved help-center articles.
Open question	An unknown that could change scope, design, cost, or risk.	Are customer-specific configuration details permitted in the assistant’s context?

This distinction is useful in interviews. It demonstrates that you can make progress amid uncertainty without pretending uncertainty does not exist.

The supplied AI Solution Canvas shows a useful structure: inputs, skills, value proposition, stakeholders, integration, outputs, testing, costs, and saved value. We will adapt it slightly for a multi-tenant GenAI SaaS feature by making success metrics, trust constraints, and risks explicit.

Before filling boxes, establish the current workflow. A plausible support workflow for the portfolio scenario is:

1. A customer encounters a problem in the SaaS product.
2. They search the help center, ask a colleague, or open a support ticket.
3. A support representative searches documentation, account context, and past tickets.
4. The representative responds, requests clarification, or escalates the issue.
5. The ticket is resolved, reopened, or converted into product feedback.

The copilot’s initial purpose is not “answer anything.” It is narrower:

Help authenticated SaaS customers obtain evidence-backed answers to common product and configuration questions, while directing ambiguous, sensitive, or unsupported requests to human support.

That statement names the user, the task, the evidence standard, and the safe fallback. It is a much better starting point for architecture.

---

Identify users, decisions, and the right level of automation

“Customer support” is not one stakeholder. For the first discovery canvas, identify at least these groups:

Stakeholder	Job to be done	What success looks like	What can go wrong
SaaS customer or tenant admin	Resolve a product question quickly and correctly.	Gets a relevant answer with a trustworthy source.	Receives incorrect instructions, exposes data, or cannot reach a human.
Support representative	Resolve complex cases efficiently.	Fewer repetitive tickets and richer context on escalations.	Receives low-quality escalations or loses the ability to correct an answer.
Support operations lead	Manage service quality and cost.	Lower cost per resolved issue without reduced customer satisfaction.	Ticket deflection appears successful while unresolved issues increase.
Tenant security or compliance owner	Protect tenant data and maintain control.	Only authorized information is used and disclosed.	Cross-tenant data exposure, weak auditability, or retention violations.
Product and engineering team	Improve the support experience sustainably.	Clear feedback about documentation gaps and user friction.	A brittle AI feature creates incidents and obscures product defects.

There is a shared value proposition: a customer’s legitimate support need is resolved accurately with appropriate effort and accountability.

That phrasing matters. If you optimize only for “answers sent,” a model can appear successful while generating fluent but unsupported responses. If you optimize only for “tickets avoided,” the system may discourage users from receiving necessary human help.

Decide what the assistant may do

For an early SaaS support copilot, use augmentation with constrained automation.

The system can automate low-risk, repeatable work:

• Find relevant approved documentation.
• Summarize documented setup steps.
• Explain a known error message when the help center covers it.
• Ask a clarifying question when the product area or tenant context is unclear.
• Offer a route to human support when it lacks sufficient evidence.

It should augment rather than replace human judgment for higher-stakes work:

• Refunds, pricing exceptions, contractual commitments, or service credits.
• Account ownership changes and permission changes.
• Security incidents or suspected account compromise.
• Legal, compliance, or data-processing commitments.
• Unclear technical incidents that require investigation.

This boundary is an early scope decision, not a limitation of ambition. It permits a credible first release while preserving a safe expansion path.

---

Turn “success” into a measurable scorecard

A useful metric has five properties:

1. It represents a meaningful outcome, rather than an activity count.
2. It has a precise definition, including numerator, denominator, time window, and exclusions.
3. It has an owner and data source.
4. It has a target or baseline comparison.
5. It has a counter-metric, so a local optimization does not create a harmful outcome elsewhere.

For the copilot, a reasonable North Star is verified self-service resolution rate:

$$\text{Verified self service resolution rate}=\frac{\text{eligible sessions resolved without human support and not reopened within 7 days}}{\text{eligible sessions}}$$

This is more meaningful than “chat sessions” or “messages sent.” It counts only eligible issues, requires that the user did not immediately need human help, and checks whether the apparent resolution held.

However, it is not enough by itself. A customer may abandon the chat because they are frustrated, not because their issue was solved. Therefore pair it with quality, safety, experience, and efficiency measures.

Provisional pilot scorecard

The following targets are reasonable initial acceptance criteria for a portfolio prototype. In a real customer engagement, confirm the baseline and calibrate thresholds with support data before committing to a business case.

Dimension	Success criterion	How to measure it	Initial target
Grounded answer quality	In-scope answers are supported by approved evidence.	Evaluate a representative test set; reviewers check whether each material claim is supported by a cited source.	At least 80 percent fully supported before pilot expansion.
Escalation quality	The assistant escalates when evidence is absent, conflicting, or the request is high risk.	Review test conversations and sampled production sessions against an escalation rubric.	At least 95 percent correct escalation decisions for defined high-risk cases.
Tenant isolation	No user receives knowledge from another tenant.	Automated cross-tenant retrieval and conversation tests.	Zero leakage tolerated.
User experience	Users receive an answer or clear escalation promptly.	Measure end-to-end response latency from submitted message to rendered response.	p95 under 10 seconds for the pilot.
Customer value	Eligible support demand shifts to successful self-service.	Compare verified self-service resolution rate and eligible-ticket volume with a pre-pilot baseline.	Target set after baseline; a provisional goal is a 20 percent reduction in eligible repetitive tickets.
Support-team value	Escalations include useful context rather than creating extra triage work.	Sample escalated tickets and ask representatives to rate context completeness.	At least 4 out of 5 average usefulness.
Cost discipline	The feature creates net value rather than unbounded model spending.	Track model, retrieval, and infrastructure cost per resolved eligible request.	Cost threshold set against current support cost per resolved request.

Notice the non-negotiable nature of tenant isolation. It is not acceptable to average away data leakage with an overall quality score. Some requirements are optimization targets; others are release gates.

Add counter-metrics

Every primary metric needs a check against undesirable behavior.

If you optimize for...	The system may behave badly by...	Counter-metric
Ticket deflection	Hiding or delaying legitimate escalation.	Reopen rate, abandonment rate, and human-escalation satisfaction.
Fast responses	Producing shallow or unsupported answers.	Citation-grounded answer rate.
Low model cost	Using too little context or declining useful requests.	Answer quality and successful self-service resolution.
High automation	Taking actions a customer did not intend.	Rate of human review, corrections, reversals, and safety escalations.
Broad coverage	Answering requests outside approved knowledge.	Unsupported-answer rate and abstention correctness.

This is the discovery equivalent of designing a reward function: decide explicitly what you want the system to optimize, what failures you will not tolerate, and what signals will tell you that a seemingly positive number is misleading.

---

Extract technical constraints before choosing an architecture

A constraint is not merely a technical preference. It is a condition that materially limits acceptable designs.

For example, “we prefer Python” is an implementation preference. “The assistant must never retrieve another tenant’s private documents” is a security constraint. “All customer data must remain in an approved region” is a compliance constraint. “A response after 30 seconds causes chat abandonment” is a user-experience constraint.

For the portfolio scenario, record the following provisional constraints:

Constraint area	Discovery statement for the SaaS support copilot	Status
Identity and tenancy	Every conversation, retrieval request, source citation, cache entry, and audit event must be scoped to the authenticated tenant and user authorization.	Required
Knowledge access	The assistant may use only approved knowledge sources; it must distinguish public product documentation from tenant-private content.	Required
Privacy	Prompts, responses, logs, and evaluation data must avoid unnecessary customer data; retention and deletion behavior must be defined.	Required, detailed policy open
Actions	Initial release is informational. It must not modify subscriptions, permissions, billing, or customer data.	Scope boundary
Latency	The interaction must feel conversational; pilot p95 target is under 10 seconds.	Provisional
Availability and fallback	If the model or retrieval service fails, the user must receive a clear support path rather than an invented answer.	Required
Cost	Per-resolution cost must be measurable by tenant and remain below the support-cost value it replaces.	Required, threshold open
Portability	The portfolio prototype should run locally or on free-tier services; model and vector-store interfaces should not be tied to one cloud vendor.	Prototype constraint
Language and accessibility	English is the initial prototype language; customer language requirements must be validated before production scope.	Assumption
Auditability	The system must record request metadata, retrieval sources, response outcome, safety events, and escalation decisions without exposing excess sensitive content.	Required

A good discovery canvas does not claim to solve every constraint. It makes constraints visible early enough that later architecture choices can be justified.

---

Build a risk register that leads to decisions

A risk is not just “the AI might hallucinate.” A useful risk statement names:

• Failure mode: What can go wrong?
• Impact: Who is harmed and how?
• Likelihood: How plausible is it in the proposed scope?
• Detection signal: How will the team know?
• Decision or control: What must happen before launch or during operation?

Here is the initial risk register for the SaaS support copilot.

Priority	Risk	Impact	Detection signal	Initial decision
Critical	Cross-tenant retrieval or disclosure	Customer data breach, contractual and reputational damage	Automated isolation tests; retrieval logs grouped by tenant; incident alerts	Tenant filtering and authorization must be enforced before retrieval; zero leakage is a release gate.
Critical	Prompt injection, including malicious content in documents	System ignores instructions, leaks data, or attempts unsafe actions	Injection test set; flagged prompt patterns; abnormal tool-request attempts	Treat retrieved text as untrusted data; constrain system behavior and prohibit high-impact actions in the first release.
High	Unsupported or incorrect answer	Customer misconfiguration, lost trust, delayed resolution	Groundedness review; citation coverage; answer correction and reopen rate	Require approved sources and clear escalation when evidence is insufficient.
High	Sensitive data in prompts, logs, or analytics	Privacy violation and compliance exposure	Data-loss scanning; log review; retention audits	Minimize stored content; define redaction, retention, and deletion requirements.
High	Stale or incomplete documentation	Repeated wrong answers and poor support outcomes	Source freshness checks; answer feedback; clusters of escalations	Assign knowledge owners and define update workflow.
Medium	Latency, rate limits, or provider outage	Abandoned conversations and support load spikes	p95 latency; timeout rate; model and retrieval error rate	Provide timeouts, retries where safe, and a visible support fallback.
Medium	Costs grow with usage or long prompts	Feature loses economic viability	Cost per request and per resolved request, tracked by tenant	Set token budgets and usage reporting; revisit model choice using evaluation evidence.
Medium	Automation hides product problems	Fewer tickets but unresolved user friction	Customer satisfaction, reopen rate, and product-feedback tagging	Preserve aggregate insight into recurring issues; do not treat deflection as the only outcome.

This register will become more detailed in the security, safety, reliability, and evaluation modules. For now, its job is to establish architecture-driving facts: tenant isolation, evidence grounding, no high-impact actions, auditability, and explicit fallback behavior.

---

The completed discovery canvas

Use the following as the first version of your portfolio artifact. Keep it in a Markdown document or repository file so it can evolve as you gather evidence.

Canvas area	Initial entry
Title and decision	Tenant-Aware SaaS Support Copilot, initial discovery. Provide evidence-backed help for authenticated SaaS users while reducing repetitive support workload without weakening tenant security or access to human support.
Customer problem	Support representatives spend material time answering recurring product, setup, and troubleshooting questions. Customers wait for answers that may already exist in documentation.
Primary users	Authenticated SaaS customers and tenant administrators. Secondary users are support representatives who receive escalations with conversation context.
Value proposition	Customers receive fast, source-backed guidance. Support teams spend less time on repeatable questions and more time on complex cases. Product teams observe recurring documentation and usability gaps.
In-scope tasks	Explain approved product documentation, guide known configuration steps, summarize relevant help articles, clarify product terminology, and create a human-support escalation when needed.
Out-of-scope tasks	Financial commitments, refunds, billing exceptions, permission changes, account ownership changes, security-incident handling, legal advice, and direct modification of customer data.
Inputs and knowledge	User message, authenticated tenant and role, approved public documentation, and explicitly authorized tenant knowledge where available. Inputs may contain untrusted or malicious instructions.
Output contract	A concise answer, cited supporting sources, a statement of uncertainty when appropriate, and an escalation path when evidence is inadequate or the request is out of scope.
Integration boundaries	SaaS application authentication, support-ticket platform, approved knowledge repository, and observability system. Initial prototype does not execute write actions.
Success measures	Verified self-service resolution, citation-grounded answer rate, correct escalation rate, customer satisfaction, p95 response time, support-ticket reduction for eligible issues, and cost per resolved request.
Non-negotiable constraints	Tenant isolation, authorization-aware knowledge access, privacy-conscious logging and retention, no direct high-impact actions, auditable request and retrieval behavior, and clear outage fallback.
Key risks	Cross-tenant exposure, prompt injection, unsupported advice, sensitive-data retention, stale knowledge, latency or provider failure, and uncontrolled cost growth.
Assumptions to validate	The majority of repetitive tickets map to approved documentation; users accept an assistant that cites sources; support tools can receive escalation summaries; English-only scope is sufficient for the first pilot.
Evidence needed next	Ticket taxonomy and volume, current resolution time, common escalation reasons, documentation quality, tenant-data classification, authorization model, retention requirements, regional requirements, and current support cost.

A concise discovery recommendation

For an architecture interview or customer conversation, you can summarize the canvas in about 90 seconds:

I would begin with a constrained support copilot, not a general autonomous agent. Its first job is to answer common authenticated customer questions from approved, authorization-aware knowledge sources and cite the evidence it used. Success is not simply fewer tickets; it is verified self-service resolution, grounded answers, correct escalation, acceptable latency, and lower cost per resolved eligible issue. Cross-tenant exposure, unsupported answers, prompt injection, and sensitive-data retention are release-critical risks. For the first release, the system provides information and escalation only; it does not modify billing, permissions, or customer data. Before selecting a model or cloud service, I would validate ticket taxonomy, current support baselines, knowledge quality, authorization boundaries, and data-retention requirements.

That recommendation is deliberately architectural: it expresses scope, measurable outcomes, constraints, and trade-offs without prematurely naming a model.

---

Key takeaways

A customer request for “an AI chatbot” becomes architecturally useful only when you can state:

• Who is served and what job they are trying to complete.
• What the system may and may not do, especially where stakes are high.
• How success will be measured, including counter-metrics and non-negotiable release gates.
• Which technical constraints shape acceptable designs.
• Which risks require early decisions, rather than post-launch monitoring alone.
• Which assumptions still require evidence from customers, support operations, and security stakeholders.

You now have the first artifact for the SaaS support-copilot portfolio: a discovery canvas that can anchor technical choices in customer value and operational reality.

Next, you will assess a set of SaaS feature requirements and decide which are genuinely suitable for an LLM-based solution, which should remain deterministic, and which need a hybrid design.