Typed Decision Contracts and Acceptance Criteria for Multi-Stage Support Automation

Welcome to the final week of the course. This capstone turns the individual Jev patterns into a production-shaped support workflow: a bounded system that classifies a ticket, checks for risk, sets service priority, and either routes safely or escalates explicitly.

This first lesson is about specification before integration. You will define the contract that sits between Jev’s typed judgments and the rest of your TypeScript application: the state Jev may inspect, the atomic questions it must answer, the application decision variants it may produce, and the acceptance rules that make automation permissible.

---

A decision contract is more than a prompt

A support automation feature is often described too vaguely:

“Use AI to triage tickets.”

That is not implementable or testable. It hides distinct decisions with different costs of error:

• Which team should own this ticket?
• Does the message contain a security-sensitive request?
• Does the customer need expedited handling?
• Is there enough certainty to take an automated action?

A typed decision contract makes those choices explicit. It has four layers:

1. Input state contract — the minimal current evidence supplied to Jev.
2. Question contract — the bounded, atomic judgments Jev returns.
3. Application outcome contract — the finite set of actions your code may take.
4. Acceptance criteria — deterministic rules for choosing an action from the returned signals.

The important boundary is this:

• Jev judges bounded properties of supplied evidence.
• Your application owns business rules, side effects, permissions, queues, audit logs, and fallbacks.

For example, “ticket is closed” is deterministic application state. Do not spend a model call rediscovering it. “The primary request concerns a duplicate charge or a refund” is a contextual classification judgment, and is appropriate for a Choice.

A Choice result gives your system three related but non-identical facts:

• choice: the most likely allowed option;
• probabilities: the full distribution over options;
• confidence: how concentrated that distribution is.

That distinction matters. A ticket may be classified as billing, yet still have enough probability on orders that automatic routing would be inappropriate.

---

Define the workflow boundary and the evidence state

For this capstone, keep the workflow deliberately narrow:

Given an active inbound support ticket, decide whether it needs security review, manual triage, or routing to a support team with a standard or expedited queue.

This is triage, not full support automation. It does not authorize refunds, alter orders, reset accounts, or generate a persuasive customer reply. Keeping the action surface narrow makes a first release safer and easier to evaluate.

Stage responsibilities

Think of the workflow as several logical stages. They may be evaluated in one Jev request, but their responsibilities remain separate.

Stage	Owner	Responsibility
Eligibility	Application code	Reject closed, malformed, or duplicate tickets before calling Jev
Evidence preparation	Application code	Construct minimal, current, structured state
Atomic judgments	Jev	Determine intent, sensitive-credential risk, and customer urgency
Acceptance policy	Application code	Apply thresholds and precedence rules
Side effects	Application code	Route, queue, escalate, record audit data, or invoke fallback

The state contract should contain evidence needed by the questions, not a serialized customer record. A reasonable initial state might be:

type SupportState = {
  ticket: {
    id: string;
    status: "open" | "pending" | "closed";
    channel: "email" | "chat" | "web";
    message: string;
  };
  customer: {
    plan: "free" | "pro" | "enterprise";
    openOrderSummaries: Array<{
      id: string;
      status: "processing" | "shipped" | "delayed";
    }>;
  };
  policy: {
    sensitiveCredentials: readonly [
      "password",
      "security code",
      "API key",
      "recovery code"
    ];
  };
};

The exact shape will change with your product, but the selection principle should remain stable:

• Include the ticket message because all three judgments inspect it.
• Include the plan only if it legitimately affects queueing policy.
• Include short, relevant order summaries only if routing needs them.
• Include the policy vocabulary because it is current business knowledge, not model background knowledge.
• Exclude payment-card data, authentication secrets, full account history, internal notes unrelated to the ticket, and broad personal-profile fields.

This is both a privacy measure and a decision-quality measure. Irrelevant context makes the decision boundary harder to inspect and can distract from the evidence that actually matters.

---

Specify atomic Jev judgments

Avoid a single question such as:

“What should we do with this support ticket?”

It combines routing, safety, urgency, and operational authority into one opaque answer. Instead, this workflow defines three judgments with different purposes.

1. Intent: a Choice

The intent question determines the ticket’s primary support destination.

const INTENTS = ["billing", "orders", "account", "other"] as const;
type Intent = (typeof INTENTS)[number];

The options must be mutually exclusive at the decision level:

Option	Includes	Excludes
billing	charges, invoices, subscriptions, refunds	shipment tracking and login trouble
orders	delivery status, cancellations, returns, missing items	payment disputes and account access
account	sign-in, profile, permissions, account-security help	billing and delivery requests
other	requests not covered by the support teams above	an unclear case that fits a listed team

other is not a substitute for uncertainty. Jev must choose an option, while confidence tells you whether it is safe to act on that choice. other means the ticket genuinely falls outside the available automated destinations.

2. Credential-request risk: a Noul

A safety question should be a single proposition:

Does ticket.message request that the recipient disclose a credential listed in policy.sensitiveCredentials?

The true criterion is narrow: it requires a request to disclose the credential itself. It should not flag legitimate discussion of credential management, such as “I need to reset my password.”

This judgment does not prove that a sender is malicious. It determines a bounded operational response: do not continue normal automated routing when a sensitive-credential request is likely.

3. Customer urgency: a Score

Urgency is different from sentiment. A calm message can report a service outage; an angry message can concern a routine policy question.

Define ordered levels using observable criteria:

Score	Name	Evidence
0	routine	General question, ordinary request, no time-sensitive consequence stated
1	elevated	Repeated unresolved issue, meaningful inconvenience, time-sensitive request
2	urgent	Account lockout, suspected compromise, service-blocking issue, or imminent material impact

The question should explicitly say: judge the operational urgency of the reported issue, not the customer’s emotional tone.

These three questions can be sent together against the same state. They are logically separate, and their results are composed by code.

---

Define the application’s finite outcome type

Do not let downstream code work directly with raw Jev answers. Instead, convert model-facing signals into a small, application-facing discriminated union.

type ChoiceSignal<T extends string> = {
  choice: T;
  confidence: number;
  probabilities: Record<T, number>;
};

type NoulSignal = {
  probability: number;
  confidence: number;
};

type ScoreSignal = {
  score: 0 | 1 | 2;
  confidence: number;
};

type TriageSignals = {
  intent: ChoiceSignal<Intent>;
  requestsSensitiveCredential: NoulSignal;
  urgency: ScoreSignal;
};

type TriageDecision =
  | {
      kind: "no_action";
      reason: "ticket_closed";
    }
  | {
      kind: "security_review";
      reason: "credential_request_likely";
      safety: NoulSignal;
    }
  | {
      kind: "manual_triage";
      reason:
        | "insufficient_evidence"
        | "safety_uncertain"
        | "intent_uncertain"
        | "unsupported_intent"
        | "urgency_uncertain";
      signals?: TriageSignals;
    }
  | {
      kind: "route";
      department: Exclude<Intent, "other">;
      queue: "standard" | "expedited";
      signals: TriageSignals;
    };

This type is not merely documentation. It creates a deliberate boundary:

• A security_review outcome cannot accidentally be treated as an ordinary route.
• A route outcome always has a permitted department and queue.
• A manual_triage outcome records why automation was declined.
• A later addition, such as fraud_review, forces every decision consumer to handle it.

For a front-end or service layer, this is a useful place to use exhaustive switching. It prevents a newly added workflow state from silently falling into a generic default UI or handler.

A typical handler can enforce exhaustiveness like this:

function assertNever(value: never): never {
  throw new Error(`Unhandled decision: ${JSON.stringify(value)}`);
}

function executeDecision(decision: TriageDecision): void {
  switch (decision.kind) {
    case "no_action":
      return;

    case "security_review":
      enqueueSecurityReview(decision);
      return;

    case "manual_triage":
      enqueueManualTriage(decision);
      return;

    case "route":
      enqueueSupportTeam(decision.department, decision.queue);
      return;

    default:
      assertNever(decision);
  }
}

The compiler becomes part of your release process: it will flag unhandled consequences of contract changes before production does.

---

Turn uncertainty into explicit acceptance criteria

A model answer is not automatically permission to act. The workflow needs deterministic acceptance rules based on the harm of being wrong.

For this first support-triage release, choose conservative provisional thresholds:

const ACCEPTANCE = {
  intentConfidence: 0.8,

  safetyClearProbability: 0.2,
  safetyBlockProbability: 0.6,
  safetyConfidence: 0.8,

  urgencyConfidence: 0.7,
} as const;

These numbers are a versioned policy, not universal truths. In a real release, they should come from your labeled evaluation data and the costs of misrouting, missed safety events, delayed urgent tickets, and human-review time.

Decision precedence

Safety must take precedence over convenience. Define the evaluation order clearly:

1. If the ticket is closed, return no_action without calling Jev.
2. If the message is absent or unusable, return manual_triage with insufficient_evidence.
3. If sensitive-credential risk is at or above safetyBlockProbability, return security_review.
4. If safety risk is not confidently low, return manual_triage with safety_uncertain.
5. If intent confidence is below intentConfidence, return manual_triage with intent_uncertain.
6. If intent is other, return manual_triage with unsupported_intent.
7. If urgency confidence is below urgencyConfidence, return manual_triage with urgency_uncertain.
8. Otherwise, route to the intent department. Use expedited only for the accepted urgent score.

Notice the asymmetric treatment of safety:

• A high probability of a credential request triggers security review.
• A middle probability, or low confidence, does not mean “probably safe.” It means the workflow lacks authority to continue unattended.
• A low risk probability and sufficient confidence permit normal routing.

That policy can be written as ordinary TypeScript:

function decideTriage(
  state: Pick<SupportState, "ticket">,
  signals: TriageSignals,
): TriageDecision {
  if (state.ticket.status === "closed") {
    return { kind: "no_action", reason: "ticket_closed" };
  }

  if (!state.ticket.message.trim()) {
    return { kind: "manual_triage", reason: "insufficient_evidence" };
  }

  const safety = signals.requestsSensitiveCredential;

  if (safety.probability >= ACCEPTANCE.safetyBlockProbability) {
    return {
      kind: "security_review",
      reason: "credential_request_likely",
      safety,
    };
  }

  const safetyIsClearlyLow =
    safety.probability <= ACCEPTANCE.safetyClearProbability &&
    safety.confidence >= ACCEPTANCE.safetyConfidence;

  if (!safetyIsClearlyLow) {
    return {
      kind: "manual_triage",
      reason: "safety_uncertain",
      signals,
    };
  }

  if (signals.intent.confidence < ACCEPTANCE.intentConfidence) {
    return {
      kind: "manual_triage",
      reason: "intent_uncertain",
      signals,
    };
  }

  if (signals.intent.choice === "other") {
    return {
      kind: "manual_triage",
      reason: "unsupported_intent",
      signals,
    };
  }

  if (signals.urgency.confidence < ACCEPTANCE.urgencyConfidence) {
    return {
      kind: "manual_triage",
      reason: "urgency_uncertain",
      signals,
    };
  }

  return {
    kind: "route",
    department: signals.intent.choice,
    queue: signals.urgency.score === 2 ? "expedited" : "standard",
    signals,
  };
}

The precise SDK adapter will be implemented in the next lesson. For now, the valuable design decision is that the rest of the application receives a stable TriageDecision, independent of the raw response format.

---

Write acceptance criteria as observable behavior

A contract is ready to implement when another engineer can determine whether it has been met without guessing at your intent. For this workflow, the acceptance criteria should include the following.

Area	Acceptance criterion
Deterministic eligibility	Closed tickets produce no_action and make no Jev request.
Data minimization	The Jev state contains only ticket evidence, approved customer facts, and relevant policy data; no secrets or unrelated profile data are included.
Intent routing	A ticket routes automatically only when safety is clearly low, intent confidence meets the configured threshold, intent is supported, and urgency is accepted.
Safety precedence	A credential-request probability at or above the block threshold always produces security_review, regardless of intent or urgency.
Uncertainty handling	Any safety, intent, or urgency signal that does not satisfy its acceptance rule produces a named manual_triage reason.
Queueing	Only an accepted urgent score produces an expedited queue. Other accepted scores use standard.
Side-effect safety	The workflow may route or escalate; it cannot disclose credentials, issue refunds, change account access, or close a ticket.
Observability	Every non-no_action decision records ticket ID, contract version, selected decision, raw typed signals, threshold configuration, latency, and escalation reason where applicable.
Type safety	Decision consumers use exhaustive handling, so a newly introduced decision variant creates a compile-time failure until explicitly addressed.

A few representative scenarios make the contract concrete:

Ticket summary	Important signals	Required decision
“My invoice includes a duplicate charge.”	billing, high intent confidence; low, confident credential risk; routine	Route to billing, standard
“Send us your API key so we can verify your account.”	High credential-request probability	security_review, even if intent appears to be account
“My package has not arrived and I was charged twice.”	orders only slightly above billing; low intent confidence	manual_triage, intent_uncertain
“I am locked out before payroll closes today.”	account, clear safety; accepted urgent score	Route to account, expedited
“Help.”	Sparse evidence and low confidence	manual_triage, usually intent_uncertain or urgency_uncertain

Keep these examples as the initial contract-test matrix. Later, recorded Jev responses can test the policy without consuming API calls, while live evaluation data tests whether the policy is good enough to deploy.

---

A practical specification artefact

Before implementation, place the following in your project as a versioned document or module:

export const TRIAGE_CONTRACT_VERSION = "2025-01";

export const TRIAGE_QUESTION_IDS = {
  intent: "intent",
  credentialRisk: "requests_sensitive_credential",
  urgency: "customer_urgency",
} as const;

Alongside it, maintain:

• the SupportState schema and its allowed fields;
• option criteria for intent;
• true and false criteria for the credential-risk proposition;
• observable anchors for each urgency score;
• TriageDecision and threshold configuration;
• the scenario matrix above;
• a changelog explaining any modification to question wording, option sets, thresholds, or side effects.

Treat a change to an option name, threshold, or outcome variant as a contract change. It can affect routing behavior just as surely as a database schema migration can affect application behavior.

---

You now have a bounded, typed specification for support triage: minimal evidence goes in; Jev provides independent intent, safety, and urgency signals; deterministic policy converts them into a finite set of permitted outcomes. Confidence is not decorative metadata—it is part of the authority model for automation.

Next, you will implement this contract as a TypeScript decision service that makes one Jev request, translates the typed answers into TriageSignals, applies the acceptance policy, and returns the exhaustive TriageDecision union.