Deterministic Fallback Policies for Low Confidence and Service Failures

Good to see the decision service take shape. In the previous lesson, Jev supplied bounded signals for intent, credential risk, and urgency, while your TypeScript policy retained authority over routing and escalation. Importantly, a successful Jev response could already produce manual_triage when evidence was insufficient.

This lesson addresses a different class of uncertainty: the model service cannot provide a usable response at all. You will make the support workflow behave predictably when confidence is low, a request times out, Jev rate-limits the service, or Jev is unavailable. The result is a deliberately conservative resilience layer: no silent retries, no accidental automated action, and no cascading failure during an outage.

Plan for about 40–45 minutes: a short study segment, then a TypeScript implementation and focused tests.

---

Two meanings of “fallback”

A robust policy must distinguish semantic uncertainty from transport failure.

Semantic uncertainty means Jev replied successfully, but its evidence was not strong enough for an automated action. Your existing decideTriage() already handles this:

• ambiguous credential-risk probability produces manual_triage;
• weak intent confidence produces manual_triage;
• uncertain urgency produces manual_triage;
• other intent produces manual_triage.

Do not retry these just because the answer is uncertain. Repeating the same bounded judgment against the same evidence usually adds no information. It only adds latency, cost, and a tempting opportunity to select a more convenient answer.

Transport failure is different: there is no trustworthy result to interpret. That includes:

• the request exceeding your latency budget;
• an HTTP 429 rate-limit response;
• an HTTP 529 overload response;
• connection or DNS failures;
• an open circuit breaker that intentionally prevents another outbound call.

The fallback must be a normal, typed application decision—not an exception that leaks through to the HTTP boundary.

A practical policy for the support workflow looks like this:

Condition	Is there a Jev result?	Immediate application behavior	Retry at this layer?
Low confidence or an uncertain probability	Yes	manual_triage, retaining signals for the reviewer	No
Timeout	No	manual_triage with jev_timeout	No
Rate limit (429)	No	Defer the ticket to a durable retry queue	No immediate retry
Overload (529) or network outage	No	manual_triage with jev_unavailable	No immediate retry
Circuit open	No call is made	manual_triage with jev_circuit_open	No
Invalid credentials or malformed request	No usable result	manual_triage, plus an operational alert	No

“Defer” is appropriate for a rate limit because the service has explicitly communicated temporary capacity pressure. A ticket should not disappear; it should be placed into a durable queue with a known future attempt time. In contrast, an interactive support endpoint should not hold a user request open while it repeatedly waits for an unhealthy dependency.

---

Treat service resilience as an explicit contract

Your original TriageDecision union represented decisions after a successful local eligibility check and, usually, a Jev evaluation. Extend its manual-review reasons and add one deferred outcome.

In lib/triage-contract.ts, keep the existing union members, but add the resilience-specific types:

export type FallbackReason =
  | "jev_timeout"
  | "jev_unavailable"
  | "jev_circuit_open"
  | "jev_misconfigured";

export type FallbackMetadata = {
  source: "jev";
  modelAttempted: boolean;
  retryable: boolean;
};

export type DeferredTriageDecision = {
  kind: "defer_triage";
  reason: "jev_rate_limited";
  retryAt: string;
  fallback: {
    source: "jev";
    modelAttempted: true;
    retryable: true;
  };
};

export type ResilientTriageDecision =
  | TriageDecision
  | DeferredTriageDecision;

Then extend the existing manual_triage member in TriageDecision so its reason field accepts the four FallbackReason values and it can carry optional fallback metadata:

type ManualTriageDecision = {
  kind: "manual_triage";
  reason:
    | "safety_uncertain"
    | "intent_uncertain"
    | "urgency_uncertain"
    | "unsupported_intent"
    | "insufficient_evidence"
    | FallbackReason;
  signals?: TriageSignals;
  fallback?: FallbackMetadata;
};

The key detail is that signals are optional. When Jev returned low confidence, you have signals to show the reviewer. When the request timed out before receiving a response, you do not. Never fabricate a probability or confidence value for a failed call.

---

Use a circuit breaker to protect both services

A timeout does not necessarily mean Jev has failed globally. It may reflect a transient network issue, a local deployment problem, or a momentary overload. But allowing every new support request to wait for the same failing dependency is harmful: it consumes application capacity and may amplify the upstream outage.

The standard solution is a circuit breaker. It owns service availability, not judgment quality.

The states are straightforward:

• Closed: Calls are allowed. The breaker counts transient failures.
• Open: Calls are rejected immediately. Your system returns its deterministic fallback without contacting Jev.
• Half-open: After a cooldown, allow one probe call. A success closes the breaker; a failure reopens it for another cooldown.

For this workflow, only availability-related errors affect the breaker:

• count timeouts, 429, 529, and confirmed connection failures;
• do not count low-confidence responses, because Jev was available and correctly returned an uncertain judgment;
• do not count request-validation or authentication failures, because they require a deployment or configuration fix, not a service-recovery cooldown.

For a single-process service, an in-memory breaker is enough to understand and test the policy. In a horizontally scaled production deployment, breaker state should eventually be shared or coordinated; otherwise each instance observes only part of the failure pattern.

---

Make the Jev call cancellable and bounded

The previous triageTicket() function owns Jev evaluation and deterministic interpretation. Refactor its signature so the transport layer can receive an abort signal:

export type TriageCallOptions = {
  signal?: AbortSignal;
};

export async function triageTicket(
  ticket: SupportTicket,
  options: TriageCallOptions = {},
): Promise<TriageDecision> {
  // Keep the deterministic eligibility checks from the previous lesson.

  // Pass options.signal to the Jev transport used by your application.
  // With TypeSafe's direct SDK, systemOne accepts timeout, retry, and
  // signal options as its second argument.
  //
  // Continue to normalize the successful Jev result into TriageSignals,
  // then return decideTriage(signals).
}

The exact provider call depends on whether your project uses TypeSafe’s SDK directly or Jev through the Vercel AI Gateway. The architectural requirement is stable: the cancellation signal must reach the actual network call.

This is the relevant platform primitive:

A deadline is not merely a UI preference. If your route has a 2-second response budget and the model call continues for 15 seconds, the request can tie up server resources long after the user has stopped waiting.

Define a small abstraction around the existing triage function. This keeps circuit-breaking and failure policy independent of a particular SDK:

import type {
  ResilientTriageDecision,
  TriageDecision,
} from "./triage-contract";
import type { SupportTicket } from "./triage-service";

export type JevFailureKind =
  | "timeout"
  | "rate_limited"
  | "unavailable"
  | "misconfigured";

export class JevTransportError extends Error {
  constructor(
    public readonly kind: JevFailureKind,
    public readonly retryAfterMs?: number,
  ) {
    super(kind);
    this.name = "JevTransportError";
  }
}

export interface TriageAttempt {
  run(
    ticket: SupportTicket,
    options: { signal: AbortSignal },
  ): Promise<TriageDecision>;
}

Create a transport adapter that catches provider-specific errors and normalizes them:

• 429 becomes new JevTransportError("rate_limited", retryAfterMs);
• 529, connection failures, and known temporary provider failures become new JevTransportError("unavailable");
• authentication failures and request-validation errors become new JevTransportError("misconfigured");
• unknown programming errors should still be thrown, logged, and fixed. Do not quietly label every unexpected bug as a model outage.

This adapter is also the right place to read a provider’s Retry-After header and convert it into retryAfterMs. Keep header parsing out of the business policy.

---

Implement a testable local circuit breaker

Create lib/jev-breaker.ts:

export type BreakerState = "closed" | "open" | "half_open";

export class LocalJevBreaker {
  private state: BreakerState = "closed";
  private failureCount = 0;
  private openedUntilMs = 0;
  private halfOpenProbeActive = false;

  constructor(
    private readonly failureThreshold = 3,
    private readonly cooldownMs = 30_000,
  ) {}

  permit(nowMs: number): boolean {
    if (this.state === "closed") {
      return true;
    }

    if (this.state === "open" && nowMs < this.openedUntilMs) {
      return false;
    }

    if (this.state === "open") {
      this.state = "half_open";
      this.halfOpenProbeActive = true;
      return true;
    }

    return false;
  }

  recordSuccess(): void {
    this.state = "closed";
    this.failureCount = 0;
    this.halfOpenProbeActive = false;
    this.openedUntilMs = 0;
  }

  recordFailure(nowMs: number): void {
    this.failureCount += 1;

    const shouldOpen =
      this.state === "half_open" ||
      this.failureCount >= this.failureThreshold;

    if (!shouldOpen) {
      return;
    }

    this.state = "open";
    this.openedUntilMs = nowMs + this.cooldownMs;
    this.halfOpenProbeActive = false;
  }

  snapshot(nowMs: number): {
    state: BreakerState;
    failures: number;
    retryAfterMs: number;
  } {
    return {
      state: this.state,
      failures: this.failureCount,
      retryAfterMs: Math.max(0, this.openedUntilMs - nowMs),
    };
  }
}

A few policy choices are embedded here:

1. The breaker opens after three availability failures.
2. It remains open for 30 seconds.
3. After the cooldown, exactly one probe is allowed.
4. A successful response closes the breaker even if that response leads to manual_triage because of low confidence.

That final point matters. A low-confidence outcome is not a transport failure. Conflating the two would open the circuit merely because users sent ambiguous tickets.

---

Add the deterministic fallback facade

Create lib/resilient-triage.ts. This function sits outside the normal Jev signal interpretation from the previous lesson.

import {
  JevTransportError,
  type TriageAttempt,
} from "./jev-transport";
import { LocalJevBreaker } from "./jev-breaker";
import type {
  ResilientTriageDecision,
  TriageDecision,
} from "./triage-contract";
import type { SupportTicket } from "./triage-service";

const POLICY = {
  jevDeadlineMs: 1_500,
  defaultRateLimitDelayMs: 30_000,
  maxRateLimitDelayMs: 5 * 60_000,
} as const;

function fallback(
  reason: "jev_timeout" | "jev_unavailable" | "jev_circuit_open" | "jev_misconfigured",
  modelAttempted: boolean,
): TriageDecision {
  return {
    kind: "manual_triage",
    reason,
    fallback: {
      source: "jev",
      modelAttempted,
      retryable:
        reason === "jev_timeout" ||
        reason === "jev_unavailable" ||
        reason === "jev_circuit_open",
    },
  };
}

function localEligibility(ticket: SupportTicket): TriageDecision | undefined {
  if (ticket.status === "closed") {
    return {
      kind: "no_action",
      reason: "ticket_closed",
    };
  }

  if (!ticket.message.trim()) {
    return {
      kind: "manual_triage",
      reason: "insufficient_evidence",
    };
  }

  return undefined;
}

async function runWithDeadline(
  attempt: TriageAttempt,
  ticket: SupportTicket,
  deadlineMs: number,
): Promise<TriageDecision> {
  const controller = new AbortController();
  let timeoutHandle: ReturnType<typeof setTimeout> | undefined;

  const timeout = new Promise<never>(function (_resolve, reject) {
    timeoutHandle = setTimeout(function () {
      controller.abort();
      reject(new JevTransportError("timeout"));
    }, deadlineMs);
  });

  try {
    return await Promise.race([
      attempt.run(ticket, { signal: controller.signal }),
      timeout,
    ]);
  } finally {
    if (timeoutHandle !== undefined) {
      clearTimeout(timeoutHandle);
    }
  }
}

function boundedRetryDelay(retryAfterMs: number | undefined): number {
  const requestedDelay = retryAfterMs ?? POLICY.defaultRateLimitDelayMs;

  return Math.min(
    Math.max(requestedDelay, 1_000),
    POLICY.maxRateLimitDelayMs,
  );
}

export class ResilientTriageService {
  constructor(
    private readonly attempt: TriageAttempt,
    private readonly breaker: LocalJevBreaker,
    private readonly now: () => number = Date.now,
  ) {}

  async triage(ticket: SupportTicket): Promise<ResilientTriageDecision> {
    const localDecision = localEligibility(ticket);

    if (localDecision !== undefined) {
      return localDecision;
    }

    const startMs = this.now();

    if (!this.breaker.permit(startMs)) {
      return fallback("jev_circuit_open", false);
    }

    try {
      const decision = await runWithDeadline(
        this.attempt,
        ticket,
        POLICY.jevDeadlineMs,
      );

      // A successful low-confidence/manual decision still proves that
      // the dependency is available.
      this.breaker.recordSuccess();

      return decision;
    } catch (error) {
      if (!(error instanceof JevTransportError)) {
        throw error;
      }

      const nowMs = this.now();

      if (error.kind === "misconfigured") {
        return fallback("jev_misconfigured", true);
      }

      this.breaker.recordFailure(nowMs);

      if (error.kind === "rate_limited") {
        const retryDelayMs = boundedRetryDelay(error.retryAfterMs);

        return {
          kind: "defer_triage",
          reason: "jev_rate_limited",
          retryAt: new Date(nowMs + retryDelayMs).toISOString(),
          fallback: {
            source: "jev",
            modelAttempted: true,
            retryable: true,
          },
        };
      }

      if (error.kind === "timeout") {
        return fallback("jev_timeout", true);
      }

      return fallback("jev_unavailable", true);
    }
  }
}

This implementation has a few intentional properties:

• Local checks occur before checking the breaker, so a closed ticket still returns no_action during a Jev outage.
• The 1.5-second deadline is application policy. Tune it against the latency budget of the route that calls this service.
• Promise.race() returns control to the application when the deadline expires; the abort signal asks the underlying transport to cancel as well.
• The circuit breaker tracks availability failures, not model uncertainty.
• The function does not perform any side effect. It only returns a typed decision that another layer can queue, audit, or display.

---

Decide where retries belong

Retries are easy to multiply accidentally:

1. the SDK retries;
2. your transport adapter retries;
3. the HTTP handler retries;
4. the queue retries again.

That stack can turn one ticket into many overlapping requests precisely when a dependency is least able to serve them.

Choose one immediate retry owner. If you use the TypeSafe SDK’s retry option, configure a small bounded retry policy there and do not add another immediate retry loop in ResilientTriageService. The resilience facade above makes one logical attempt and then returns a deterministic outcome.

For this support workflow:

• a low-confidence result is reviewed, never retried automatically;
• an interactive timeout falls back to manual triage;
• a 429 is persisted for delayed retry, ideally honoring the server-provided retry delay;
• a 529 or network failure falls back to manual triage and contributes to opening the breaker;
• once open, the breaker prevents new requests from adding pressure to Jev.

The downstream dispatcher should make the side effects explicit:

Returned outcome	Downstream action
route	Enqueue the ticket in the selected department queue
security_review	Enqueue the specialist security-review workflow
Existing manual_triage with signals	Show signals and reasons to the reviewer
Fallback manual_triage without signals	Place in the ordinary manual queue, preserving fallback reason
defer_triage	Persist a retry job for retryAt
no_action	Do nothing

Do not implement defer_triage as an in-memory setTimeout. A process restart would lose tickets. The durable queue belongs at the application boundary, outside this decision function.

---

Verify behavior with focused tests

Because the resilience service depends on a small TriageAttempt interface and an injected clock, it is easy to test without live Jev credentials.

Your minimum test set should establish these behavioral guarantees:

• A successful manual_triage caused by intent_uncertain leaves the breaker closed.
• A timeout returns manual_triage with jev_timeout.
• A 429 returns defer_triage with a deterministic future retryAt.
• Three availability failures open the breaker.
• Once open, the service returns jev_circuit_open and does not call attempt.run().
• After the cooldown, one successful probe closes the breaker.
• A 401 or validation failure returns jev_misconfigured and does not count toward the breaker threshold.
• A closed ticket returns no_action without calling Jev, even while the breaker is open.

For logs and metrics, record only operational metadata: decision kind, fallback reason, latency, model version when available, breaker state, and whether a model call was attempted. Do not put full ticket messages, credentials, or other customer content into generic error logs.

---

You now have a deterministic failure policy around Jev: low-confidence outputs remain reviewable model decisions; timeouts and outages become explicit application outcomes; rate limiting becomes delayed work rather than uncontrolled pressure; and the circuit breaker prevents a failing dependency from degrading the rest of the support system.

Next, you will build a minimal decision inspector that makes these outcomes visible: the selected values, probabilities, confidence, latency, and the precise reason for escalation, deferral, or fallback.