Speculative Fan-Out and Branch-Specific Answer Selection

Welcome back. Last time, you combined independent Jev Scores into a deterministic priority policy. The important boundary was: Jev supplies narrow judgments; your code owns weighting, thresholds, and operational actions.

This lesson applies that same boundary to a routing workflow that has several possible paths. You will ask Jev for a small set of potentially useful answers in one request, then let deterministic code select only the answers that belong to the chosen route. This is the speculative fan-out pattern.

---

Fan out judgments, then commit in code

Imagine a support ticket:

“I was charged twice for order A-104. The package is also late. Please help.”

A serial model workflow might first ask, “Which department owns this?” Then, after receiving “billing,” it might make a second model request: “Is a refund requested?” That works, but it adds a network round trip for every decision.

Speculative fan-out asks the router question and the likely branch-detail questions together:

Question	Why ask it?	Used when
department	Select the primary support route	Always
billing_request	Distinguish refund, invoice explanation, or another billing request	department is billing
shipping_issue	Identify delayed, missing, or damaged delivery	department is shipping
account_issue	Identify reset, lockout, or permission problem	department is account
tone	Flag cases that may need senior handling	Any route, if confidence is sufficient

Jev evaluates each question against the same state. It may return a plausible shipping_issue even when the selected department is billing. That answer is not wrong; it is simply irrelevant to the route your policy chose.

The commitment happens only in code:

1. Inspect the routing answer and its confidence.
2. If the route is uncertain, send the ticket to manual triage.
3. If the route is sufficiently clear, select the detail answer associated with that route.
4. Ignore branch-specific answers from all other routes.
5. Apply any side effects only after these deterministic checks.

The “speculative” part resembles speculative execution in a processor in one useful respect: several possible inputs are computed before the system commits to one path. The important difference is that Jev questions should have no side effects. They are evidence gathering, not actions.

---

When fan-out is the right shape

Speculative fan-out is useful when all of the following are true:

• You have a small, bounded set of plausible branches.
• Each prospective question can be answered independently from the same state.
• The route-selection question is itself narrow and reviewable.
• Lower latency matters more than avoiding every potentially unused answer.
• Your deterministic code can state precisely which answer is valid on which path.

For example, a customer-support system may support three high-level teams: billing, shipping, and account support. Each team has a small amount of branch-specific metadata it needs. Asking those details together is usually reasonable.

It is not a license to ask every imaginable question. Questions still add tokens and outputs. A useful rule is:

Include an answer when a plausible route in the current decision can consume it immediately, or when it is an explicitly global signal.

In particular, do not include questions merely because they might be useful to an analyst someday. That produces unnecessary cost, more sensitive-data exposure, and an increasingly difficult decision contract to evaluate.

TypeSafe’s architecture guide frames the efficiency benefit directly:

Independence does not mean unrelated

Your fan-out questions may be correlated. A frustrated customer may be more likely to report a severe shipping failure, for example. That is fine.

“Independent” here has an architectural meaning:

• shipping_issue can be evaluated from the ticket state without knowing Jev’s department answer.
• billing_request can be evaluated from the same ticket without waiting for another request.
• Your application can inspect each answer separately.

A bad fan-out question depends on an unobserved model result:

“Given that the department is billing, what refund policy applies?”

That question assumes a prior department result, so it belongs in a second-stage design or should be rephrased into a direct state-based question. A good replacement might be:

“Does ticket.message explicitly request a refund or account credit?”

That can be evaluated directly from the supplied evidence.

---

Build a fan-out contract

We will implement a Python ticket-triage service. Python is convenient here because it follows directly from the uv-managed project you created earlier, but the architectural split is the same in a TypeScript service:

• one Jev request,
• a typed answer collection,
• an ordinary deterministic selector,
• side effects performed only after selection.

Start with five Choice questions. The exact option descriptions should evolve with real support data, but keep the boundaries contrastive from the beginning.

from __future__ import annotations

from dataclasses import dataclass
from typing import Any, Literal, Mapping

from typesafe_sdk import Choice, TypeSafeClient


TRIAGE_QUESTIONS = {
    "department": Choice(
        instructions=(
            "Which support team should own the customer's primary request "
            "in `ticket.message`?"
        ),
        criteria={
            "billing": (
                "Charges, invoices, subscriptions, payment failures, "
                "refunds, or account credits."
            ),
            "shipping": (
                "Delivery status, tracking, late packages, missing packages, "
                "or damage during delivery."
            ),
            "account": (
                "Sign in, password reset, account lockout, profile, "
                "permissions, or security access."
            ),
            "other": (
                "A request that does not fit billing, shipping, or account support."
            ),
        },
    ),
    "billing_request": Choice(
        instructions=(
            "What billing outcome does the customer request in `ticket.message`?"
        ),
        criteria={
            "refund_or_credit": "Explicitly asks for money back or an account credit.",
            "charge_explanation": "Asks why a charge, invoice, or payment occurred.",
            "subscription_change": "Wants to start, cancel, or change a subscription.",
            "other_or_unspecified": "No clear billing outcome is requested.",
        },
    ),
    "shipping_issue": Choice(
        instructions=(
            "What delivery issue, if any, does the customer describe in "
            "`ticket.message`?"
        ),
        criteria={
            "not_delivered": "The package never arrived or is considered lost.",
            "delayed": "The package is late but may still be in transit.",
            "damaged_in_transit": "The package arrived damaged.",
            "other_or_unspecified": "No specific delivery issue is described.",
        },
    ),
    "account_issue": Choice(
        instructions=(
            "What account-access issue, if any, does the customer describe "
            "in `ticket.message`?"
        ),
        criteria={
            "password_reset": "The customer needs to reset or recover a password.",
            "locked_out": "The customer cannot access an account or is locked out.",
            "permissions": "The customer lacks an expected role or permission.",
            "other_or_unspecified": "No specific account-access issue is described.",
        },
    ),
    "tone": Choice(
        instructions="What is the customer's expressed tone in `ticket.message`?",
        criteria={
            "calm": "Matter of fact, neutral, or constructive.",
            "frustrated": "Clearly dissatisfied or annoyed, but still civil.",
            "angry": "Hostile, highly escalated, or threatens cancellation.",
        },
    ),
}

Two design decisions matter here.

First, all questions are answerable against the supplied ticket. shipping_issue does not ask Jev to assume that shipping was selected; it asks what delivery issue is described, if any.

Second, each branch-detail question includes an other_or_unspecified option. This prevents the API contract from forcing a false level of precision for irrelevant or incomplete tickets.

Keep state narrow and shared

All fan-out questions inspect one state object. Include evidence that one or more defined questions need, but omit unrelated customer data.

def build_triage_state(
    ticket: Mapping[str, Any],
    customer: Mapping[str, Any],
) -> dict[str, object]:
    open_orders = [
        {
            "id": order["id"],
            "status": order["status"],
            "items": order.get("items", []),
        }
        for order in customer.get("orders", [])
        if order["status"] != "delivered"
    ]

    return {
        "ticket": {
            "message": ticket["message"],
            "order_reference": ticket.get("order_reference"),
            "links": ticket.get("links", []),
        },
        "customer": {
            "plan": customer.get("plan"),
            "open_orders": open_orders,
        },
    }

This state may help a human or the model associate “order A-104” with an open order, but it avoids sending material such as payment-card information, IP address history, internal account identifiers, or broad analytics data. Those fields do not answer the five questions above.

---

Make the selection policy explicit

The important implementation is not the API call. It is the function that turns parallel answers into one bounded action.

We will return an auditable decision object containing:

• the route,
• the action permitted on that route,
• the IDs of answers actually used,
• the IDs intentionally ignored,
• the original routing probabilities for inspection.

Route = Literal[
    "billing",
    "shipping",
    "account",
    "other",
    "manual_triage",
]

ROUTING_CONFIDENCE_MINIMUM = 0.70
DETAIL_CONFIDENCE_MINIMUM = 0.70
TONE_CONFIDENCE_MINIMUM = 0.70


@dataclass(frozen=True)
class TriageDecision:
    route: Route
    action: str
    priority: str
    used_answer_ids: tuple[str, ...]
    ignored_answer_ids: tuple[str, ...]
    department_probabilities: Mapping[str, float]
    reason: str

Now implement the selector. It performs no API calls. It also performs no side effects, such as assigning a ticket, granting a refund, or changing an account. That separation makes it straightforward to test and safe to reuse.

def select_triage_action(answers: Mapping[str, Any]) -> TriageDecision:
    department = answers["department"]

    # A branch is not authoritative until deterministic policy accepts it.
    if department.confidence < ROUTING_CONFIDENCE_MINIMUM:
        return TriageDecision(
            route="manual_triage",
            action="send_to_manual_triage",
            priority="normal",
            used_answer_ids=("department",),
            ignored_answer_ids=tuple(
                question_id
                for question_id in TRIAGE_QUESTIONS
                if question_id != "department"
            ),
            department_probabilities=department.probabilities,
            reason="Department classification confidence is below the routing threshold.",
        )

    used_answer_ids = ["department", "tone"]

    tone = answers["tone"]
    priority = (
        "senior"
        if tone.confidence >= TONE_CONFIDENCE_MINIMUM
        and tone.choice == "angry"
        else "normal"
    )

    if department.choice == "billing":
        billing_request = answers["billing_request"]
        used_answer_ids.append("billing_request")

        if billing_request.confidence < DETAIL_CONFIDENCE_MINIMUM:
            action = "assign_billing_and_request_clarification"
            reason = "Billing route is clear, but the requested billing outcome is unclear."
        elif billing_request.choice == "refund_or_credit":
            action = "assign_billing_and_flag_refund_review"
            reason = "Customer explicitly requests a refund or credit."
        else:
            action = "assign_billing"
            reason = "Billing route and requested billing outcome are clear."

    elif department.choice == "shipping":
        shipping_issue = answers["shipping_issue"]
        used_answer_ids.append("shipping_issue")

        if shipping_issue.confidence < DETAIL_CONFIDENCE_MINIMUM:
            action = "assign_shipping_and_request_clarification"
            reason = "Shipping route is clear, but the delivery issue is unclear."
        else:
            action = f"assign_shipping_{shipping_issue.choice}"
            reason = "Shipping route and delivery issue are clear."

    elif department.choice == "account":
        account_issue = answers["account_issue"]
        used_answer_ids.append("account_issue")

        if account_issue.confidence < DETAIL_CONFIDENCE_MINIMUM:
            action = "assign_account_support_and_request_clarification"
            reason = "Account route is clear, but the access issue is unclear."
        else:
            action = f"assign_account_support_{account_issue.choice}"
            reason = "Account route and account issue are clear."

    else:
        action = "assign_general_support"
        reason = "The ticket does not fit a supported specialist route."

    ignored_answer_ids = tuple(
        question_id
        for question_id in TRIAGE_QUESTIONS
        if question_id not in used_answer_ids
    )

    return TriageDecision(
        route=department.choice,
        action=action,
        priority=priority,
        used_answer_ids=tuple(used_answer_ids),
        ignored_answer_ids=ignored_answer_ids,
        department_probabilities=department.probabilities,
        reason=reason,
    )

Notice what happens on the billing path:

• department, tone, and billing_request are read.
• shipping_issue and account_issue remain in the API response but are deliberately ignored.
• A low-confidence billing_request does not turn into a guessed refund decision. The ticket can still reach billing, but the specialist receives a clarification-oriented action.

This is a more precise policy than treating every returned answer as a command.

---

Issue one Jev request

With the state builder and selector ready, the live call is small. The questions in TRIAGE_QUESTIONS are submitted together in one system_one request.

def triage_ticket(
    ticket: Mapping[str, Any],
    customer: Mapping[str, Any],
) -> TriageDecision:
    # Deterministic short-circuit: no model call is needed.
    if ticket["status"] == "closed":
        return TriageDecision(
            route="other",
            action="no_action",
            priority="normal",
            used_answer_ids=(),
            ignored_answer_ids=tuple(TRIAGE_QUESTIONS),
            department_probabilities={},
            reason="Ticket is already closed.",
        )

    state = build_triage_state(ticket, customer)

    with TypeSafeClient() as client:
        response = client.system_one(
            state=state,
            questions=TRIAGE_QUESTIONS,
        )

    return select_triage_action(response.answers)

A caller can log the decision without treating the model response as an opaque blob:

decision = triage_ticket(ticket, customer)

print(f"route: {decision.route}")
print(f"action: {decision.action}")
print(f"priority: {decision.priority}")
print(f"used: {decision.used_answer_ids}")
print(f"ignored: {decision.ignored_answer_ids}")
print(f"reason: {decision.reason}")
print(f"routing probabilities: {decision.department_probabilities}")

For a ticket routed to billing, an audit record might look conceptually like this:

route: billing
action: assign_billing_and_flag_refund_review
priority: normal
used: ('department', 'tone', 'billing_request')
ignored: ('shipping_issue', 'account_issue')
reason: Customer explicitly requests a refund or credit.
routing probabilities: {
    'billing': 0.81,
    'shipping': 0.14,
    'account': 0.03,
    'other': 0.02,
}

The shipping_issue answer may still exist in response.answers. Do not silently delete it from raw telemetry if it is valuable for later evaluation, but do make it clear in your decision record that it had no role in the chosen action.

---

Treat routing uncertainty differently from detail uncertainty

Fan-out creates two different uncertainty points.

Uncertain answer	Correct policy response
department is uncertain	Do not commit to a specialist branch; use manual triage or a safe fallback
Chosen branch detail is uncertain	Route to the selected team, but do not automate the detail-specific action
Unchosen branch detail is uncertain or confident	Ignore it; it does not apply to the committed route
Global tone is uncertain	Do not use it to elevate priority automatically

This distinction prevents a subtle but common bug.

Suppose Jev returns:

• department: billing, confidence $0.84$
• billing_request: refund_or_credit, confidence $0.91$
• shipping_issue: not_delivered, confidence $1.00$

The correct billing decision can use the first two answers. It must not create a shipping incident merely because the speculative shipping question gave a confident result. That result is evidence about a route your application did not select as primary.

Conversely, consider:

• department: billing, confidence $0.39$
• probabilities: billing $0.60$, shipping $0.38$
• billing_request: refund_or_credit, confidence $1.00$

The billing detail is very clear, but the primary route is not. A high-confidence detail does not rescue an ambiguous router. Manual triage can see that both teams are plausible; your automatic policy should not treat the billing detail as permission to commit.

If your product needs to notify a plausible second team, use the department.probabilities explicitly and separately. For example, a policy might send a non-actionable copy to a secondary team when its probability exceeds a reviewed threshold. That notification is not equivalent to using that team’s branch-specific answer to take an action.

---

Verify with live tickets before broadening the contract

Since your goal is real API use rather than a mock-only workflow, run a small live validation set against your early-access key. Keep the tickets sanitized and representative.

Use at least these cases:

Ticket shape	Expected decision behavior
“I was charged twice. Please refund the duplicate charge.”	Billing route; billing_request used; shipping and account details ignored
“Order A-104 has not arrived after ten days.”	Shipping route; shipping_issue used; billing and account details ignored
“I cannot sign in after resetting my password.”	Account route; account_issue used
“My package is late and I was charged twice.”	Likely lower routing confidence; confirm that manual triage handles ambiguity safely
“Your support is useless.”	Tone may be clear, but route may not be; confirm that tone does not invent a department

For each live result, inspect three things:

1. Was the selected branch sensible?
   Look at both choice and the full routing probabilities, not just the top label.

2. Did your code use exactly the expected answer IDs?
   used_answer_ids and ignored_answer_ids make this visible.

3. Did low confidence lead to a safe fallback?
   A low-confidence route should never trigger a specialist side effect.

Keep these records. In Week 4, they become the beginning of an evaluation dataset and regression suite rather than one-off manual checks.

---

Key takeaways

Speculative fan-out lets you trade a small amount of extra question cost for a single, low-latency Jev request:

• Ask the router and a bounded set of independent branch-detail questions together.
• Keep Jev questions side-effect free and answerable directly from one shared state payload.
• Let deterministic code accept or reject the routing decision using explicit confidence policy.
• Use only the details that correspond to the accepted route.
• Treat unselected branch answers as irrelevant to the current action, even if they are confident.
• Record used and ignored answers so a reviewer can reconstruct why the application acted.
• Distinguish uncertain routing from uncertain route detail; they require different fallbacks.

Next, you will implement candidate re-ranking in Python: using Jev probabilities to reorder a bounded set of candidates while preserving deterministic tie-breaking and explicit cutoffs.