Interpreting Jev Responses: Values, Probabilities, and Confidence

Hello again. Your first live Jev call returned a bounded choice answer. This lesson is about reading that answer correctly before you connect it to any routing or automation.

A Choice response carries three related but distinct signals:

• the selected value your code can branch on;
• the probability distribution across every allowed option;
• a derived confidence value that summarizes how concentrated or spread out that distribution is.

By the end, you will be able to inspect the complete typed response from your support-ticket script and explain why “Jev chose billing” is not the same statement as “billing has probability 0.84” or “the answer has confidence 0.596.”

---

A Choice answer is a compact decision record

Return to the script from the previous lesson. Your question declared a closed set of possible categories:

category: choice("What is this ticket about?", {
  billing: "Payment, refund, or subscription issue",
  technical: "Product bug or integration issue",
  account: "Login or account access issue",
  other: "None of the listed categories",
}),

Jev’s response is returned under the question ID, category. Conceptually, its shape is:

{
  type: "choice",
  choice: "billing",
  probabilities: {
    billing: 0.84,
    technical: 0.159,
    sales: 0.001,
  },
  confidence: 0.596,
}

The exact values will vary by state, question definition, and call. This example illustrates how to interpret the fields, not an expected result for the earlier ticket.

Read the official Choice documentation now. Its response section establishes the exact semantics of these fields.

A useful way to retain the distinction is this:

Field	What it means	What it does not mean
choice	The option with the greatest returned probability	Permission to perform an irreversible action
probabilities	Jev’s complete distribution over the options you supplied	A list of arbitrary alternatives to ignore
confidence	A scalar summary of how concentrated the distribution is	The same number as the winning option’s probability, or a guarantee that the answer is correct

The selected choice is the winner of the distribution. If billing has $0.84$, technical has $0.159$, and sales has $0.001$, then billing is selected because it has the largest value.

The values in probabilities form a distribution:

$$\sum _{\text{option}}P(\text{option})=1$$

For the example:

$$0.84+0.159+0.001=1.0$$

This is useful because the runner-up may carry business meaning. A ticket primarily about an integration failure could also mention a charge, so a technical primary assignment with non-trivial billing probability may justify notifying billing or displaying a richer triage view. That is materially different from treating every non-selected option as zero.

---

Inspect the full response in TypeScript

In src/route-ticket.ts, replace the simple final console.log with the following inspection code. Leave the request itself unchanged.

const answer = response.answers.category;

const rankedOptions = Object.entries(answer.probabilities)
  .sort(([, leftProbability], [, rightProbability]) => {
    return rightProbability - leftProbability;
  });

const probabilityTotal = rankedOptions.reduce(
  (sum, [, probability]) => sum + probability,
  0
);

console.log("Selected category:", answer.choice);
console.log(
  "Selected category probability:",
  answer.probabilities[answer.choice]
);
console.log("Confidence:", answer.confidence);
console.log("Probability total:", probabilityTotal);

console.table(
  rankedOptions.map(([option, probability]) => ({
    option,
    probability,
    selected: option === answer.choice,
  }))
);

Run it again:

npm run check
npm run start

You should now see all three signals separately. The table makes it particularly easy to inspect the ranking rather than only the winning value.

A likely output shape is:

Selected category: billing
Selected category probability: 0.91
Confidence: 0.84
Probability total: 1

The values above are illustrative. Do not hard-code an expectation that the response will always choose the same value or return the same numeric values on repeated calls.

Two implementation details matter here:

1. answer.choice is safe to use as a key into answer.probabilities, because it is one of the declared Choice options.
2. The small difference between probabilityTotal and exactly 1 that you might occasionally see in more complex numerical work is usually ordinary floating-point representation. For display and normal application logic, treat the response as the documented probability distribution rather than trying to reconstruct it yourself.

For an application UI, logging all options is useful during development. In production, be deliberate: internal probability distributions may be valuable in an operator-facing decision inspector, but not necessarily appropriate to expose directly to an end user.

---

Why confidence is separate from probability

It is tempting to read a response such as this:

{
  choice: "billing",
  probabilities: {
    billing: 0.84,
    technical: 0.159,
    sales: 0.001,
  },
  confidence: 0.596,
}

and conclude that confidence ought to be 0.84. It is not.

The selected probability answers a narrow question:

Of the allowed options, how much probability was placed on the selected option?

Confidence instead summarizes the shape of the full distribution. A sharply concentrated distribution signals a clearer separation among the alternatives. A flatter distribution signals that multiple answers remain plausible.

Read the Confidence documentation’s explanation of that shape relationship.

Consider these two conceptual distributions over the same support categories:

Distribution shape	Selected value	Interpretation
billing: 0.98, technical: 0.01, account: 0.01, other: 0.00	billing	A strong, isolated winner
billing: 0.41, technical: 0.35, account: 0.20, other: 0.04	billing	Billing technically wins, but competing interpretations are substantial

Both responses select billing. They should not necessarily cause the same application behavior.

The first may be suitable for automatic placement into the billing queue, assuming the action is low risk. The second could still be placed into billing as the primary queue, while also being marked for triage or sent to an operator-facing worklist. The difference is not the selected value; it is the uncertainty expressed across the distribution and summarized by confidence.

Three important boundaries follow:

• Confidence is not a correctness guarantee. A high-confidence response can still be wrong, especially if the relevant evidence was missing or the options were poorly defined.
• Confidence is not a universal threshold. A safe threshold for choosing a help-center article may be inadequate for initiating a financial action.
• Confidence is not a substitute for the distribution. Two responses may deserve different handling even if their scalar confidence values are similar, because the second-ranked option differs.

Later in the course, you will evaluate confidence against labeled data and set thresholds based on real error and review costs. For now, the essential discipline is to preserve all three fields rather than discarding the response after reading choice.

---

Give each field a distinct role in application code

A clean architecture keeps semantic judgment and product policy separate.

Jev provides the observation:

const answer = response.answers.category;

Your application owns the policy:

const REVIEW_CONFIDENCE_FLOOR = 0.6;
const SECONDARY_QUEUE_PROBABILITY_FLOOR = 0.25;

const primaryQueueByCategory = {
  billing: "support-billing",
  technical: "support-technical",
  account: "support-account",
  other: "support-general-triage",
} as const;

const secondaryCandidates = Object.entries(answer.probabilities)
  .filter(([option, probability]) => {
    return (
      option !== answer.choice &&
      probability >= SECONDARY_QUEUE_PROBABILITY_FLOOR
    );
  })
  .map(([option]) => option);

const triageDecision = {
  primaryQueue: primaryQueueByCategory[answer.choice],
  selectedCategory: answer.choice,
  selectedProbability: answer.probabilities[answer.choice],
  confidence: answer.confidence,
  needsReview: answer.confidence < REVIEW_CONFIDENCE_FLOOR,
  secondaryCandidates,
};

console.dir(triageDecision, { depth: null });

The constants here are intentionally examples, not production recommendations. Their value is in demonstrating the distinct responsibilities:

Response property	Deterministic application use
choice	Select the primary branch or queue
probabilities	Surface meaningful alternatives, prioritize review context, or collect richer diagnostics
confidence	Choose whether to proceed normally, ask for clarification, or route to review

Notice what the code does not do:

• It does not ask Jev to create queues.
• It does not let Jev authorize a refund, issue a credit, or mutate customer data.
• It does not assume choice alone is sufficient to automate every downstream action.
• It does not use the model’s probability values as a replacement for deterministic business rules.

The distinction also prevents a common observability mistake. This log is weak:

console.log({ category: answer.choice });

It answers only, “What did the system do?” A more useful structured log is:

console.log({
  event: "support_category_assessed",
  selectedCategory: answer.choice,
  probabilities: answer.probabilities,
  confidence: answer.confidence,
});

That record can later reveal whether low-quality routes arose from close decisions, missing categories, ambiguous ticket text, or a policy threshold that was too aggressive.

One final primitive-specific caution: this lesson concerns Choice. Choice and Score answers include a probability distribution and a derived confidence. A Noul question represents the probability of one precise proposition, such as whether a ticket expresses urgency; it does not use the same Choice response shape or carry a separate confidence field. Do not write generic response-handling code that assumes every Jev primitive has choice and confidence.

---

Key takeaways

A Choice response is more informative than a category label:

• choice is the option with the highest returned probability.
• probabilities contains the full distribution over every option you declared, and the values sum to $1$.
• confidence is a separate statistic derived from the distribution’s shape; it is not the winning probability and not a guarantee of correctness.
• Use the selected value for deterministic branching, probabilities for competing interpretations, and confidence as an uncertainty signal for your policy.
• Preserve the complete response in development logs or internal diagnostics so that you can later evaluate decisions rather than merely observe outcomes.

Next, you will set up the same authenticated Jev pattern in a uv-managed Python project and make a live request with the Python SDK.