Conditional Probability and Bayes’ Rule in Diagnostic Testing

Welcome back. In the previous lesson, you estimated partial derivatives and used the negative gradient to take a step downhill on a loss surface. That was about how to improve a model’s parameters. This lesson shifts to reasoning under uncertainty: how new evidence should change an estimate.

You will calculate conditional probabilities, distinguish the two directions of a conditional statement, and apply Bayes’ rule to a small diagnostic-test example. This is foundational for interpreting classification models later—especially when a rare outcome produces apparently convincing positive predictions.

---

Conditioning changes the reference group

Let $A$ and $B$ be events. The notation

$$P(A\mid B)$$

is read as “the probability of $A$, given $B$.” Once you know that $B$ occurred, you no longer compare against the entire population. You restrict attention to the cases where $B$ is true.

Formally, provided $P(B)>0$,

$$P(A\mid B)=\frac{P(A\cap B)}{P(B)}$$

where $A\cap B$ means that both events occur.

For example, suppose 30 of 100 customers are subscribers, and 18 of those subscribers renew their plan. If $R$ means “renews” and $S$ means “is a subscriber,” then:

$$P(R\mid S)=\frac{18}{30}=0.60$$

The denominator is 30—not 100—because the condition “is a subscriber” narrows the sample space to subscribers.

Order matters

A frequent and consequential error is to reverse the direction of a conditional probability:

$$P(A\mid B)\ne P(B\mid A)$$

For a diagnostic test, define:

• $D$: the person truly has the condition.
• $+$: the test result is positive.

Then:

$$P(+\mid D)$$

means “among people who have the condition, what fraction test positive?” This is the sensitivity or true-positive rate.

But the question a person normally asks after receiving a positive test is:

$$P(D\mid +)$$

meaning “among people with positive test results, what fraction truly have the condition?” These are different questions because they use different denominators.

A test may be highly sensitive, yet a positive result may still have modest evidential value when the condition is rare or when false positives occur.

---

Bayes’ rule: reverse the conditional probability correctly

Bayes’ rule provides a disciplined way to go from $P(+\mid D)$ to $P(D\mid +)$:

$$P(D\mid +)=\frac{P(+\mid D)P(D)}{P(+)}$$

The overall probability of a positive result, $P(+)$, must include both ways a positive result can occur:

1. the person has the condition and tests positive;
2. the person does not have the condition but receives a false positive.

Thus,

$$P(+)=P(+\mid D)P(D)+P(+\mid \overline{D})P(\overline{D})$$

Combining these gives the practical binary-diagnostic form:

$$P(D\mid +)=\frac{P(+\mid D)P(D)}{P(+\mid D)P(D)+P(+\mid \overline{D})P(\overline{D})}$$

The terms have useful names:

Quantity	Meaning
$P(D)$	Prior or base rate: prevalence before seeing the test
$P(+\mid D)$	Likelihood: sensitivity / true-positive rate
$P(+\mid \overline{D})$	False-positive rate
$P(+)$	Evidence: overall chance of a positive result
$P(D\mid +)$	Posterior: updated probability after a positive result

---

A diagnostic example using both probabilities and counts

Consider a fictional screening process with these properties:

$$P(D)=0.02$$ $$P(+\mid D)=0.90$$ $$P(+\mid \overline{D})=0.05$$

In words:

• The condition affects $2\%$ of the population.
• The test is positive for $90\%$ of people who have the condition.
• The test is falsely positive for $5\%$ of people without it.

First calculate the joint probability of having the condition and receiving a positive result:

$$P(D\cap +)=P(+\mid D)P(D)=0.90\times 0.02=0.018$$

Next, calculate the overall rate of positive tests:

$$P(+)=(0.90)(0.02)+(0.05)(0.98)$$ $$P(+)=0.018+0.049=0.067$$

Now Bayes’ rule gives:

$$P(D\mid +)=\frac{0.018}{0.067}\approx 0.269$$

So a positive result corresponds to about a $26.9\%$ probability that the condition is truly present—not $90\%$.

A count-based view makes the denominator intuitive. Imagine exactly 10,000 screened people:

Actual status	Positive test	Negative test	Total
Has condition	180	20	200
Does not have condition	490	9,310	9,800
Total	670	9,330	10,000

Now the two conditional probabilities are visibly different:

$$P(+\mid D)=\frac{180}{200}=0.90$$ $$P(D\mid +)=\frac{180}{670}\approx 0.269$$

The $490$ false positives matter because the condition is uncommon. There are many more people without the condition, so even a $5\%$ false-positive rate creates a substantial number of false alarms.

In later classification lessons, this posterior has a familiar interpretation. If $D$ is the actual positive class and $+$ is a model’s positive prediction, then:

$$P(D\mid +)$$

is the model’s precision (also called positive predictive value) for that population and decision threshold. Its value depends not only on the model’s ability to detect positive cases, but also on the outcome prevalence and false-positive rate.

---

Concept-Level Micro-Challenge: calculate a posterior with NumPy

Type and run this 15-line program. It calculates a conditional probability from the model assumptions, applies Bayes’ rule, and confirms the posterior using expected counts from a 10,000-person population.

import numpy as np
p_disease = 0.02
sensitivity = 0.90
false_positive_rate = 0.05
p_no_disease = 1 - p_disease
p_disease_and_positive = sensitivity * p_disease
p_positive = p_disease_and_positive + false_positive_rate * p_no_disease
p_disease_given_positive = p_disease_and_positive / p_positive
n = 10_000
true_positives = n * p_disease_and_positive
false_positives = n * false_positive_rate * p_no_disease
from_counts = true_positives / (true_positives + false_positives)
print(f"overall +: {p_positive:.3f}, posterior: {p_disease_given_positive:.3f}")
assert np.isclose(p_disease_and_positive / p_disease, sensitivity)
assert np.isclose(from_counts, p_disease_given_positive)

Expected output:

overall +: 0.067, posterior: 0.269

Read the checks as mathematical claims:

• p_disease_and_positive / p_disease recovers $P(+\mid D)$, the sensitivity.
• from_counts calculates $180/(180+490)$, which must agree with the Bayes posterior.
• The code uses expected counts, not a random simulation, so it gives the exact proportions implied by the assumptions.

A useful parameter check: change p_disease from 0.02 to 0.20 while keeping test performance fixed. The posterior will rise sharply. This isolates the effect of the base rate: a positive result contains different information in different populations.

---

Common mistakes to catch now

1. Reversing the condition.
   Sensitivity $P(+\mid D)$ does not answer the post-test question $P(D\mid +)$.

2. Using sensitivity as the final answer.
   A $90\%$ sensitivity means the test detects $90\%$ of true cases. It does not mean that $90\%$ of positive results are true cases.

3. Ignoring false positives.
   The denominator $P(+)$ must count positives from both actual cases and non-cases.

4. Ignoring the base rate.
   A rare event can produce a low posterior probability even after a positive test.

5. Treating association as certainty.
   A posterior probability updates uncertainty; it does not establish a causal explanation or a guaranteed individual outcome.

---

Takeaways

Conditional probability narrows the reference group:

$$P(A\mid B)=\frac{P(A\cap B)}{P(B)}$$

The direction of conditioning matters:

$$P(A\mid B)\ne P(B\mid A)$$

Bayes’ rule reverses a conditional probability by combining the prior rate, likelihood, and total probability of the observed evidence:

$$P(D\mid +)=\frac{P(+\mid D)P(D)}{P(+\mid D)P(D)+P(+\mid \overline{D})P(\overline{D})}$$

For diagnostic and classification settings, a positive prediction must be interpreted in light of false positives and the base rate—not test sensitivity alone.

Next, you will calculate mean, variance, standard deviation, and covariance with NumPy, moving from probability rules to the numerical summaries used to describe real datasets.