The Paradox of Continuous Probability

Hello! Welcome to the first lesson in our course on measure theory.

Your goal is to understand what measure theory is and how it connects to the statistics and econometrics you've studied. This course is designed to build that bridge, starting from the foundational problems that motivated the theory in the first place.

This first lesson addresses the following learning outcome: Explain the paradoxes that arise when trying to assign equal probability to all points in a continuous interval, leading to the idea of non-measurable sets.

We'll explore why our intuition for probability, which works well for discrete cases like coin flips or dice rolls, breaks down when we move to continuous spaces. You've worked extensively with continuous random variables and their probability density functions in your econometrics and statistics courses. This lesson will delve into the fundamental reasons why we need the machinery of PDFs and can't simply assign a probability to each individual outcome.

Let's begin by examining the intuitive principle we often use to assign probabilities when we feel we are in a state of ignorance.

1. The Principle of Indifference

A common-sense starting point for assigning probabilities is the Principle of Indifference. As formulated by John Maynard Keynes, it states that if we have no known reason to favor one outcome over another, we should assign them equal probabilities. This works perfectly for a fair die roll: with no reason to believe one face is more likely than another, we assign P(1) = P(2) = ... = P(6) = 1/6.

However, this seemingly obvious principle can quickly lead to contradictions, even in simple discrete cases.

As you saw, the paradox arises from how we choose to partition the space of possibilities. Applying the principle to the partition {red, not-red} gives P(red) = 1/2. Applying it to {black, not-black} gives P(black) = 1/2. Since red and black are mutually exclusive, their probabilities must sum to a value less than or equal to 1, but here 1/2 + 1/2 is already 1, leaving no room for any other colors.

This problem becomes far more acute when we move from a finite set of colors to a continuous interval of real numbers.

2. Paradoxes in Continuous Spaces

Let's consider picking a random number from the interval [0, 1]. What is the probability of picking exactly 0.5?

This question exposes a deep paradox at the heart of continuous probability.

The video articulates the problem perfectly:

• If the probability of picking any specific number is some positive value $p>0$, then the total probability sums to infinity, because there are uncountably many numbers in the interval.
• If the probability of picking any specific number is 0, then the total probability sums to 0, not 1.

Both conclusions contradict the basic axioms of probability. The video hints at the solution: we must talk about the probability of a range of values, which corresponds to the area under a probability density function (PDF). You are familiar with PDFs from your work in statistics, but measure theory is what provides the rigorous foundation for this approach.

The problem is even more subtle than just assigning probability to single points. The book-cover paradox was about choosing the "right" partition. In a continuous space, there is often no uniquely "natural" way to apply the Principle of Indifference, a problem known as a reparameterization paradox.

This is a crucial insight. Wavelength and frequency are two equally valid ways to describe color. However, applying the Principle of Indifference (i.e., assuming a uniform distribution) to one parameterization leads to a non-uniform distribution for the other. There is no a priori reason to prefer one over the other, so we cannot derive a unique probability assignment from ignorance alone. Bertrand's paradox, which you read about, is a classic geometric illustration of this very same problem: different, seemingly valid ways of "choosing a chord at random" lead to different answers for the probability calculation.

These paradoxes show that our intuition is failing us. We need a more robust way to assign "size" or "weight" to subsets of the real line.

3. Zero-Measure Sets: The Rationals

The modern approach to resolving these paradoxes begins by accepting that the probability of any single point in a continuous distribution is indeed zero. This leads to the question: how can we build up to a non-zero probability for an interval if all its individual points have probability zero?

The key lies in the fact that not all infinite sets are "the same size." Some are "countable" (like the integers or rational numbers), while others are "uncountable" (like the real numbers). This difference has profound consequences for our ability to measure them.

Let's see how we can assign a "length" or measure to the set of all rational numbers.

This is a remarkable result. Let's summarize the key steps:

1. Countable Additivity: A measure must have the property that for a countable collection of disjoint sets, the measure of their union is the sum of their individual measures.
2. The Argument: The set of rational numbers $\mathbb{Q}$ is countably infinite. This means we can list them: $q_1,q_2,q_3,\dots$.
3. We can cover each rational number $q_n$ with a small interval of length $ϵ/2^n$.
4. The total length of this covering is the sum of a geometric series:
   ${\sum }_{n=1}^{\infty }\frac{ϵ}{2^n}=ϵ{\sum }_{n=1}^{\infty }{\left(\frac{1}{2}\right)}^n=ϵ\cdot 1=ϵ$
5. Since the set of all rational numbers is contained within this covering, its true "length" or measure must be less than or equal to $ϵ$.
6. But $ϵ$ can be any arbitrarily small positive number. The only non-negative number that is smaller than every positive number is 0.
7. Therefore, the measure of the set of rational numbers is 0.

This shows that even though the rational numbers are dense in the real line (between any two reals there's a rational), they take up zero "space". If the measure of the interval [0,1] is 1, and the measure of the rationals within it is 0, then the measure of the irrationals must be 1. This helps resolve the paradox: if you pick a number at random from [0,1], the probability of it being rational is 0, and the probability of it being irrational is 1.

4. The Limit: Non-Measurable Sets

We've seen that we can assign a measure of 0 to the set of rational numbers. This might lead us to hope that we can assign a consistent measure (a generalization of length) to any subset of the real numbers.

Unfortunately, this is not the case. Using a foundational axiom of modern mathematics, the Axiom of Choice, it's possible to construct truly pathological sets that defy any attempt to be measured.

Let's walk through the logic of that construction, as it's the culmination of today's learning outcome.

1. Axiom of Choice: This axiom allows us to create a new set by picking exactly one element from each set in a collection of non-empty sets. It's essential for this construction.
2. The Setup: We partition the entire circle into disjoint sets called "orbits," where each orbit consists of points that can be reached from each other by a rational rotation.
3. Construction: We use the Axiom of Choice to construct a set, $X$, by picking exactly one point from each of these orbits.
4. The Paradox:
   • If we take $X$ and all its (countably infinite) rational rotations, these rotated sets are all disjoint from one another, and their union is the entire circle.
   • Let's assume a measure exists that is rotation-invariant (rotating a set doesn't change its size) and countably additive. Let the measure of $X$ be $m(X)$.
   • By rotation invariance, every rotated copy of $X$ must also have measure $m(X)$.
   • By countable additivity, the measure of the whole circle must be the sum of the measures of all these copies.
   • Case 1: If $m(X)=0$, then the sum of the measures of all the copies is $0+0+\cdots =0$. This is a contradiction, as the circle's circumference is $2\pi$.
   • Case 2: If $m(X)>0$, then the sum of the measures of all the copies is an infinite sum of a positive number, which is $\infty$. This is also a contradiction.

Since both possibilities lead to a contradiction, our initial assumption must be wrong. We cannot assign a consistent measure to the set $X$. It is a non-measurable set.

Conclusion

In this lesson, we've journeyed from intuitive ideas about probability to the strange frontiers of mathematics.

Key Takeaways:

• The simple Principle of Indifference fails in continuous spaces, leading to paradoxes when a system can be described by different but equally valid parameters (e.g., wavelength vs. frequency).
• Assigning a non-zero probability to every individual point in a continuous interval leads to a contradiction with the axioms of probability.
• A first step towards a solution is to assign a measure (a generalized length) to sets of points. We saw that some infinite sets, like the rational numbers, have a measure of zero.
• The existence of non-measurable sets (like Vitali sets), which can be constructed using the Axiom of Choice, shows that we cannot consistently assign a measure to every possible subset of the real line.

This final point is the critical motivation for the next stage of our journey. If we can't assign a measure to every set, we must restrict our attention to a collection of "well-behaved" sets that we can measure.

Preview of the Next Lesson:

In our next lesson, "Describe how a 'measure' generalizes the concepts of length, area, and volume to more abstract sets," we will formalize the properties that any "measure" must have. This will set the stage for defining the specific collections of measurable sets ($\sigma$-algebras) that form the bedrock of modern probability theory.