Why Riemann Integration Falls Short for Probability

Hello! Welcome back.

In our last lesson, we established a formal definition of a measure as a way to generalize concepts like length, area, and volume. We saw how this abstract definition could encompass not only geometric size (Lebesgue measure) but also concepts central to probability, like the counting measure for discrete outcomes and probability measures for random variables.

Today, we complete our motivational module by addressing the question: if we have a measure, why can't we just use the standard integral from calculus? The learning outcome for this lesson is to: Identify the limitations of Riemann integration that motivate the need for a more powerful theory of integration for probability.

You're well-acquainted with the Riemann integral from your studies in mathematics and economics. We'll examine where this familiar tool falls short, particularly when dealing with the kinds of functions and limiting arguments that are foundational to modern probability and statistics.

1. The Riemann Integral and its Breaking Point

Let's start with a quick refresher. The Riemann integral calculates the area under a curve by partitioning the domain (the x-axis) into small vertical rectangles and summing their areas. The integral is the limit of this sum as the width of the rectangles approaches zero.

This method works beautifully for continuous functions and functions with a finite number of "jumps." But what happens when a function is pathologically discontinuous?

The following video introduces the classic counterexample: the Dirichlet function. It's a function that is "all jumps."

As the video explains, the Dirichlet function, $f(x)$, is defined on $[0,1]$ as:

$$f(x)=\{\begin{array}{ll}1& \text{if }x\in \mathbb{Q}\text{ (rational)}\\ 0& \text{if }x\notin \mathbb{Q}\text{ (irrational)}\end{array}$$

The problem for Riemann integration is that any interval on the real line, no matter how small, contains both rational and irrational numbers.

• When we form the upper Riemann sum (taking the supremum of $f(x)$ in each subinterval), the height of every rectangle is 1. The total area is 1.
• When we form the lower Riemann sum (taking the infimum), the height of every rectangle is 0. The total area is 0.

Since the upper and lower sums never converge to the same value, the function is not Riemann integrable.

To see this laid out more formally, you can review the following text.

This first limitation is critical: The Riemann integral fails for functions with "too many" discontinuities. While the Dirichlet function might seem like a contrived mathematical curiosity, the theoretical framework of probability must be robust enough to handle such functions.

2. The Problem with Limits

A more profound weakness of Riemann integration, especially for a field like analysis that is built on limits, is how it behaves when we take the limit of a sequence of functions. Ideally, we would want the space of integrable functions to be complete—that is, the limit of a sequence of integrable functions should also be integrable. The Riemann integral fails this test.

The next video demonstrates this failure powerfully. It constructs the non-integrable Dirichlet function as the pointwise limit of a sequence of perfectly well-behaved, Riemann-integrable functions.

Let's break down the argument:

1. Let $\{r_1,r_2,r_3,\dots \}$ be an enumeration of all rational numbers in $[0,1]$.
2. Define a sequence of functions $f_n(x)$ such that: $$f_n(x)=\{\begin{array}{ll}1& \text{if }x\in \{r_1,\dots ,r_n\}\\ 0& \text{otherwise}\end{array}$$
3. Each function $f_n$ is Riemann integrable. It is zero everywhere except at a finite number of points. The integral of each $f_n$ is 0, since the "area" at a finite number of single points is zero. $${\int }_0^1{f}_n(x)dx=0\text{for all }n$$
4. The pointwise limit of this sequence as $n\to \infty$ is the Dirichlet function $f(x)$, because eventually every rational number is included. $$\underset{n\to \infty }{\lim }{f}_n(x)=f(x)=\{\begin{array}{ll}1& \text{if }x\in \mathbb{Q}\\ 0& \text{if }x\notin \mathbb{Q}\end{array}$$

We have a sequence of Riemann-integrable functions whose limit is not Riemann integrable. This leads to a major theoretical problem concerning the interchange of limits and integrals. Notice that:

$$\underset{n\to \infty }{\lim }{\int }_0^1{f}_n(x)dx=\underset{n\to \infty }{\lim }0=0$$

But,

$${\int }_0^1\left(\underset{n\to \infty }{\lim }{f}_n(x)\right)dx={\int }_0^{1}f(x)dx\text{(which does not exist!)}$$

This inability to reliably swap limits and integrals, $\lim \int \leftrightarrow \int \lim$, is a severe handicap. The theorems that allow this swap for Riemann integrals require very strong conditions, like uniform convergence, which are often not met in the stochastic settings you've encountered in econometrics and time series analysis.

3. Why This Matters for Probability and Statistics

These limitations are not just abstract mathematical problems; they are direct obstacles to building a rigorous theory of probability.

• Expectation is an Integral: The expected value of a function of a random variable, $E[g(X)]$, is defined by an integral. For a continuous random variable with PDF $p(x)$, this is $E[g(X)]=\int g(x)p(x)dx$. We need an integration theory that can handle the wide variety of functions and densities that arise in practice.

• Limit Theorems are Central: The foundational results of statistics—the Law of Large Numbers (LLN) and the Central Limit Theorem (CLT)—are statements about the limits of sequences of random variables. Their proofs fundamentally rely on being able to swap limits and expectations (integrals). A theory of integration where this is difficult or impossible is not fit for the job.

The solution, developed by Henri Lebesgue, was to rethink integration from the ground up. Instead of partitioning the domain (the x-axis), Lebesgue's brilliant insight was to partition the codomain (the y-axis).

Let's revisit the first video, which gives a beautiful analogy for this conceptual shift.

The analogy is perfect:

• Riemann: Takes bills and coins out of a pocket in the order they are found.
• Lebesgue: Sorts all the bills and coins by value first, then pays them out in sorted groups.

For the Dirichlet function, the Lebesgue approach is simple:

1. What is the "size" of the set of points where $f(x)=1$? This is the set $\mathbb{Q}\cap [0,1]$. As we discussed in a previous lesson, the measure of this countable set is 0.
2. What is the "size" of the set of points where $f(x)=0$? This is the set of irrationals in $[0,1]$. The measure of this set is 1.
3. The Lebesgue integral is then the sum of each value multiplied by the measure of the set where it occurs: $${\int }_{[0,1]}fd\mu =(1\times \text{measure}(\mathbb{Q}))+(0\times \text{measure}(\mathbb{R}\setminus \mathbb{Q}))=(1\times 0)+(0\times 1)=0$$

This is a clean, unambiguous result that matches our intuition from the limit of the $f_n$ integrals. This new approach, built on the concept of a measure that we defined last time, solves the problems that plague the Riemann integral.

Conclusion

In this lesson, we've identified the critical weaknesses of the Riemann integral that necessitate a more powerful framework for probability theory.

Key Takeaways:

• Failure on Discontinuous Functions: The Riemann integral is not defined for functions with dense discontinuities, like the Dirichlet function, because its upper and lower sums fail to converge.
• Incompleteness under Limits: The set of Riemann-integrable functions is not "closed" under pointwise limits. A sequence of integrable functions can converge to a non-integrable function.
• Difficulty with Limit Interchange: Swapping limits and integrals ($\lim \int \leftrightarrow \int \lim$) is fundamental for proving limit theorems in probability. This operation is highly restricted under Riemann integration but is handled much more elegantly in Lebesgue theory via powerful results like the Monotone and Dominated Convergence Theorems (which we will meet later).

These deficiencies are the primary motivation for developing the Lebesgue integral, which is built on the foundation of measure theory.

Preview of the Next Lesson:

We have now established the "why" of measure theory. We've seen the paradoxes that arise from naive probability and the limitations of classical integration. In the next module, "The Core Machinery," we will begin building the "how." The first step is to formalize the collection of sets to which we can assign a measure. This leads us to the crucial concept of a $\sigma$-algebra, which will be the topic of our next lesson.