The Dominated Convergence Theorem: Swapping Limits and Expectations

Hello! Welcome to the fifth lesson in our course.

Introduction

In our last lesson, we contrasted the Riemann and Lebesgue integrals, concluding that the Lebesgue integral's construction—partitioning the codomain—makes it more powerful and general. A key advantage we highlighted was its superior behavior when dealing with limits of functions. We ended by posing a critical question for statistics and econometrics: under what conditions can we swap the order of a limit and an expectation? That is, when is it true that:
${\lim }_{n\to \infty }\mathbb{E}[X_n]=\mathbb{E}\left[{\lim }_{n\to \infty }{X}_n\right]$
Today's lesson directly answers this question by focusing on the Dominated Convergence Theorem (DCT). This addresses the learning outcome: Explain the conceptual importance of the Dominated Convergence Theorem for justifying the interchange of limits and expectations in statistical theory.

You'll see that the DCT isn't just a theoretical curiosity; it's a workhorse theorem that provides the justification for many analytical steps you've likely taken for granted in advanced econometrics and statistics, from deriving moments of distributions to proving the consistency of estimators.

1. The Problem: Why Swapping Limits and Expectations is Tricky

First, let's establish that interchanging limits and expectations is not something we can do freely. A simple counterexample can make this clear.

To translate this into the language of probability theory, consider the probability space $(\Omega ,\mathcal{F},P)$ where $\Omega =(0,1]$, $\mathcal{F}$ is the Borel $\sigma$-algebra, and $P$ is the Lebesgue measure (so the probability of an interval is just its length).

Let's define a sequence of random variables $X_n$ similar to the ones in the video:
$X_n(\omega )=n\cdot 1_{(0,1/n]}(\omega )$
Here, $1_A$ is the indicator function for the set $A$.

• Pointwise Convergence: For any specific outcome $\omega \in (0,1]$, we can always find a large enough $N$ such that for all $n>N$, we have $1/n<\omega$. For these $n$, $X_n(\omega )=0$. Therefore, the sequence of numbers $X_n(\omega )$ converges to 0 for every $\omega$. So, the limit random variable is $X(\omega )=0$.
  ${\lim }_{n\to \infty }{X}_n(\omega )=0\forall \omega \in (0,1]$

• Expectation: Now let's compute the expectation of $X_n$. Since expectation is the Lebesgue integral with respect to the probability measure, we have:
  $\mathbb{E}[X_n]={\int }_{\Omega }{X}_n(\omega )dP(\omega )={\int }_0^{1}n\cdot 1_{(0,1/n]}(\omega )d\omega =n\cdot P((0,1/n])=n\cdot \frac{1}{n}=1$
  The expectation is 1 for all $n$.

Putting it together, we have a stark contradiction:
${\lim }_{n\to \infty }\mathbb{E}[X_n]={\lim }_{n\to \infty }1=1$
But:
$\mathbb{E}\left[{\lim }_{n\to \infty }{X}_n\right]=\mathbb{E}[0]=0$
Clearly, $1\ne 0$. The interchange failed. The intuitive reason is that the "probability mass" of the function escapes "vertically" to infinity, even as its support shrinks. We need a condition to prevent this from happening.

2. The Solution: A Trio of Convergence Theorems

Lebesgue's theory provides three key results that give us a handle on this problem. They build on each other in elegance and utility.

Let's formalize these ideas.

a) The Monotone Convergence Theorem (MCT)

The MCT is the simplest case. It states that if you have a sequence of non-negative random variables $X_n$ that are monotonically increasing to a limit $X$ (i.e., $0\le X_1\le X_2\le \dots$ and $X_n\to X$), then you can swap the limit and expectation:
${\lim }_{n\to \infty }\mathbb{E}[X_n]=\mathbb{E}[X]$
This is intuitive: if the functions are always non-negative and increasing, the mass can't "disappear" or cancel out in some strange way.

b) Fatou's Lemma

Fatou's Lemma is more of a technical tool, but it's the key to proving the DCT. It provides an inequality instead of an equality. For a sequence of non-negative random variables $X_n$, it states:
$\mathbb{E}\left[{\mathrm{liminf}}_{n\to \infty }{X}_n\right]\le {\mathrm{liminf}}_{n\to \infty }\mathbb{E}[X_n]$
This lemma essentially says that in the limit, some mass might "escape," so the expectation of the limit can't be more than the limit of the expectations, but it could be less (as we saw in our counterexample, where $0\le 1$).

c) The Dominated Convergence Theorem (DCT)

This is the most powerful and widely used of the three. It provides a condition that explicitly prevents the "escape of mass" we saw earlier.

The Dominated Convergence Theorem states that if:

1. A sequence of random variables $X_n$ converges to $X$ almost surely (i.e., $P(\{\omega \mid X_n(\omega )\to X(\omega )\})=1$).
2. There exists an integrable random variable $Y$ (meaning $\mathbb{E}[|Y|]<\infty$) that dominates the sequence, i.e., $|X_n(\omega )|\le Y(\omega )$ for all $n$ (almost surely).

Then, you can interchange the limit and expectation:
${\lim }_{n\to \infty }\mathbb{E}[X_n]=\mathbb{E}[X]$

The dominating function $Y$ acts as an "envelope" or "ceiling" that the sequence $X_n$ cannot break through. Since $Y$ has a finite expectation, it guarantees that the expectations of the $X_n$ are uniformly bounded and prevents any probability mass from escaping to infinity.

Let's revisit our counterexample: $X_n=n\cdot 1_{(0,1/n]}$. Is there a dominating function $Y$? The smallest possible dominating function would be the supremum of the sequence, $Y(\omega )={\sup }_n{X}_n(\omega )$. You can see that for $\omega \in (1/2,1]$, $Y(\omega )=1$; for $\omega \in (1/3,1/2]$, $Y(\omega )=2$, and so on. As detailed in the Duke notes (e680a), the expectation of this $Y$ is infinite. No integrable dominating function exists, so the DCT does not apply.

3. The Payoff: DCT in Statistical Theory

The true power of the DCT is revealed when we use it to justify common operations in statistics. A prime example is differentiating under the integral sign.

From your background in econometrics and statistics, you know that we often find moments of a distribution by differentiating its moment-generating function (MGF) or characteristic function (chf). For example, for an MGF $M_X(t)=\mathbb{E}[e^{tX}]$, we claim that $M_X^{\prime }(0)=\mathbb{E}[X]$. Let's see why this requires the DCT.

The derivative is defined by a limit:
$M_X^{\prime }(t)=\frac{d}{dt}\mathbb{E}[e^{tX}]=\frac{d}{dt}\int e^{tx}f(x)dx={\lim }_{h\to 0}\int \frac{e^{(t+h)x}-e^{tx}}{h}f(x)dx$

To proceed, we need to swap the limit and the integral:
$\stackrel{?}{=}\int {\lim }_{h\to 0}\frac{e^{(t+h)x}-e^{tx}}{h}f(x)dx=\int \left(\frac{d}{dt}{e}^{tx}\right)f(x)dx=\int x{e}^{tx}f(x)dx=\mathbb{E}[X{e}^{tX}]$
Setting $t=0$ gives $\mathbb{E}[X]$. The DCT is what justifies the crucial step marked with ?.

As shown in the notes, to justify this interchange, we need to find a dominating function for the sequence of functions inside the integral:
$g_h(x)=\frac{e^{(t+h)x}-e^{tx}}{h}f(x)$
By the Mean Value Theorem, for some $t^{\ast }$ between $t$ and $t+h$, the term $\frac{e^{(t+h)x}-e^{tx}}{h}$ equals $x{e}^{t^{\ast }x}$. If the MGF exists in a neighborhood around $t$, say $[t-\delta ,t+\delta ]$, then we can bound $|x{e}^{t^{\ast }x}|$ by $|x|e^{(t+\delta )|x|}$. The dominating function can then be constructed using this bound, and if its integral (expectation) is finite, the DCT applies.

This same logic is fundamental to:

• Deriving the score function in maximum likelihood estimation by differentiating the log-likelihood.
• Proving properties of estimators, where we often need to show that the limit of a sample average (which is an expectation) converges to a population parameter.
• Analyzing stochastic processes, where we might take limits of conditional expectations over time.

Conclusion

Let's summarize the key takeaways from today's lesson.

• The Problem: Interchanging limits and expectations ($\lim \mathbb{E}[X_n]\stackrel{?}{=}\mathbb{E}[\lim X_n]$) is not always valid. Without the right conditions, probability mass can "escape to infinity," leading to incorrect results.
• The Solution: The Dominated Convergence Theorem (DCT) provides a powerful set of sufficient conditions. If $X_n\to X$ (almost surely) and the entire sequence is "dominated" by a single integrable function $Y$ ($|X_n|\le Y$ with $\mathbb{E}[|Y|]<\infty$), the interchange is justified.
• The Importance: The DCT is the theoretical bedrock for many essential analytical steps in statistics and econometrics. It justifies differentiating under the integral sign to find moments from MGFs/chfs and is a critical tool in the proofs of the asymptotic properties of estimators.

You now have a clear understanding of one of the major "payoffs" of Lebesgue integration. You see not just that we can swap limits and expectations, but why and under what conditions.

Preview of the Next Lesson:
With the DCT in our toolkit, we are now equipped to tackle some of the crown jewels of probability theory. In the next lesson, we will explore the Strong Law of Large Numbers (SLLN). We will see how measure theory allows for a precise statement of this fundamental theorem and how the convergence theorems we've discussed today are essential ingredients in its proof.