Introduction to Measure Theory

Hello! Welcome back to our course.

In the last lesson, we explored the paradoxes that arise when we try to apply our intuition about probability to continuous spaces. We discovered that not only does the "Principle of Indifference" lead to contradictions, but there exist "non-measurable" sets that seem to defy any consistent assignment of size. This demonstrated why we need a more rigorous foundation for probability than simple intuition.

Today's lesson directly addresses that need. Our learning outcome is to: Describe how a 'measure' generalizes the concepts of length, area, and volume to more abstract sets.

We will distill the essential, shared properties of these familiar geometric concepts into a formal, axiomatic definition. This abstract definition of a "measure" is the first building block of the modern framework that underpins the statistical and econometric models you've worked with.

1. The Essential Properties of "Size"

Before we can generalize, we must first identify the fundamental characteristics of the concepts we wish to generalize. What do length, area, and volume have in common? They are all ways of quantifying the "size" of a set of points.

The following reading from Carnegie Mellon University's statistics department does an excellent job of breaking down these core properties.

As the text explains, while properties like translation invariance are characteristic of geometric size in Euclidean space, they are not essential for a more general concept. The indispensable properties are:

1. Non-negativity: The measure of a set $A$, denoted $\mu (A)$, must be greater than or equal to zero. $\mu (A)\ge 0$. A size cannot be negative.
2. Null Empty Set: The measure of the empty set $\varnothing$ is zero. $\mu (\varnothing )=0$. The size of "nothing" is zero.
3. Additivity: For any collection of non-overlapping (pairwise disjoint) sets, the measure of their union is the sum of their individual measures.

This third property, additivity, is the most powerful and requires a crucial refinement. The text distinguishes between finite additivity (holding for any finite number of pieces) and countable additivity (holding for a countably infinite number of pieces). For the purposes of building a theory powerful enough to handle limits and continuous spaces, we require the stronger condition of countable additivity.

You've already seen this in action. In our last lesson, the argument that the set of rational numbers $\mathbb{Q}$ has measure zero relied on covering a countably infinite number of points with intervals whose total length was a countably infinite sum. Without countable additivity, that argument would not hold.

2. The Formal Definition of a Measure

With these essential properties identified, we can now state the formal mathematical definition of a measure.

Let's formalize what you just saw.

A measure $\mu$ is a function that assigns a size to sets. It is defined on a specific collection of "measurable" subsets of a larger set $X$. We call this collection of measurable sets a $\sigma$-algebra, which we will denote by $\mathcal{F}$. We will explore the properties of $\sigma$-algebras in detail in the next module; for now, you can simply think of $\mathcal{F}$ as the "allowable" or "well-behaved" sets for which our measure is defined.

Given a set $X$ and a $\sigma$-algebra $\mathcal{F}$ on $X$, a function $\mu :\mathcal{F}\to [0,\infty ]$ is a measure if it satisfies two conditions:

1. $\mu (\varnothing )=0$
2. (Countable Additivity) For any countable collection of pairwise disjoint sets $A_1,A_2,\dots$ that are all in $\mathcal{F}$, we have: $$\mu \left(\bigcup _{i=1}^{\infty }{A}_i\right)=\sum _{i=1}^{\infty }\mu (A_i)$$

The triplet $(X,\mathcal{F},\mu )$ is called a measure space. This is one of the most fundamental structures in modern analysis and probability.

3. A Menagerie of Measures: From Geometry to Probability

This abstract definition is powerful precisely because it is not limited to geometric contexts. It provides a unified language for quantifying size in many different domains, including the statistical ones you are familiar with.

The Lebesgue Measure: Geometry Generalized

The most direct generalization of length, area, and volume is the Lebesgue measure, denoted ${\mu }_{\text{Leb}}$.

• On $\mathbb{R}$, the Lebesgue measure of an interval $[a,b]$ is its length, $b-a$.
• On ${\mathbb{R}}^2$, the Lebesgue measure of a rectangle $[a,b]\times [c,d]$ is its area, $(b-a)(d-c)$.
• On ${\mathbb{R}}^k$, it gives the corresponding k-dimensional "hyper-volume".

The Lebesgue measure is the measure we implicitly use when working with probability density functions over the real numbers. The integral ${\int }_{A}f(x)dx$ is, as we will see, an integral with respect to the Lebesgue measure.

Abstract Measures

The true power of the definition comes from measures that have no immediate geometric interpretation. The MEASURES document you read earlier provides several excellent examples.

Let's highlight a few of these and connect them to your background:

• Counting Measure: For a set $A$, the counting measure simply gives its cardinality, $\#(A)$. If you have a discrete random variable that takes integer values, the probability mass function is defined on a space where the underlying 'natural' measure is the counting measure on the integers. An integral with respect to a counting measure is simply a sum. This is the first hint that measure theory unifies summation and integration.

• Dirac Measure: The point mass measure, ${\delta }_x$, is defined as ${\delta }_x(A)=1$ if $x\in A$ and 0 otherwise. This is the formal basis for a probability mass function. The probability of a discrete outcome $k$ occurring, $P(X=k)=p_k$, can be thought of as being built from a measure $p_k\cdot {\delta }_k$. This measure assigns a "lump" of size $p_k$ to the point $k$ and zero everywhere else.

• Probability Measure: This is the most important example for our purposes. A probability measure $\mathbb{P}$ is simply any measure on a space $\Omega$ with the additional property that the measure of the entire space is 1, i.e., $\mathbb{P}(\Omega )=1$.

  • The Binomial measure and Gaussian (Normal) measure mentioned in the text are prime examples. When you say a random variable is distributed $N(\mu ,{\sigma }^2)$, you are defining a specific probability measure on the real line. The probability of that variable falling into a set $A$ is the Gaussian measure of that set: $$\mathbb{P}(X\in A)={\int }_A\frac{1}{\sqrt{2\pi {\sigma }^2}}{e}^{-\frac{(x-\mu {)}^2}{2{\sigma }^2}}dx$$

  This integral calculates the "size" of the set $A$ as weighted by the Gaussian density function. All the named distributions you studied in econometrics and statistics are, at their core, probability measures.

Conclusion

In this lesson, we have taken a significant step from the "why" to the "what". We have constructed a formal, abstract definition of a measure by identifying the essential properties of familiar concepts like length, area, and volume.

Key Takeaways:

• A measure is a function that assigns a non-negative "size" to sets, where the size of the empty set is zero and the size of a countable union of disjoint sets is the sum of their individual sizes (countable additivity).
• This abstract definition successfully generalizes geometric concepts. The Lebesgue measure corresponds to our intuitive ideas of length, area, and volume.
• More importantly, the definition is broad enough to include non-geometric measures that are fundamental to probability, such as the counting measure, the Dirac measure (point mass), and, most generally, any probability measure.
• The concepts of summation and integration can both be seen as integration with respect to different measures (counting measure and Lebesgue measure, respectively).

Preview of the Next Lesson:

In the next lesson, we will address the final learning outcome of this module: "Identify the limitations of Riemann integration that motivate the need for a more powerful theory of integration for probability." We will see why the standard integral from calculus is insufficient for the needs of advanced probability theory and how our new concept of a measure is the key to building a more powerful replacement: the Lebesgue integral.