Understanding Price Indices and Deflation Measurement

Hello! Welcome back to our course on deflation.

Introduction

In our last lesson, you built a Python script to download US CPI data and programmatically identify periods of deflation. We learned how to find deflation in the data. Today, we ask a more fundamental question: Can we trust the data itself?

This lesson directly addresses the learning outcome: "Critically evaluate the construction and limitations of common price indices (e.g., CPI, GDP deflator) and the impact of measurement biases on identifying deflation."

We will deconstruct the very tools we use to measure inflation, exploring how they are built and, more importantly, where their flaws lie. For an economist and statistician like yourself, this critical perspective is paramount. A seemingly small measurement bias of 0.5% can be the difference between concluding an economy is in low inflation versus dangerous deflation. Understanding these nuances is essential for a rigorous analysis of Japan's Lost Decades.

This lesson should take approximately 60 minutes to complete.

Recap from Last Lesson:
You successfully used the FRED API to fetch the US CPI series (CPIAUCSL), calculated its year-over-year percentage change to get the inflation rate, and created a plot highlighting periods where this rate fell below zero (deflation).

---

1. The Anatomy of a Price Index: Constructing the CPI

Before we can critique a tool, we must understand how it's made. The Consumer Price Index (CPI) is the most cited measure of inflation, but its construction is a complex statistical process.

To start, let's get a high-level overview of the steps involved, from surveying households to calculating the final index number.

Please watch the section from 1:37 to 6:27. This provides a clear, step-by-step explanation of the process: the expenditure survey, forming the consumer basket, weighting the goods, and calculating the index.

As the video explains, the process is essentially:

1. Survey: A statistical agency (like the ONS in the UK or the BLS in the US) surveys thousands of households to see what they buy.
2. Basket: A representative "basket" of several hundred goods and services is created based on these spending patterns.
3. Weights: Each item in the basket is assigned a weight corresponding to its share of total household expenditure. Fuel, for example, has a much higher weight than movie tickets.
4. Price Collection: Each month, the prices of the items in the basket are collected from thousands of retail outlets.
5. Index Calculation: The total cost of the basket in the current month is compared to its cost in a designated "base year" to create an index value.

The change in this index value over time is the inflation rate.

The Underlying Formula

Conceptually, the standard CPI is a Laspeyres Price Index. Given your background in statistics and economics, you'll recognize the formula:

$P_L=\frac{\sum (p_{i,t}\cdot q_{i,0})}{\sum (p_{i,0}\cdot q_{i,0})}$

Where:

• $p_{i,t}$ is the price of item $i$ at time $t$.
• $p_{i,0}$ is the price of item $i$ in the base period.
• $q_{i,0}$ is the quantity of item $i$ consumed in the base period.

The key takeaway is that the quantities ($q_{i,0}$) are from the base period. The basket is fixed. This simple fact is the source of several major biases.

---

2. The Achilles' Heel of Fixed Baskets: Substitution Bias

What happens when the price of chicken rises, but the price of beef stays the same? Rational consumers might buy less chicken and more beef. However, a Laspeyres index, with its fixed basket from a past period, assumes they keep buying the same amount of the now more-expensive chicken. This overstates the true increase in their cost of living. This is substitution bias.

This short, intuitive video provides an excellent explanation of the concept.

1. Highlight 1: 0:00 - 0:00
2. Highlight 2: 0:00 - 0:00
3. Highlight 3: 0:00 - 0:00
4. Highlight 4: 0:00 - 0:00

Watch the video from 0:00 to 4:44. It uses a simple two-good model to perfectly illustrate how using a fixed basket for comparison leads to a biased conclusion about price levels.

As the video explains, using a fixed basket (e.g., the "French basket") to measure prices in another context (Thailand) overstates the price level because it ignores the fact that consumers in Thailand will substitute away from goods that are expensive there.

The same logic applies over time. The CPI basket is based on past consumption patterns. As relative prices change, consumers adjust their current consumption, but the CPI calculation doesn't capture this adjustment in real-time. The result is that the CPI has a tendency to overstate inflation.

For a concise summary, please read the following text.

4.2 Measuring the economy: Inflation – Macroeconomics (Issues in Measuring Inflation: Quality Changes and Substitution Bias)
Focus on the paragraph under the heading "Issues in measuring inflation" that begins with "Substitution bias." This provides a clear, textbook definition of the concept.

Implication for Deflation: If CPI systematically overstates inflation, it will systematically understate deflation. An economy might be experiencing a -0.2% deflation rate, but a biased CPI could report it as +0.1% inflation, leading policymakers to misdiagnose the economic situation.

---

3. Alternative Measures: The GDP Deflator and PCE

To address the limitations of the CPI, economists use other indices. The two most important are the GDP Deflator and the Personal Consumption Expenditures (PCE) Price Index.

The GDP Deflator

The GDP deflator measures the prices of all goods and services produced domestically. Unlike the CPI, its basket is not fixed; it changes every period based on what the economy is producing.

4.2 Measuring the economy: Inflation – Macroeconomics (GDP Deflator: Definition and Scope)
Please read the section defining the GDP deflator. Pay close attention to the differences in scope compared to the CPI (e.g., it includes exports but excludes imports).

The GDP deflator is an example of a Paasche Price Index:

$P_P=\frac{\sum (p_{i,t}\cdot q_{i,t})}{\sum (p_{i,0}\cdot q_{i,t})}$

Notice that it uses current quantities ($q_{i,t}$). This solves substitution bias but introduces other issues; for instance, it's harder to interpret as a pure "cost of living" index because changes can be driven by shifts in production, not just prices.

The PCE Price Index

The Federal Reserve's official 2% inflation target is based on the PCE Price Index, not the CPI. The PCE has a broader scope than the CPI (e.g., it includes healthcare services paid for by employers) and uses a chain-weighted formula (a Fisher index) that is superior at accounting for substitution. For these reasons, many economists consider it a more accurate measure of consumer inflation.

To understand these different measures, please read the following section from the PIIE working paper.

and Evaluating Recent Critiques of the Consumer Price Index (Other Widely Used Measures of Inflation)
Reading time: ~5 minutes. Please read the section titled "Other Widely Used Measures of Inflation". It provides an excellent overview of CPI variants (like Core CPI and Chained CPI) and contrasts the CPI with the PCE price index.

---

4. A Deeper Critique of the CPI

Beyond substitution bias, several other measurement issues can distort our view of inflation. The Sichel & Mackie paper provides a fantastic, modern analysis of these critiques.

A. Quality Adjustment & New Goods Bias

How do you compare the price of a 2024 smartphone to one from 2014? The new phone is vastly more powerful. If the price is 20% higher, is that inflation, or are you paying for a much better product? Statistical agencies use "hedonic quality adjustments" to try and account for this, but it's an incredibly difficult task. Generally, it's believed that the CPI does not fully account for quality improvements, leading to another source of upward bias (overstating inflation).

Similarly, the fixed basket of the CPI is slow to incorporate new products. The consumer welfare gained from entirely new inventions (from the microwave to the internet) is not captured, which is another form of measurement bias.

B. The Housing Conundrum

Housing is the largest expense for most households, accounting for over a third of the CPI. Yet, it's measured in a unique and often criticized way. For renters, the BLS tracks rent. But for homeowners, it doesn't track house prices or mortgage payments. Instead, it uses Owners' Equivalent Rent (OER)—an estimate of how much a homeowner would have to pay to rent their own home.

This approach is conceptually sound (it separates the consumption cost of shelter from the investment aspect of owning a house), but it has practical problems. The OER measure is known to lag actual changes in market rents by a year or more.

and Evaluating Recent Critiques of the Consumer Price Index (Critiques of the CPI; Proposed Steps to Modernize)
Reading time: ~10 minutes. Please read the section "Critiques of the CPI; Proposed Steps to Modernize," focusing specifically on Criticism 3 regarding the measurement of shelter. This gives a detailed account of the OER methodology and why it generates controversy and lags.

This lag is not just a theoretical curiosity. In your previous lesson, you plotted recent inflation. The high shelter inflation that persisted in the CPI long after market rents started to cool is a direct consequence of this measurement methodology.

---

5. Synthesis: Putting Theory into Practice

Let's consolidate our understanding with a summary and a practical exercise.

Feature	Consumer Price Index (CPI)	GDP Deflator	Personal Consumption Exp. (PCE)
Index Type	Laspeyres (fixed base-year basket)	Paasche (current-year basket)	Fisher (chain-weighted)
Key Bias	Substitution Bias (overstates inflation)	Less substitution bias, but harder to interpret	Mitigates substitution bias
Scope	Consumer out-of-pocket spending (incl. imports)	All domestically produced goods/services (incl. exports)	All consumer spending (incl. on-behalf-of)
Weights	Updated annually (with a lag)	Change each period	Change each period
Use Case	Widely cited "cost of living" index	Measuring economy-wide price levels	Fed's preferred inflation target

Coding Exercise: CPI vs. PCE

Let's see the difference in the data. Modify the Python script from your last lesson to download and plot both CPI and PCE inflation on the same graph.

1. The FRED series ID for CPI is CPIAUCSL.
2. The FRED series ID for PCE is PCEPI.
3. Use the same units='pc1' transformation for both to get the year-over-year percentage change.

import pandas as pd
import matplotlib.pyplot as plt
from fredapi import Fred

# --- Your FRED API Key ---
fred_key = 'YOUR_API_KEY_HERE'
fred = Fred(api_key=fred_key)
plt.style.use('seaborn-v0_8-whitegrid')

# --- Download Data ---
start_date = '1960-01-01'
cpi_inflation = fred.get_series('CPIAUCSL', observation_start=start_date, units='pc1')
pce_inflation = fred.get_series('PCEPI', observation_start=start_date, units='pc1')

# --- Create Plot ---
fig, ax = plt.subplots(figsize=(14, 8))

ax.plot(cpi_inflation.index, cpi_inflation, label='CPI Inflation', color='blue', linewidth=1.5)
ax.plot(pce_inflation.index, pce_inflation, label='PCE Inflation', color='red', linewidth=1.5)

ax.axhline(0, color='grey', linestyle='--', linewidth=1)
ax.set_title('CPI vs. PCE Inflation (Year-over-Year %)', fontsize=16)
ax.set_xlabel('Year', fontsize=12)
ax.set_ylabel('Percentage Change', fontsize=12)
ax.legend()

plt.show()

# --- Analysis ---
# Calculate the average difference
avg_diff = (cpi_inflation - pce_inflation).mean()
print(f"On average, CPI inflation has been {avg_diff:.4f} percentage points higher than PCE inflation.")

Analysis Question:
Run the script. You will observe that the blue line (CPI) is almost always above the red line (PCE). Based on what you've learned in this lesson, what are the two primary reasons for this systematic difference?

Click to reveal the answer

1. Formula/Substitution Bias: The CPI's Laspeyres formula doesn't account for consumer substitution, while the PCE's chain-weighted formula does. This imparts a known upward bias to the CPI relative to the PCE.
2. Scope and Weights: The two indices cover different goods and services and weight them differently. For example, the PCE gives a much higher weight to healthcare because it includes expenditures made on behalf of consumers by employers and the government. These different weighting schemes contribute to the divergence.

---

Conclusion

You have now moved from being a user of economic data to a critical evaluator of it. This is a crucial step for any serious analyst.

Key Takeaways:

• No price index is a perfect measure of the "true" cost of living. Each is a model with specific assumptions and resulting biases.
• The CPI (a Laspeyres index) uses a fixed basket, making it vulnerable to substitution bias, quality adjustment bias, and new goods bias, all of which tend to make it overstate inflation.
• The GDP Deflator (a Paasche index) and PCE Price Index (a Fisher index) use different formulas that better account for substitution, but they measure different baskets of goods.
• The measurement of key components, especially housing (OER), can introduce significant lags and distortions into the headline inflation number.
• The choice of index matters. As you saw in the exercise, the PCE consistently tells a story of lower inflation than the CPI. When analyzing deflation, this difference can be the deciding factor.

Preview of the Next Lesson:

We have now established an empirical framework and critically evaluated our measurement tools. In the next lesson, we will turn to one of the foundational theories of the price level: the Quantity Theory of Money (MV=PY). We will empirically test the stability of one of its core assumptions—the velocity of money (V)—using data from a major deflationary period.