Vectorized Numerical Computing with NumPy

Hello again. In the previous lesson, you created an isolated Python environment, pinned its dependencies, and registered a notebook kernel. That setup now gives you a reliable place to run NumPy code and reproduce what you build.

This lesson introduces the computational style behind much of machine learning: operating on whole numerical arrays rather than writing Python loops that process one value at a time. You will learn to recognize element-wise numerical work, express it with NumPy, validate that it matches a loop-based implementation, and measure why the vectorized version is usually preferable.

---

Why numerical Python code should avoid ordinary loops

Suppose an application records model scores in a Python list and you need to convert every score from a proportion to a percentage. A direct Python approach is familiar:

scores = [0.81, 0.63, 0.94, 0.72]

percentages = []
for score in scores:
    percentages.append(score * 100)

print(percentages)

This works. But most ML work applies the same numerical operation to thousands, millions, or billions of values: image pixels, feature columns, prediction scores, gradients, embeddings, or model weights. Repeating Python-level work for every value introduces substantial interpreter and object-management overhead.

NumPy provides the ndarray, a homogeneous numerical array. Instead of describing the iteration yourself, you state the mathematical operation you want:

import numpy as np

scores = np.array([0.81, 0.63, 0.94, 0.72])
percentages = scores * 100

print(percentages)
# [81. 63. 94. 72.]

This is vectorization: your Python code contains no explicit element-by-element loop. NumPy performs the repeated low-level work in optimized compiled code.

Read the official explanation of the difference between an ordinary Python loop and a vectorized NumPy operation.

Vectorization is not a promise that every element literally runs at the same instant or that every operation automatically uses all CPU cores. Actual performance depends on the operation, array size, CPU, NumPy build, memory layout, and available optimized libraries. The reliable engineering takeaway is simpler: for regular numerical array work, NumPy can avoid the cost of repeated Python interpretation and use efficient low-level routines.

For inference optimization later in your pathway, this distinction becomes very concrete. A model service spends much of its time in optimized tensor kernels, not in Python loops over individual weights or input values. Learning to express computations as array operations now is the first version of that discipline.

---

Arrays have numerical semantics; lists do not

Python lists and NumPy arrays look similar when printed, but their operators mean different things.

python_values = [1, 2, 3]
array_values = np.array([1, 2, 3])

print(python_values * 2)  # [1, 2, 3, 1, 2, 3]
print(array_values * 2)   # [2, 4, 6]

For a list, multiplication means repetition. For a NumPy array, multiplication means element-wise multiplication.

Similarly:

print([1, 2] + [10, 20])              # [1, 2, 10, 20]
print(np.array([1, 2]) + np.array([10, 20]))  # [11, 22]

List addition concatenates containers; array addition adds corresponding numerical values.

This makes NumPy expressions read much like the intended formula:

temperature_c = np.array([18.0, 21.5, 25.0, 19.5])

temperature_f = temperature_c * 9 / 5 + 32
print(temperature_f)
# [64.4 70.7 77.  67.1]

Each operator is applied to each matching array element:

• + performs element-wise addition
• - performs element-wise subtraction
• * performs element-wise multiplication
• / performs element-wise division
• ** performs element-wise powers

A scalar such as 32 or 9 / 5 is applied across the array. NumPy calls this behavior broadcasting; you will examine the shape rules behind it in the next lesson. For now, the safe mental model is: an array combined with one number applies the operation to every entry.

The following official NumPy guide is worth using as a compact reference while you work through the examples.

One important distinction to keep in view: * is element-wise multiplication, not matrix multiplication. Matrix products use @ or np.matmul; you will treat those operations rigorously in the mathematics module.

---

Translating a loop into a vectorized expression

A useful workflow is to first articulate what one loop iteration does, then replace the loop index with an array expression.

Imagine a preprocessing step that converts a batch of product prices in cents to dollars and applies a platform fee of 2.9%.

Loop version

prices_cents = np.array([499, 1299, 2500, 875], dtype=float)

net_prices_loop = np.empty_like(prices_cents)

for i in range(len(prices_cents)):
    price_dollars = prices_cents[i] / 100
    net_prices_loop[i] = price_dollars * (1 - 0.029)

print(net_prices_loop)

The loop’s calculation for position $i$ is:

$${\text{net price}}_i=\frac{{\text{price cents}}_i}{100}(1-0.029)$$

Every position is independent. That independence is the signal that vectorization is appropriate.

Vectorized version

prices_cents = np.array([499, 1299, 2500, 875], dtype=float)

net_prices = (prices_cents / 100) * (1 - 0.029)

print(net_prices)

The vectorized version has no i, no indexing, no append, and no manually allocated output. NumPy applies the same sequence of arithmetic operations to all elements.

When converting loops, use this checklist:

1. Identify the per-item formula. Ignore the mechanics of the loop and state what happens to one value.
2. Check independence. If each output depends only on the corresponding input value or fixed constants, vectorization is likely available.
3. Replace indexed variables with entire arrays. Replace x[i] with x.
4. Use NumPy operations and functions. Arithmetic operators, np.sqrt, np.log, np.exp, and similar functions apply element-wise.
5. Validate the result before deleting the old code.

For example, a common score transformation might require clipping values to a permitted range:

raw_scores = np.array([-0.4, 0.2, 0.8, 1.3])

valid_scores = np.clip(raw_scores, 0.0, 1.0)
print(valid_scores)
# [0.  0.2 0.8 1. ]

np.clip expresses the intent more clearly than a hand-written loop with nested if statements.

---

Vectorized reductions: many values become one result

Not every numerical computation returns an array. A reduction combines all values into a smaller result, often a single number.

latencies_ms = np.array([41.2, 38.5, 44.7, 39.1, 42.6])

print(latencies_ms.sum())   # total
print(latencies_ms.mean())  # average
print(latencies_ms.min())   # smallest value
print(latencies_ms.max())   # largest value
print(latencies_ms.std())   # standard deviation

These operations also avoid writing a Python loop. They matter throughout ML engineering:

• mean() is used in loss calculations and metric summaries.
• sum() is used to accumulate quantities.
• min() and max() help inspect data ranges.
• std() helps describe spread and will later support feature normalization.

For now, use these methods on a whole array. Applying reductions along rows or columns requires a clear understanding of axes, which is deliberately the focus of the next lesson.

---

A realistic ML formula: mean squared error

Vectorization is especially valuable because many ML formulas are defined across a batch of examples.

For regression predictions $\hat{y}$ and true values $y$, mean squared error is:

$$\mathrm{MSE}=\frac{1}{n}\sum _{i=1}^n({\hat{y}}_i-y_i{)}^2$$

Here is the direct NumPy implementation:

predictions = np.array([305_000, 280_000, 410_000, 365_000], dtype=float)
labels = np.array([300_000, 295_000, 400_000, 350_000], dtype=float)

errors = predictions - labels
squared_errors = errors ** 2
mse = squared_errors.mean()

print(mse)

Read it from top to bottom:

1. predictions - labels calculates one error per example.
2. errors ** 2 squares every error.
3. .mean() reduces the squared-error array to one scalar metric.

You could write this in a loop, but the vectorized form makes the formula visible in the code. That is a correctness advantage as well as a performance advantage.

For a compact version, you may write:

mse = np.mean((predictions - labels) ** 2)

During development, however, the staged version is often easier to inspect and debug. If mse seems wrong, printing errors and squared_errors immediately reveals which stage produced an unexpected result.

---

Measure performance, but measure fairly

“Vectorized is faster” is usually true for sizeable regular numerical workloads, but engineering claims should be measured rather than assumed. In a notebook, %timeit is convenient because it runs a statement repeatedly and reports stable timing statistics.

Create notebooks/vectorization.ipynb and run:

import numpy as np

rng = np.random.default_rng(42)

x = rng.random(1_000_000)
y = rng.random(1_000_000)

def multiply_loop(x, y):
    result = np.empty_like(x)
    for i in range(len(x)):
        result[i] = x[i] * y[i]
    return result

def multiply_vectorized(x, y):
    return x * y

First verify that both versions compute the same numerical result:

loop_result = multiply_loop(x, y)
vectorized_result = multiply_vectorized(x, y)

print(np.allclose(loop_result, vectorized_result))
# True

Then time each version in separate notebook cells:

%timeit multiply_loop(x, y)

%timeit multiply_vectorized(x, y)

You should observe a substantial advantage for the vectorized operation, though the exact factor is machine-dependent. The outcome matters less than the pattern:

• the outputs agree;
• the vectorized code is shorter;
• NumPy performs the repeated numerical work outside the Python interpreter.

Do not benchmark tiny arrays and draw broad conclusions. For a three-element array, setup costs and timing noise can dominate. Performance differences become meaningful as numerical workloads grow.

Also avoid timing unrelated work. If a function reads a file, makes a network request, prints thousands of lines, or repeatedly constructs random input data, those activities can overwhelm the cost of the calculation you meant to compare.

---

When a loop is still appropriate

“Use vectorization instead of loops” is a strong default for numerical transformations, not a ban on for.

A Python loop remains reasonable when:

• each step depends on the result of the previous step in a genuinely sequential way;
• you are coordinating I/O, requests, files, or database operations;
• you are processing heterogeneous Python objects rather than a regular numerical array;
• no suitable NumPy operation expresses the logic clearly.

Even then, first check whether NumPy provides a specialized operation. For example, many cumulative and filtering operations have optimized array functions rather than requiring a manually written loop.

The criterion is not “can I eliminate every loop?” It is: am I doing regular arithmetic across a numerical collection? If yes, start by looking for a NumPy expression or function.

---

Key takeaways

NumPy arrays provide numerical semantics that Python lists do not: arithmetic operators act element-wise rather than concatenating or repeating containers. Vectorization means expressing repeated numerical work as array operations rather than explicit Python loops; NumPy then executes the repetitive work in optimized compiled code.

The practical pattern is to translate the formula for one item into an expression over the full array, use reductions such as .mean() when many values should become one summary, and validate a new vectorized implementation against a loop-based baseline with np.allclose.

Next, you will make those operations reliable for real multidimensional data by working with array shapes, axes, indexing, and broadcasting rules.