Consistent Shape Notation for Scalars, Vectors, Matrices, and Tensors

Welcome. This first module builds the mathematical language used throughout machine learning: how we describe data, how dimensions constrain computations, and later how models learn from change and uncertainty.

Today’s goal is foundational but operationally important: represent scalars, vectors, matrices, and tensors with unambiguous shape notation. In speech systems, a shape such as (batch, time, features) is not incidental metadata; it is part of the contract between data loading, a model, and inference code.

---

One organizing idea: data has axes

A tensor is a numerical container organized along zero or more axes. The number of axes determines its array dimensionality:

• 0 axes: scalar
• 1 axis: vector
• 2 axes: matrix
• 3 or more axes: higher-order tensor

The visualization below shows the central idea: each additional axis creates another organized direction in which data is indexed. A three-dimensional array is best imagined as a stack of two-dimensional tables, not necessarily as a physical cube.

A useful caution: people use dimension in two distinct ways.

1. A tensor’s number of dimensions means its number of axes. A tensor with shape (8, 100, 80) has three axes, so it is three-dimensional.
2. A vector’s dimension often means its number of components. A vector with shape (80,) is a one-axis tensor, but it is an 80-dimensional vector.

When ambiguity is possible, prefer saying number of axes, shape, or length of an axis.

---

Shape notation: the compact data contract

The shape of an array is a tuple listing the size of each axis, from first to last. In NumPy, inspect it with .shape; inspect the number of axes with .ndim.

Here is the consistent notation we will use:

Object	Mathematical notation	NumPy/PyTorch-style shape	Meaning
Scalar	$\alpha \in \mathbb{R}$	()	One number
Vector	$\mathbf{x}\in {\mathbb{R}}^d$	(d,)	Ordered list of $d$ numbers
Matrix	$\mathbf{A}\in {\mathbb{R}}^{m\times n}$	(m, n)	$m$ rows and $n$ columns
3-axis tensor	$\mathsf{X}\in {\mathbb{R}}^{a\times b\times c}$	(a, b, c)	Array indexed along three axes

For example, consider:

$$\mathbf{A}=\left[\begin{array}{ccc}2& 5& 1\\ 7& 3& 4\end{array}\right]$$

This has two rows and three columns, so:

$$\mathbf{A}\in {\mathbb{R}}^{2\times 3}$$

and its array shape is:

(2, 3)

The total number of stored elements is the product of the shape values:

$$2\times 3=6$$

For a tensor of shape (8, 100, 80), the number of elements is:

$$8\times 100\times 80=64,000$$

Shape answers how data is organized. Data type answers what kind of value is stored, such as float32, int64, or bool. Keep those concepts separate. Two tensors can have identical shapes but different data types and radically different meanings.

---

Scalars, vectors, matrices, and tensors in code

Although ordinary Python numbers are scalars, they do not carry a .shape attribute. To work consistently with array libraries, create a zero-dimensional NumPy array when you need a scalar tensor.

import numpy as np

gain = np.array(0.75, dtype=np.float32)
print(gain)
print(gain.shape)
print(gain.ndim)

Expected interpretation:

0.75
()
0

The empty tuple () means: “there are no axes.” This is still one stored value.

A vector is a one-axis array. It has an ordered sequence of entries:

frame = np.array([0.12, -0.08, 0.31, 0.05], dtype=np.float32)

print(frame.shape)  # (4,)
print(frame.ndim)   # 1
print(frame[0])     # 0.12

Mathematically, if the vector has four entries, we can write:

$$\mathbf{x}=\left[\begin{array}{c}{x}_1\\ x_2\\ x_3\\ x_4\end{array}\right]\in {\mathbb{R}}^4$$

In Python, indexing is zero-based, so frame[0] selects the first value. In mathematical notation, authors usually begin indexing at one, so $x_1$ denotes the first component. The convention changes, but the underlying ordering does not.

A matrix has two axes. The first index selects a row; the second selects a column.

features = np.array(
    [
        [0.12, -0.08, 0.31],
        [0.05,  0.22, 0.17],
    ],
    dtype=np.float32,
)

print(features.shape)  # (2, 3)
print(features[1, 2])  # 0.17

The value selected by features[1, 2] is a scalar: after providing one index for each axis, no axes remain.

selected_value = features[1, 2]
print(selected_value.shape)  # ()

This “one index per axis” rule scales cleanly:

batch = np.zeros((8, 100, 80), dtype=np.float32)

print(batch.shape)       # (8, 100, 80)
print(batch[3].shape)    # (100, 80)
print(batch[3, 10].shape)  # (80,)
print(batch[3, 10, 7].shape)  # ()

Each integer index removes the axis it selects:

• batch[3] chooses one item from the first axis.
• batch[3, 10] chooses one item and one time position.
• batch[3, 10, 7] chooses one numerical value.

That is one reason clear axis ordering is essential: the same shape numbers have no meaning until you specify what each axis represents.

---

Speech-model examples: shapes carry semantics

Speech machine learning is particularly shape-heavy because audio has time structure, models often operate in batches, and each time step may contain many features.

At this stage, treat the names below as labels for axes; later modules will explain the underlying audio transformations.

Tensor	Typical shape	Axis meanings
One waveform	(T,)	T audio samples over time
Batch of equal-length waveforms	(B, T)	B examples, T samples per example
One frame-level feature sequence	(F, M)	F frames, M features per frame
Batch of feature sequences	(B, F, M)	batch, frames, features
Batch of model representations	(B, F, D)	batch, frames, learned feature width

For example, a common representation in speech processing might be:

$$\mathsf{X}\in {\mathbb{R}}^{B\times F\times M}$$

where:

• $B$ is the batch size,
• $F$ is the number of time frames,
• $M$ is the number of features for each frame.

Suppose a training batch has 16 clips. Each clip has been represented by 200 frames, and each frame has 80 features:

speech_features = np.zeros((16, 200, 80), dtype=np.float32)

The shape (16, 200, 80) alone does not tell another engineer what the axes mean. A robust interface documents the semantic contract:

# Shape: (batch, frames, features)
speech_features = np.zeros((16, 200, 80), dtype=np.float32)

This is a small habit with large consequences. In a model pipeline, a mistaken ordering such as (batch, features, frames) may not always raise an error. Code can run while silently treating time as features and features as time. Shape assertions and explicit axis names are therefore practical correctness tools, not merely mathematical formality.

For audio represented directly as samples, many PyTorch-oriented systems use a convention like:

# Shape: (batch, channels, samples)
audio = np.zeros((16, 1, 16000), dtype=np.float32)

Here, 1 means mono audio. Stereo audio could use two channels:

# Shape: (batch, channels, samples)
stereo_audio = np.zeros((16, 2, 16000), dtype=np.float32)

There is no universal axis order across all libraries. The important discipline is to establish a convention, document it, and validate it at component boundaries.

---

The subtle distinction: vector versus row or column matrix

A common early source of bugs is assuming that a one-dimensional array is automatically a row vector or a column vector.

v = np.array([10, 20, 30])
print(v.shape)  # (3,)

This is a one-dimensional vector with one axis. It is not a two-dimensional row matrix.

Compare it with these arrays:

row = np.array([[10, 20, 30]])
column = np.array([[10], [20], [30]])

print(row.shape)     # (1, 3)
print(column.shape)  # (3, 1)

Their shapes are different because they have different numbers of axes:

Representation	Shape	Interpretation
v	(3,)	1D vector with three entries
row	(1, 3)	2D matrix with one row and three columns
column	(3, 1)	2D matrix with three rows and one column

This distinction becomes crucial in the next lesson, where multiplication depends on compatible shapes.

It also explains a NumPy behavior that surprises many people:

print(v.T.shape)       # (3,)
print(row.T.shape)     # (3, 1)
print(column.T.shape)  # (1, 3)

Transposition swaps axes. A genuine one-dimensional vector has only one axis, so NumPy has nothing to swap; v.T remains shape (3,). A row matrix and a column matrix have two axes, so their transpose changes shape.

Mathematical texts often treat vectors as column vectors by default. In NumPy and PyTorch, however, a tensor with shape (d,) has no explicit row/column orientation. When orientation matters, use a two-axis representation such as (d, 1) or (1, d).

---

A reliable shape-reading workflow

When you encounter any tensor in code, do not infer its meaning from variable names alone. Read it in a fixed order:

1. Inspect the shape. How many axes are present, and how long is each one?
2. Name each axis. For example: (batch, frames, features).
3. Identify one element. What does indexing all axes return? It should generally be a scalar.
4. Check the data type. Audio features are commonly floating-point; class labels are often integers.
5. Confirm the expected contract at boundaries. A loader, feature extractor, model, and decoder should agree about order and size.

In NumPy, this often looks like:

def describe_tensor(name, x):
    print(f"{name}:")
    print(f"  shape = {x.shape}")
    print(f"  axes  = {x.ndim}")
    print(f"  size  = {x.size}")
    print(f"  dtype = {x.dtype}")

describe_tensor("speech_features", speech_features)

During future PyTorch work, the equivalent shape reasoning will remain unchanged even though you will also track a tensor’s device, such as CPU or GPU, and whether it participates in gradient computation.

---

Key takeaways

• A scalar, vector, matrix, and higher-order tensor have zero, one, two, and three-or-more axes respectively.
• Shape notation lists the size of each axis in order: (d,), (m, n), and (B, F, M).
• A vector of shape (d,) is not the same as a row matrix (1, d) or a column matrix (d, 1).
• The shape is a data contract only when every axis has a stated meaning.
• In speech systems, common axis names include batch, channels, samples, frames, features, and learned representation width.

Next, we will use these shapes to determine when dot products and matrix products are valid—and what shape their results must have.