Linux Early Memory Initialization: From Memblock to Buddy and SLUB

Welcome back. In the previous lesson, Linux had just crossed the ARM64 handoff boundary: U-Boot passed the physical DTB address in x0, early assembly enabled the MMU, and execution reached start_kernel() with a minimal kernel runtime in place.

Now Linux faces a foundational problem: DRAM exists, but it is not yet a general pool from which the kernel can safely allocate memory. The kernel must first determine which physical ranges are usable, protect everything that must survive boot, create bookkeeping for every usable page, and only then make free pages available to the normal allocators.

By the end of this lesson, you should be able to trace that progression from Device Tree memory information through memblock, the buddy page allocator, and finally SLUB object caches. This is the allocation substrate beneath everything that follows: driver probing, process creation, VFS, networking, and your eventual gateway services.

---

DRAM is present, but initially it is only a claim on a map

The bootloader has initialized DRAM electrically and loaded the kernel into it. Linux has enabled its early mappings. Yet none of that means the kernel can call its usual allocation interfaces safely.

At this point, Linux must answer four questions:

1. Which physical address ranges are RAM that Linux may use?
2. Which ranges must never be allocated?
3. How will Linux represent and manage each usable physical page?
4. Which allocator should serve page-sized versus small-object requests?

On an ARM64 board, the first answer normally comes from the DTB. The DTB’s /memory node describes installed RAM available to Linux, while /reserved-memory and other boot information describe areas that must remain unavailable to general allocation.

Typical reserved regions include:

• the executing kernel image, its data, and its early page tables;
• the DTB itself while Linux still needs it;
• an initramfs, if one was passed;
• a Contiguous Memory Allocator (CMA) region reserved for devices that need physically contiguous DMA buffers;
• shared DMA pools;
• firmware carveouts, remote-processor memory, or device-specific buffers;
• no-map regions, which Linux must not include in its normal direct physical-memory mapping.

For an AM62x-class gateway, a Device Tree error in a memory or reserved-memory range can be severe. If Linux treats a firmware carveout, DMA pool, or remote-processor region as ordinary free RAM, the failure may surface later as corrupted Ethernet traffic, broken media pipelines, remoteproc crashes, or an intermittent kernel panic. The original cause is not necessarily near the eventual symptom.

Pages and PFNs

Linux manages normal RAM in units called pages. On common ARM64 configurations, the base page is often $4\mathrm{KiB}$, though this is a kernel build choice rather than an ARM64 universal constant.

A physical address can be expressed as a page frame number (PFN):

$$\mathrm{PFN}=\frac{\mathrm{physical}\mathrm{address}}{\mathrm{PAGE}\mathrm{SIZE}}$$

The PFN identifies a physical page, while Linux’s metadata records what that page currently represents: free memory, a kernel allocation, page-cache data, a DMA buffer, a page-table page, and so on.

The transition in this lesson is therefore not merely “memory allocation starts.” It is a conversion from a firmware-described collection of byte ranges into a managed population of page frames and then into efficiently reusable kernel objects.

---

memblock: Linux’s allocator before the normal allocators exist

Early boot cannot use kmalloc() or alloc_pages(), because those interfaces depend on the page allocator that has not been initialized yet. Linux instead uses memblock, a deliberately simple early-boot memory manager.

Memblock represents physical memory as sorted, contiguous ranges. Its primary logical collections are:

Collection	Meaning
memory	Physical RAM Linux is allowed to manage
reserved	Regions excluded from general allocation
physmem	On architectures that support it, physical RAM detected during boot even if later restricted

The fundamental free-memory expression is:

$$\mathrm{free}\mathrm{memblock}\mathrm{range}=\mathrm{memory}\cap \neg \mathrm{reserved}$$

This model is intentionally direct. Memblock does not initially need sophisticated per-page free lists, reclaim logic, or allocator caches. It needs to prevent overlap while providing contiguous memory for structures required to finish boot.

How ARM64 reaches a valid memblock map

The exact functions vary by kernel version and vendor tree, but the conceptual trace is stable:

Stage	Kernel responsibility	Result
Early FDT scan	Read RAM ranges from /memory	Usable ranges are registered with memblock
Reservation discovery	Reserve the kernel, DTB, initramfs, /reserved-memory areas, and early allocations	Protected ranges cannot be reused
Command-line processing	Apply restrictions such as mem= where applicable	Usable RAM may be reduced deliberately
Early architecture setup	Allocate metadata, permanent page tables, and other boot structures through memblock	More regions become reserved
Page-management setup	Prepare page descriptors, memory zones, and allocator structures	The normal page allocator can be made operational
Final handover	Release only genuinely free pages to the buddy allocator	Normal allocation begins

A useful engineering rule is:

memblock.memory says what Linux could manage; memblock.reserved says what Linux must protect; only their difference becomes general-purpose page allocator memory.

Memblock allocations are themselves reservations. If early code allocates a region for page metadata or a permanent mapping structure, memblock records it so a later allocator cannot overwrite it.

The documentation distinguishes two common API families:

• memblock_phys_alloc*() returns a physical address.
• memblock_alloc*() returns a virtual address through Linux’s available mapping of physical memory.

That distinction matters around MMU bring-up. A DMA engine, a page-table register, and an ordinary kernel pointer do not necessarily consume the same form of address.

Memblock is current; bootmem is historical vocabulary

Older Linux explanations often describe an early allocator called bootmem. Modern Linux kernels use memblock for this boot-time role. You may still encounter bootmem in legacy architecture code, older vendor documentation, or discussions of kernel history. Treat it as useful context, but do not assume a current ARM64 kernel necessarily has a separate active “bootmem phase” after memblock.

---

From ranges to managed pages: struct page, memory models, and zones

Memblock knows about ranges. The buddy allocator, however, needs to manage individual pages and page groups. Linux therefore creates page metadata, conventionally represented by struct page.

For each valid physical page, its metadata eventually lets Linux track information such as:

• whether the page is free, reserved, allocated, or used by a compound allocation;
• reference counts and mapping state;
• flags describing the page’s role;
• its NUMA node and memory zone;
• its relationship to the buddy allocator’s free lists.

This bookkeeping has a cost: the metadata itself occupies RAM. Linux must allocate and reserve it before it can declare the corresponding pages generally allocatable. That apparent circularity is exactly why memblock exists.

Memory models are build-time representations

Linux supports multiple physical-memory models. The two most relevant conceptual models are:

Model	Idea	Typical relevance
FLATMEM	One global logical page-metadata array spans physical memory	Simple systems with contiguous or mostly contiguous RAM
SPARSEMEM	Physical memory is divided into sections, supporting holes and large address spaces efficiently	Common on modern 64-bit systems and required for several advanced features

With SPARSEMEM_VMEMMAP, Linux presents page metadata through a virtually contiguous vmemmap region even when the underlying metadata pages are not physically contiguous. The details are architecture and configuration dependent, but the debugging consequence is general:

A page may exist physically, yet Linux may not regard it as a valid allocatable page unless its metadata was initialized as part of the configured memory model.

Zones are allocation policy domains, not simply physical partitions

Linux also divides managed pages into zones. A zone groups pages with similar addressing or allocation constraints. Zone composition varies by architecture and kernel configuration. Common names include DMA, DMA32, Normal, and sometimes Movable.

For an embedded ARM64 system, zones matter because a device may have addressing limitations. For example, an older or constrained DMA master may need buffers below an address limit, while normal kernel allocations can use a broader set of RAM.

Do not assume that every AM62x system has the same visible zones or sizes. Inspect the actual target configuration and boot log. The stable idea is that Linux must classify pages into appropriate zones before it can satisfy allocation requests with the required constraints.

---

The handover: making free pages available to the buddy allocator

The crucial transition occurs when architecture-specific initialization, conventionally centered around mem_init(), hands genuinely free pages to the page allocator. The operation is commonly represented by:

memblock_free_all();

This does not mean that all DRAM is freed. It means that all pages in memblock’s usable-memory ranges that are not reserved are released into the buddy allocator’s zone-managed free lists.

Before that can happen safely, Linux has already done the necessary preparation:

1. It has identified valid RAM ranges.
2. It has recorded reservations.
3. It has allocated page metadata and allocator bookkeeping.
4. It has initialized page descriptors for valid PFNs.
5. It has established zones and their free-area structures.
6. It has marked permanently reserved pages as unavailable.

Only then can Linux place free pages into the allocator.

The diagram is useful, but avoid interpreting it as a one-way linear pipeline. Once boot completes, these mechanisms coexist:

• Buddy manages free physical pages and contiguous power-of-two page blocks.
• SLUB obtains backing pages from buddy and divides them into small kernel objects.
• vmalloc obtains pages, usually from the page allocator, and maps them into a virtually contiguous range that need not be physically contiguous.
• Some kernel code uses buddy directly through page-allocation APIs rather than passing through SLUB.

For example, a large physically contiguous DMA buffer may require an alloc_pages()-style request, while an inode or a task_struct is generally allocated from a slab cache.

---

Buddy allocation: efficient management of physical page blocks

The buddy allocator is Linux’s general allocator for physical pages. It manages blocks sized as powers of two in units of base pages.

An allocation has an order:

Order	Number of base pages	Allocation size
0	$1$	$1\times \mathrm{PAGE}\mathrm{SIZE}$
1	$2$	$2\times \mathrm{PAGE}\mathrm{SIZE}$
2	$4$	$4\times \mathrm{PAGE}\mathrm{SIZE}$
$n$	$2^n$	$2^n\times \mathrm{PAGE}\mathrm{SIZE}$

If the kernel requires an order-2 allocation, it needs four physically contiguous base pages. If no order-2 block is free, buddy can find a larger free block and repeatedly split it until it produces the requested order. The unused halves return to the relevant free lists.

When freeing memory, buddy checks whether the block’s same-sized partner, its buddy, is also free. If so, it merges the pair into the next higher-order block. This recursive splitting and merging limits external fragmentation better than maintaining unrelated arbitrary-size blocks.

In Linux, each zone maintains free areas organized by order. Allocation requests start at the requested order and seek a larger order only if needed. This makes a ready matching block fast to obtain, while preserving a structured way to split larger blocks and later coalesce them.

The important limitation is physical contiguity. A system might report plenty of free memory in total while still failing a high-order allocation because free pages are scattered into small blocks. This is a frequent practical distinction in device-driver work:

• Free memory answers “how much?”
• Available high-order blocks answer “how contiguous?”

After boot, /proc/buddyinfo shows the number of free blocks by zone and order. It is often more useful than /proc/meminfo when investigating failures of large contiguous DMA or graphics allocations.

---

SLUB: turning pages into efficient kernel objects

The buddy allocator is inappropriate for every small kernel request. Allocating a whole page for a structure that is only a few hundred bytes wastes memory and creates unnecessary page-level allocation traffic.

Linux therefore layers an object allocator on top. Most modern Linux configurations use SLUB, selected by CONFIG_SLUB.

SLUB organizes frequently allocated objects into caches. A cache serves one object size or one particular object type. It obtains one or more backing pages from the buddy allocator, divides that memory into fixed-size objects, and maintains free objects for rapid reuse.

Common examples include caches for:

• task_struct objects used to represent tasks and threads;
• inodes and dentries used by the VFS;
• networking structures;
• device and driver-model objects;
• general kmalloc() size classes.

The interfaces reflect this distinction:

Interface	What it expresses	Typical use
alloc_pages()	Request a physically contiguous page block	Page tables, DMA-oriented buffers, page-cache internals
kmem_cache_alloc()	Request an object from a specific typed cache	Kernel structures with known layout
kmalloc()	Request a general small kernel allocation	Flexible-size internal buffers
vmalloc()	Request virtually contiguous memory	Large kernel virtual allocations without physical contiguity

Why kmalloc() is not “kernel malloc” in the userspace sense

kmalloc() often draws from SLUB’s size-class caches. A request is rounded to an appropriate internal object size, then served from a cache if possible. This is fast, but it can introduce internal fragmentation: a request for a slightly smaller size may occupy a larger cache object.

The trade-off is appropriate for kernel execution:

• allocation and free operations are very fast;
• frequently reused objects tend to preserve cache locality;
• per-CPU fast paths reduce shared-lock contention;
• backing memory can ultimately be supplied by buddy-managed pages.

SLUB commonly keeps a fast per-CPU allocation state, so an allocation on one CPU can often be serviced without immediately taking a global lock. It also has node-level management for partial slabs. This hierarchy matters on multicore SoCs: allocation scalability is a core design goal, not a cosmetic optimization.

The sequence is therefore:

Layer	Granularity	Main purpose
memblock	Contiguous physical ranges	Safe early boot discovery, reservations, and allocations
buddy allocator	Powers-of-two physical page blocks	General physical-page management
SLUB	Fixed-size kernel objects	Fast small-object allocation and reuse
vmalloc	Virtually contiguous mappings	Large virtual regions from potentially noncontiguous pages

During initialization, kmem_cache_init() bootstraps the slab allocator and establishes essential caches, including the infrastructure needed to support general kmalloc() allocations. It can do this only because buddy-managed pages are now available.

---

Evidence on a running target

A boot log is the best evidence for the early sequence because /proc reflects the post-boot state. On a running development image, these commands provide complementary views:

dmesg | grep -Ei 'memblock|reserved mem|Memory:|Zone|SLUB|CMA'
cat /proc/iomem
cat /proc/zoneinfo
cat /proc/buddyinfo
cat /proc/slabinfo | head -n 20

Use each output for a different purpose:

Evidence	What it can confirm
dmesg	Early RAM discovery, reservations, CMA initialization, memory totals, and SLUB startup messages
/proc/iomem	The kernel’s current physical resource map, including System RAM and reserved regions
/proc/zoneinfo	Zone configuration, watermarks, and per-zone accounting
/proc/buddyinfo	Free blocks by order; evidence of physical fragmentation
/proc/slabinfo	Active slab caches, object sizes, and cache occupancy

Record the first boot log before beginning peripheral enablement on a new board. It is a valuable baseline for later device-tree and memory-carveout changes. In a private bring-up repository, retain the raw serial log and the exact kernel commit, DTB hash, and boot arguments. A public portfolio can later show a sanitized memory-map diagram and conclusions without publishing sensitive production carveout addresses.

Source-navigation landmarks

In a kernel tree, function placement varies somewhat by release and vendor patches, but these paths are durable starting points:

cd <kernel-source>

git grep -n "memblock_free_all"
git grep -n "kmem_cache_init"
git grep -n "void __init mm_init"
git grep -n "memblock_add" arch/arm64
git grep -n "reserved-memory" drivers arch/arm64

Then inspect the surrounding code in these broad locations:

init/main.c
arch/arm64/mm/
mm/memblock.c
mm/page_alloc.c
mm/slub.c
mm/slab_common.c

Follow the call chain rather than relying on one exact label. For your selected kernel revision, establish these facts:

• where /memory ranges are added to memblock;
• where /reserved-memory regions are reserved or initialized;
• where page metadata and zones are prepared;
• where memblock_free_all() occurs;
• where kmem_cache_init() makes SLUB available.

---

Key takeaways

Linux early memory initialization is a controlled ownership transfer, not a single allocator call.

• The DTB describes RAM and reserved regions; Linux must convert those descriptions into a safe physical-memory map.
• Memblock manages contiguous memory ranges during boot, maintaining distinct memory and reserved collections.
• Before pages can be allocated normally, Linux creates and initializes per-page metadata, zones, and allocator bookkeeping.
• memblock_free_all() releases only memory that is both usable and unreserved into the buddy allocator.
• The buddy allocator manages physically contiguous page blocks in powers of two and can split or merge buddy blocks to manage fragmentation.
• SLUB is layered on buddy allocation. It turns backing pages into fast reusable caches of small kernel objects and supports kmalloc()-style allocation.
• vmalloc is not a replacement for buddy or SLUB; it supplies virtually contiguous mappings using pages that may not be physically contiguous.

Next, you will follow the systems that rely on this allocation substrate: driver-model population, the creation of core kernel threads through rest_init(), VFS root mounting, and the transition to PID 1.