Distinguishing Request Types and Completion Requirements

Welcome back. Last time, you separated PCIe into Transaction, Data Link, and Physical layers. The key boundary for this lesson is that a Data Link Layer ACK only confirms correct delivery across one adjacent link; it is not a device-level response to the original request.

Now we focus on the Transaction Layer and the request types you will encounter while managing a PCIe switch, NIC, or storage controller. By the end, you should be able to identify whether an operation is a configuration request, memory request, or message, and predict whether the requester must receive a Completion TLP.

Plan for roughly 40 minutes, including the video and reading.

---

A request is not always a round trip

At the software level, reads and writes can look similar:

value = readl(register);
writel(value, register);

At the PCIe transaction level, though, they are fundamentally different.

A PCIe request is either:

• Non-posted: the requester must receive a Completion TLP from the final destination.
• Posted: the requester does not wait for a Completion TLP.

“Posted” does not mean “the device has definitely finished the operation.” It means only that PCIe does not return a transaction-layer completion to the requester.

This distinction is particularly important for a management application. A write to a switch or NIC control register may be accepted into the PCIe fabric, yet the device may later reject, defer, or fail to act on its new value. For a posted write, software validates the effect using a documented status register, a safe readback, a device event, or an error-reporting path, not an automatic write response.

Completion versus Data Link ACK

Keep these two acknowledgements separate:

Mechanism	Layer	Scope	What it establishes
ACK DLLP	Data Link	One physical link, between neighboring ports	The neighboring port received a TLP correctly.
Completion TLP	Transaction	From the ultimate completer back to the requester, potentially across switches	A non-posted request received a transaction-layer response, often including read data or a completion status.

A PLX switch may be an intermediate forwarding device for a host-to-NIC read. In that case, it does not generate the Completion. The NIC is the completer. However, when the host accesses the switch's own configuration space or a switch-owned MMIO register, the switch itself is the completer.

---

The three request families that matter most

PCIe defines several transaction categories. For normal Linux management and device operation, the three central ones are Configuration, Memory, and Message requests.

Configuration requests: discover and control PCIe functions

A Configuration Request accesses a PCIe function's configuration space. Firmware and the operating system use these requests during enumeration, and drivers or management tools use them to inspect and set standardized PCIe controls.

Examples include:

• Reading Vendor ID and Device ID to determine whether a function exists.
• Reading a switch Downstream Port's Link Status register.
• Writing a PCIe Command register bit.
• Enabling an error-reporting capability.
• Reading a PCIe capability structure.

Configuration requests use the function's identity within the hierarchy, primarily its bus, device, and function number. The details of Type 0 versus Type 1 configuration requests and BDF routing come in the next module, but the immediate rule is simple:

Configuration Reads and Configuration Writes are non-posted. They require a Completion TLP.

A configuration read normally receives a Completion with Data, containing the requested configuration value. A configuration write receives a Completion without Data, reporting the result of processing that write.

This is an important exception to a tempting but incorrect rule: not every write is posted. Configuration writes require a Completion; memory writes do not.

Memory requests: MMIO and DMA

A Memory Request accesses an address in the PCIe memory address space. This is the transaction family used for both:

1. CPU-initiated MMIO, such as a Linux driver accessing registers exposed through a BAR.
2. Device-initiated DMA, such as a NIC writing received packets to host memory or a storage controller fetching a command from host memory.

Memory requests carry an address, not a PCIe function identity as their principal destination. Later, you will see how BARs and bridge resource windows make an address reachable through a switched hierarchy.

The completion rule depends on whether the operation is a read or write:

Memory operation	Representative example	Completion required?
Memory Read	Driver reads a NIC status register with readl()	Yes. The device returns a Completion with Data.
Memory Write	Driver writes a switch-port control register with writel()	No. It is posted.
DMA Memory Read	NVMe controller reads a command from host memory	Yes. Host returns Completion(s) with Data.
DMA Memory Write	NIC writes a received packet into host memory	No. It is posted.

A Memory Read is naturally non-posted because the requester needs data back. A large read can be returned in multiple Completion-with-Data TLPs; each contains part of the requested data.

A Memory Write instead contains its data payload in the original request. PCIe can therefore treat it as posted, avoiding the additional traffic and latency of a separate response.

---

Reading a Memory Write TLP

The following annotated packet is a one-DWord Memory Write request. It illustrates the transaction-layer information that lets the fabric and destination interpret the operation.

Read the diagram from the header toward the payload:

• Fmt states the header format and indicates that this request has data.
• Type identifies the request as a Memory request.
• Length says this request carries one DWord, or four bytes, of data.
• Requester ID identifies the function that originated the request.
• Address [31:2] identifies the aligned target location. The low two address bits are represented through byte-enable information for this DWord-aligned request.
• First BE is 0xf, enabling all four bytes of the one-DWord payload.
• Data DW 0 is the value written to the addressed register or memory location.

Notice the Tag is marked unused in this example. A posted Memory Write does not need a later Completion matched back to it. In contrast, a non-posted read uses requester-identifying information and a tag so that the requester can associate a returning Completion with the outstanding request that caused it.

For software, this gives a useful mental model:

• A Memory Read is a question sent to an address, followed by an answer.
• A Memory Write is data delivered to an address, with no transaction-layer answer.
• Both can pass through one or more switch ports, but each switch routes them based on the destination information in the TLP.

---

Message requests: in-band events, not loads or stores

A Message Request communicates an event or protocol signal within PCIe. It is not an access to a BAR and does not retrieve or store ordinary data at a memory address.

PCIe messages are used for purposes such as:

• Power-management events.
• Legacy interrupt signaling.
• Error and event notification.
• Certain reset, slot, or system-management-related signals.

The practical rule is:

Messages are posted. They do not require a Completion TLP.

A Message TLP includes a message code that tells the receiver how to interpret it. Depending on the particular message, it may also carry data. But it is still an event-signaling transaction rather than a read or write of a device register.

One useful terminology trap to avoid: MSI and MSI-X interrupts are Memory Write transactions, not Message TLPs. A device raises MSI or MSI-X by issuing a posted Memory Write to an interrupt address configured by software. They behave like Memory Writes at the transaction layer even though their purpose is interrupt delivery.

For a Linux-based management service, direct Message TLP generation is much less common than configuration accesses and BAR-based MMIO. You will nevertheless encounter messages indirectly in error handling, power management, and platform logs.

---

The completion decision table

Use this compact table when reading documentation, packet traces, or driver code.

Request type	Typical purpose	Posted or non-posted?	Completion behavior
Configuration Read	Read identity, capability, or control state	Non-posted	Completion with Data
Configuration Write	Change configuration-space state	Non-posted	Completion without Data
Memory Read	Read a BAR register or DMA-read host memory	Non-posted	Completion with Data, possibly split across several TLPs
Memory Write	Write a BAR register or DMA-write host memory	Posted	No Completion
Message	Signal a PCIe event	Posted	No Completion

PCIe also retains I/O request types for compatibility with older systems. They are non-posted, but modern PCIe device management almost always uses configuration space and memory-mapped I/O instead. Treat I/O as a legacy category unless your platform documentation specifically requires it.

---

What posted writes mean for troubleshooting

The absence of a Completion is not a weakness in PCIe. It is a deliberate performance choice. Writes are common in DMA-heavy systems, and requiring a full response for every data write would consume bandwidth and increase latency.

It does change how you reason about failures.

Suppose your management application writes a control bit to a switch register through an MMIO BAR:

1. The host sends a posted Memory Write TLP.
2. Each PCIe hop provides local ACK/NAK and replay protection.
3. The switch receives the write and presents it to its register logic.
4. No Completion returns to the requester.

If the switch detects an issue processing a posted transaction, the original requester does not receive a Completion status for that write. Depending on the condition and platform support, the issue can surface through PCIe error reporting, an interrupt, a device status bit, or a later observable failure.

Therefore, a robust register-write workflow is usually:

1. Read and preserve reserved bits when modifying a shared control register.
2. Write only a documented, permitted value.
3. Verify the intended state using a documented status or readback register when it is safe and meaningful.
4. Record relevant PCIe or device error status if verification fails.

Do not assume that reading back the same control register is always valid verification. Some registers are write-only, self-clearing, read-to-clear, or have side effects. The device documentation defines the correct verification method.

---

A reliable classification method

When you encounter an unfamiliar PCIe operation, classify it in this order:

1. Is it accessing configuration space?
   It is a Configuration Read or Write, and it requires a Completion.

2. Is it reading from a memory address, including a BAR or DMA buffer?
   It is a Memory Read and requires a Completion with Data.

3. Is it writing data to a memory address, including a BAR or DMA buffer?
   It is a posted Memory Write and requires no Completion.

4. Is it signaling an event or PCIe protocol condition rather than accessing normal memory?
   It is likely a Message request, which is posted and requires no Completion.

This method deliberately starts with the semantic intent, rather than looking only at whether application code says “read” or “write.” The common source of confusion is configuration writes: they are writes, but they are still non-posted.

---

Key takeaways

• Configuration Requests access PCIe configuration space. Both reads and writes are non-posted and receive Completions.
• Memory Requests access BAR-mapped registers or ordinary memory through DMA. Memory Reads are non-posted; Memory Writes are posted.
• Message Requests carry PCIe events or signals rather than ordinary memory accesses. They are posted.
• A Completion TLP is an end-to-end transaction-layer response from the final completer. It is distinct from a link-local Data Link Layer ACK.
• A PCIe switch forwards requests and completions when it is intermediate, but it becomes the completer when software accesses the switch's own configuration space or MMIO resources.
• For posted writes, validate behavior through documented state or error reporting rather than expecting a response TLP.

Next, you will use the domain, bus, device, and function identifier to locate a specific PCIe function within the hierarchy. That is the addressing vocabulary needed to make configuration requests understandable in real Linux PCIe output.