Create your own
Lesson illustration

How Modern Binary Protections Constrain Exploit Strategies

Good to see you again. Last lesson established a runtime fact that drives almost every modern Linux pwn challenge: ASLR changes bases between fresh processes, while offsets inside a loaded image remain stable. You located the executable, libc, stack, heap, and loader in process maps, and saw why GDB’s default ASLR setting can give misleadingly stable addresses.

Now we turn those observations into an exploit-planning method. By the end of this lesson, you should be able to look at checksec output plus a confirmed bug primitive and state precisely which assumptions are invalid, which targets remain plausible, and what additional evidence—usually a leak or a different target—you need.


Mitigations are constraints, not verdicts

A useful first correction to a common CTF instinct: a binary with every green mitigation in checksec is not “unexploitable.” It is a binary that refuses several simple assumptions at once.

Each mitigation protects a different link in a typical control-flow attack:

MitigationIt removes this assumptionIt does not prove
Stack canary“I can overwrite a saved return address through this stack buffer without being noticed.”There is no useful stack corruption.
NX“I can execute bytes I injected into the stack or heap.”I cannot execute existing code.
ASLR“An address learned in an earlier run will work now.”Relative offsets within a module changed.
PIE“Main-binary gadgets and global-data addresses are fixed.”The program contains no usable gadgets or data.
RELRO“I can redirect an imported function by overwriting its GOT entry.”There are no other writable control-flow or data targets.

The central planning question is therefore not “How do I bypass all mitigations?” Instead, ask:

Given my primitive, what does my chosen exploit route require, and which of those requirements are presently missing?

For example, a conventional stack ROP chain requires all of the following:

  1. A way to reach and change the saved return address.
  2. If a canary lies between the vulnerable buffer and that return address, the correct canary value.
  3. Correct runtime addresses for every gadget, function, and string pointer used.
  4. Executable code already mapped in the process, because NX prevents executing the chain itself as newly injected code.

Notice that NX is not the reason a ROP chain fails: ROP intentionally executes instructions in existing executable mappings. Canary and address uncertainty are separate constraints.

Watch this short CTF walkthrough segment before going deeper. It gives a compact example of a binary where canaries and NX are enabled but PIE is absent.

Bypassing Stack Canaries and NX/DEP (Ret2Lib-C) - Bird - [Intigriti 1337UP LIVE CTF 2022]

In “Bypassing Stack Canaries and NX/DEP (Ret2Lib-C),” CryptoCat begins with exactly the kind of file and checksec triage you should now be doing. Watch it as an example of turning mitigation output into an initial list of constraints, rather than treating it as a ready-made exploit recipe.

Watch initial triage for the protection profile, then the canary overview. Later, watch the NX constraint, followed by GOT and ASLR. Focus on what each observation changes: direct stack shellcode is ruled out by NX, while a dynamically linked libc address becomes valuable because ASLR changes libc's base.


Stack canaries: a guard between local data and the return path

A stack canary is a value placed by compiler-generated code into a protected function’s stack frame. Before the function returns, the compiler-generated epilogue compares that saved value with the thread’s reference guard. A mismatch calls __stack_chk_fail, terminating the program before the corrupted return address can be used.

The supplied Stack canary assembly image shows the standard x86-64 pattern:

x86-64 disassembly of a protected function: it loads the guard from `fs:[0x28]`, stores it at `[rbp-0x8]`, and compares it before `leave; ret`; a mismatch branches to `__stack_chk_fail`.

Read it from top to bottom:

  1. The prologue allocates the stack frame with sub rsp, 0x30.
  2. mov rax, qword fs:[0x28] loads the reference canary on a common Linux x86-64 setup.
  3. mov qword [rbp-0x8], rax saves a copy in this function’s stack frame.
  4. The vulnerable input is placed in the local buffer at [rbp-0x30].
  5. Near the end, the program loads the saved copy and compares it with the current reference value.
  6. Equal values reach leave and ret; a mismatch reaches __stack_chk_fail.

The diagram often used for this layout is conceptually correct:

Higher-address direction in a typical frameRole
Saved return addressValue consumed by ret
Saved frame pointerPrevious frame metadata
CanaryGuard checked before return
Local bufferOften the overflow origin

However, treat this as a model, not a byte-perfect promise. Compiler options, optimization level, local variables, and frame-pointer omission can change offsets. In a challenge, confirm the actual distance with disassembly, GDB, and a cyclic pattern where appropriate.

What a canary blocks

Suppose an input overflows upward from a local buffer toward the saved return address. It must overwrite the canary first. If your payload writes arbitrary padding in its place, the function aborts:

*** stack smashing detected ***: terminated

That crash means something useful: your write likely reached the guard, but you have not demonstrated instruction-pointer control. The program stops before ret.

A canary changes a stack-overflow plan in one of three broad ways:

  • Preserve it. If the intended corruption is limited to locals before the canary, you may not need to cross it at all. Corrupting an authentication flag, size field, pointer, or function-local state can still matter.
  • Learn it during the same process. A separate disclosure bug may reveal the guard; then an overflow payload can place the exact eight bytes back in their original position before continuing toward saved control data.
  • Choose a different primitive. Heap corruption, a global function pointer, an arbitrary write, or another non-stack target is not automatically stopped by a stack canary.

A canary leak is not interchangeable with an address leak. It solves only the “survive the protected function epilogue” requirement. You may still need a PIE or libc address leak to use ROP reliably.

Also, do not overread checksec:

  • Canary found is a strong indication that the binary contains stack-protector support, commonly through __stack_chk_fail.
  • Stack-protector instrumentation is selected by compiler heuristics unless the binary was built with an option that protects every function.
  • Therefore, inspect the vulnerable function. The fs:[0x28] load and final comparison are better evidence than a global summary line.

NX: injected bytes are data, not instructions

NX, also called DEP, marks memory pages such as the stack and heap as non-executable. In last lesson’s vmmap terms, it turns the usual stack mapping into rw- rather than rwx.

If an exploit places shellcode in a stack buffer and redirects RIP to that buffer, the CPU attempts instruction fetch from a non-executable page. The typical result is SIGSEGV. This is an execution-permission failure, not a failure to write the shellcode.

NX therefore divides payloads into two categories:

Payload styleDoes NX permit it?Main remaining requirement
Inject shellcode into stack or heap, then jump to itUsually noWould require an executable writable page
Return to an existing win functionYesCorrect function address and control of RIP
ROP chain using .text, libc, or another executable mappingYesCorrect gadget addresses and ABI-correct setup
Corrupt a data-only valueYes, because no injected code executesA useful writable target and a reachable write

A ROP chain can reside in non-executable stack memory because the stack holds addresses and values, not instructions to be fetched. Each ret takes the next address from the stack; that address must point into an existing executable mapping such as the main binary or libc.

This is why “NX enabled” usually changes the goal from inject code to reuse code. It does not itself reveal any addresses and does not guarantee that a particular ROP chain will work.

The following reading is a concise, technically solid reference for the four protections. Read it now, concentrating on the distinctions between memory permissions, randomized locations, and writable relocation entries.

Compiler Options Hardening Guide for C and C++

The OpenSSF Compiler Options Hardening Guide explains the compiler and loader mechanisms behind canaries, NX, RELRO, and PIE. It is useful here because it separates what each defense actually protects from what is merely a common exploitation consequence.

Read the subsection “Enable run-time checks for stack-based buffer overflows” and its “Synopsis.” Follow the canary mechanism, paying attention to the prologue, epilogue, and call to __stack_chk_fail. Next, in “Enable data execution prevention,” read the “Synopsis” through the linker explanation. Begin at the NX explanation. Relate it to the rw- stack mapping you saw in the previous lesson. Then read “Mark relocation table entries resolved at load-time as read-only,” especially the paragraphs defining partial and full RELRO. Read the RELRO discussion, then continue through the paragraph that contrasts the writable .got.plt in partial RELRO with eager resolution in full RELRO. Finally, read “Build as position-independent code,” focusing on why PIE is necessary for the main executable to benefit from ASLR. In the synopsis, follow the PIE and ASLR connection. Do not focus on compiler performance details for this lesson.


ASLR and PIE: address knowledge becomes a runtime requirement

You already verified the operational distinction:

  • ASLR randomizes the load locations of regions such as the stack, heap, libc, loader, and normally other shared libraries.
  • PIE makes the main executable position-independent, allowing ASLR to randomize the main executable too.

PIE does not independently “randomize the whole program.” Rather, it makes the executable capable of being loaded at a different base; ASLR supplies that different base at runtime.

For any module whose base is randomized, the working rule is:

If a leak identifies a known symbol in that module, the reverse calculation is:

Then another target within the same module can be calculated:

This arithmetic is straightforward. The difficult part is correctly classifying the leak:

Leaked valueWhat it may revealWhat you must know
A pointer into libclibc base for this processThe matching libc file and the leaked symbol or reliable offset
A pointer into a PIE executablePIE base for this processWhich main-image object or code location it references
A stack pointerA stack location for this processWhether the target is at a predictable relative position
A canaryThe guard needed for a protected returnNothing about any module base by itself

A main executable with No PIE commonly retains stable code and data addresses even when ASLR is active. Thus, a non-PIE binary may still offer fixed ret2win or main-image ROP gadget addresses. But libc, the stack, heap, and loader remain randomized, so a chain relying on libc still needs a runtime libc base.

With PIE enabled, a static Ghidra address such as 0x1234 is normally an offset-like value within the executable image, not an address you can paste directly into a payload. You need the current PIE base first.

One practical consequence is worth memorizing:

A leak needs to identify the same module as the target address you want to calculate.

Leaking a libc pointer derives libc’s base, not the PIE base. Leaking a PIE code pointer derives the main executable base, not libc’s. A strong exploit plan lists these bases separately instead of treating “an address leak” as a universal solution.


RELRO: classify GOT overwrite ideas before pursuing them

RELRO means relocation read-only. It is closely connected to dynamically linked functions and the Global Offset Table (GOT).

A dynamically linked binary often calls an imported function through a Procedure Linkage Table (PLT) stub. The PLT obtains the function’s current runtime address from the GOT. Since libc is mapped at an ASLR-selected base, entries in the GOT eventually contain runtime pointers that are meaningful for the current process.

That produces two distinct attacker interests:

  • Read a GOT entry to disclose a resolved libc function address.
  • Write a GOT entry to redirect a future imported-function call.

RELRO mostly targets the second interest.

checksec RELRO resultWhat it means for GOT overwrite ideas
No RELRORelocation-related areas may remain writable; GOT overwrite targets are worth investigating.
Partial RELROSome relocation data is read-only, but .got.plt remains writable to support lazy binding. Imported-function GOT overwrite may still be plausible.
Full RELROThe dynamic linker resolves relevant entries at startup, then makes the PLT GOT read-only. A normal write to an imported function’s GOT entry is not a viable redirect target.

Two important boundaries keep this precise:

  1. Full RELRO does not make every writable section read-only. It specifically protects relocation-related regions. Global application data, heap objects, stack data, and custom function pointers may still be writable if your primitive reaches them.
  2. Full RELRO does not prevent reading GOT entries. If another vulnerability lets you disclose memory and you can identify a GOT entry, it may still be useful as a libc pointer leak. Full RELRO removes the redirection route, not the information value.

A partial-RELRO GOT overwrite, if you ever consider one, also does not make ASLR disappear. You still need the correct runtime address to write as the replacement pointer, often requiring a matching address leak.


Build a mitigation-aware plan, not a mitigation checklist

Use this workflow whenever you have a new ELF and an initial bug hypothesis.

1. State the primitive in plain language

Avoid vague claims such as “buffer overflow.” Record the strongest fact you have actually proven:

  • “I can overwrite 96 bytes of a stack buffer before the program returns.”
  • “I can write an attacker-selected 8-byte value to an attacker-selected writable address.”
  • “I can disclose words from the stack.”
  • “I can change only a local integer, not a return address.”
  • “The crash places cyclic bytes in saved RIP.”

The primitive, not the function name, determines which mitigations are relevant.

2. Record the protection profile

For a local challenge, begin with:

checksec --file=./chall

Then translate its output into claims such as:

Canary:    target function may require a valid guard before return
NX:        stack shellcode is not a default route
PIE:       main-image absolute addresses need a runtime base
RELRO:     GOT overwrite may be unavailable

Treat the word “may” seriously for stack canaries until you inspect the vulnerable function. Likewise, verify PIE and ASLR behavior with runtime maps rather than assuming a debugger has left randomization on.

3. Make the missing requirement explicit

Here are three common profiles and the correct first conclusion.

Confirmed situationImmediate conclusionMissing requirement or next target
Non-PIE, no canary, NX enabled, stack RIP controlStack shellcode is blocked, but fixed main-image code addresses may support a code-reuse or direct-function route.Find a useful fixed function, gadget, or writable data target.
PIE, canary, NX, full RELRO, linear stack overflow but no leakA direct overwrite of the return address will abort at the canary; even a preserved canary would not give known PIE or libc addresses. GOT overwrite is excluded.Seek an information disclosure, a pre-canary local target, or an entirely different bug class.
Partial RELRO plus a reliable arbitrary writeA writable PLT GOT slot can be a candidate control-flow target, but only if a useful imported function is called after the overwrite.Verify the exact slot is writable and derive the runtime replacement address despite ASLR.

This is the key discipline: do not call a mitigation “bypassed” until you can state which required condition was satisfied.

For instance, in a hypothetical PIE, NX, canary, full-RELRO challenge, a future disclosure of a stack canary plus a PIE code pointer would establish only two things:

  • the overflow can preserve the guard;
  • main-executable addresses can be calculated for that same process.

It does not automatically reveal libc’s base. If the final code-reuse goal needs libc, you still need a libc leak, or you need to construct a route using only known main-image capabilities. This kind of accounting prevents many hours of building payloads with one unknown address hidden inside them.

A short hands-on triage routine

Choose one authorized local CTF binary you have already solved or are currently analyzing. Spend about ten minutes making a small text file named plan.md containing:

Primitive proved:
Canary evidence in vulnerable function:
NX consequence:
PIE and ASLR consequence:
RELRO consequence:
Known runtime addresses or leaks:
Needed next fact:
Rejected route and reason:

Do not try to solve the challenge during this activity. The goal is to turn checksec into a falsifiable plan. If you write “need a leak,” specify whether it must be a canary, PIE, libc, heap, or stack leak, and why.


Key takeaways

Mitigations remove assumptions rather than eliminating every exploit path:

  • Stack canaries detect a stack overwrite that crosses the guard before a protected function returns. They do not protect unrelated memory targets or corruption that remains before the guard.
  • NX blocks executing injected stack or heap bytes, pushing control-flow attacks toward existing executable code such as functions and ROP gadgets.
  • ASLR makes absolute addresses process-specific while preserving offsets inside a loaded module.
  • PIE allows ASLR to randomize the main executable, so main-image gadgets and globals need a PIE base at runtime.
  • RELRO determines whether GOT overwrite ideas are viable; full RELRO blocks normal writes to PLT GOT entries but does not make those entries unreadable.

Most importantly, keep separate ledgers for control, canary knowledge, and each required module base. A canary leak, PIE leak, and libc leak solve different problems.

Next lesson shifts from protections back to the program itself: you will trace attacker-controlled input from an entry point to the actual memory-corruption site using Ghidra and GDB/GEF.

Can't find a good explanation? Sign up and we'll make it for you

Sign up