Binary Hardening on Linux: What Canaries, NX, ASLR, RELRO, FORTIFY and CET Actually Stop

Most of what runs on a Linux machine is still C and C++, and a memory-safety bug is still the most common way from "sends input" to "runs code". The mitigations in this post do not fix those bugs. Each removes one step from the classic path between a corrupted byte and a hijacked instruction pointer and leaves the others open, which is why they are layered rather than chosen. Knowing what each layer stops is what lets you read a checksec report as a threat model rather than a row of green words.

One program, two builds

The sample is the smallest program that has a real vulnerability: a fixed buffer, an unbounded strcpy, and a printf of the result.

#include <stdio.h>
#include <string.h>

void greet(const char *name) {
    char buf[64];
    strcpy(buf, name);              /* no bounds check: the classic bug */
    printf("hello, %s\n", buf);
}

int main(int argc, char **argv) {
    if (argc > 1)
        greet(argv[1]);
    return 0;
}

It was built twice, once with every protection deliberately switched off and once with the flags a hardened distribution uses:

# weakened on purpose: no canary, fixed load address, writable GOT, lazy binding, no FORTIFY
gcc -O2 -fno-stack-protector -no-pie -U_FORTIFY_SOURCE \
    -Wl,-z,norelro -Wl,-z,lazy -o greet-weak greet.c

# hardened
gcc -O2 -fstack-protector-strong -fPIE -pie -Wl,-z,relro,-z,now \
    -D_FORTIFY_SOURCE=2 -fstack-clash-protection -fcf-protection=full \
    -o greet-hardened greet.c

One note on provenance: the listings were produced on a Mac with a clang-based cross toolchain targeting x86-64 glibc (zig cc -target x86_64-linux-gnu). The ELF headers and dynamic sections match what gcc and GNU ld produce; the one place the generated code differs is called out below.

Stack canaries

A canary is a random machine word placed between a function's local buffers and its saved frame pointer and return address. The function prologue copies it from thread-local storage (fs:0x28 on x86-64 glibc, filled at process start from the kernel's AT_RANDOM bytes, with the lowest byte zeroed so that string functions stop at it) and the epilogue checks it before ret. If an overflow ran through the buffer into the return address, it ran through the canary first.

$ objdump -d --no-show-raw-insn -M intel greet-hardened | sed -n '/<greet>:/,/^$/p'
00000000000015b0 <greet>:
    15b0:  endbr64
    15b4:  push   rbp
    15b5:  mov    rbp,rsp
    15b8:  push   rbx
    15b9:  sub    rsp,0x48
    15bd:  mov    rsi,rdi
    15c0:  mov    rax,QWORD PTR fs:0x28          ; load the canary
    15c9:  mov    QWORD PTR [rbp-0x10],rax       ; park it above the locals
    15cd:  lea    rbx,[rbp-0x50]
    15d1:  mov    edx,0x40                       ; sizeof buf, from FORTIFY
    15d6:  mov    rdi,rbx
    15d9:  call   17c0 <__strcpy_chk@plt>
    15de:  mov    rdx,rbx
    15e1:  xor    eax,eax
    15e3:  call   1610 <printf>                  ; fortified wrapper, see below
    15e8:  mov    rax,QWORD PTR fs:0x28
    15f1:  cmp    rax,QWORD PTR [rbp-0x10]       ; still the same word?
    15f5:  jne    15fe <greet+0x4e>
    15f7:  add    rsp,0x48
    15fb:  pop    rbx
    15fc:  pop    rbp
    15fd:  ret
    15fe:  call   17d0 <__stack_chk_fail@plt>    ; abort, never returns

The weakened build has none of this: greet calls strcpy, calls printf, and returns through whatever is on the stack. GCC emits the same check with a sub/je pair instead of cmp/jne; the behaviour is identical.

-fstack-protector only instruments functions with character arrays of at least eight bytes; -fstack-protector-strong extends that to any local array and any function that takes the address of a local, which is the distribution default; -fstack-protector-all instruments everything at measurable cost. What it stops: a sequential overflow from a stack buffer into the return address. What it does not stop: overwriting other locals or function pointers that sit between the buffer and the canary, bugs on the heap, an arbitrary write that skips over the canary, and a format-string or out-of-bounds read that leaks the canary value first. In a forking server every child inherits the same canary, so it can be guessed one byte at a time. The canary is a tripwire, not a wall.

NX: data is not code

Every mapping in a process carries read, write and execute permissions, and since the arrival of the NX bit on x86-64 the kernel enforces them in hardware. The linker records the intended stack permission in a PT_GNU_STACK program header, and the kernel maps the stack, heap and all anonymous memory without execute permission unless that header says otherwise.

$ readelf -lW greet-hardened | grep GNU_STACK
  GNU_STACK      0x000000 0x0000000000000000 0x0000000000000000 0x000000 0x1000000 RW  0

RW is what you want; RWE means an input object lacked the .note.GNU-stack marker (hand-written assembly is the usual culprit) and the linker fell back to an executable stack for the whole program. Both builds show RW because toolchains default to it. What it stops: the original shellcode-on-the-stack attack, and injecting code into any data region. What it does not stop: reusing code that is already executable. Return-to-libc and return-oriented programming exist precisely because NX made injected code useless, and a program that calls mprotect on attacker-controlled memory hands the protection back.

ASLR and PIE

Code reuse needs addresses. ASLR makes the kernel place the stack, the heap and every mmap-ed library at a fresh random offset in each process; on x86-64 the mmap base alone carries 28 bits of entropy by default (vm.mmap_rnd_bits, raisable to 32), and kernel.randomize_va_space = 2 randomises the brk heap too. The one region it cannot move is a classic EXEC binary, which is linked for a fixed address. A position-independent executable is linked as a shared object (DYN) and relocated wherever the loader puts it.

$ readelf -h greet-weak | grep Type
  Type:                              EXEC (Executable file)
$ readelf -h greet-hardened | grep Type
  Type:                              DYN (Position-Independent Executable file)
$ readelf -dW greet-hardened | grep FLAGS
 0x000000000000001e (FLAGS)              BIND_NOW
 0x000000006ffffffb (FLAGS_1)            Flags: NOW PIE

Every address in the weak binary's disassembly starts with 0x1000000, and it will be there in every run; the hardened one is linked at offset zero and lands somewhere different each time. What it stops: hard-coding the address of system, of a ROP gadget, or of the GOT. What it does not stop: information leaks. Within a module the layout is fixed, so one leaked pointer (a stack address printed by a format string, an uninitialised field, a partial overwrite of the low bytes) reveals the base of the whole module. On 32-bit, or in a forking server that keeps the same layout across children, the entropy can simply be brute-forced.

RELRO: a read-only GOT

Calls into shared libraries go through the Global Offset Table, an array of function pointers that the dynamic loader fills in. With lazy binding the entries are resolved on first call, so the table has to stay writable for the lifetime of the process, and an attacker with a single arbitrary write can replace printf's slot with system and wait for the next call. Partial RELRO (-z relro, the default) makes .init_array, .fini_array, .dynamic and the non-PLT part of the GOT read-only after relocation but leaves .got.plt writable. Full RELRO adds -z now: resolve every symbol at startup, then mprotect the entire table.

$ readelf -lW greet-hardened | grep -E "GNU_RELRO"
  GNU_RELRO      0x0007f0 0x00000000000027f0 0x00000000000027f0 0x000188 0x000810 R   0x1
$ readelf -SW greet-hardened | grep -E "\.got"
  [17] .got              PROGBITS        0000000000002940 000940 000008 00  WA  0   0  8
  [18] .got.plt          PROGBITS        0000000000002948 000948 000030 00  WA  0   0  8
$ readelf -lW greet-weak | grep -c GNU_RELRO
0

The GNU_RELRO segment runs from 0x27f0 for 0x810 bytes, to 0x3000, which covers both .got and .got.plt: after ld.so finishes, there is no writable function pointer left in the loader's tables. What it stops: GOT overwrites, the most convenient target for a write-what-where primitive. What it does not stop: every other writable pointer in the process, including C++ vtables, callback fields in heap objects, and .data tables the program itself keeps. It also slows startup for large programs, which is why some distributions enable it selectively.

FORTIFY_SOURCE

FORTIFY is a glibc feature enabled by -D_FORTIFY_SOURCE=n at -O1 or above. Where the compiler can prove a destination's size with __builtin_object_size, calls to strcpy, memcpy, sprintf, read and dozens of other functions are rewritten to _chk variants that take the size as an extra argument and abort on overflow. Level 2 adds checks that can change the behaviour of odd but conforming programs, such as refusing %n in a writable format string. Level 3 (GCC 12 or clang, with glibc 2.34 or later) uses __builtin_dynamic_object_size to cover sizes only known at run time, including malloc(n).

$ readelf --dyn-syms -W greet-hardened | grep _chk
     2: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND __strcpy_chk@GLIBC_2.3.4 (3)
     3: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND __stack_chk_fail@GLIBC_2.4 (4)
     4: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND __vprintf_chk@GLIBC_2.3.4 (3)
$ readelf --dyn-syms -W greet-weak | grep -c _chk
0

In the disassembly earlier the mov edx,0x40 before __strcpy_chk is the 64-byte size of buf; a longer argument aborts with *** buffer overflow detected *** before the canary is even consulted. The printf call became an inlined wrapper around __vprintf_chk because that is how clang expands glibc's fortified header; GCC emits a direct __printf_chk instead. What it stops: overflows through the standard library when the destination size is statically or dynamically knowable, and the %n write primitive in format-string bugs. What it does not stop: overflows done with pointer arithmetic and loops, library calls where the size is unknowable, and anything outside libc.

Control-flow integrity and shadow stacks

Everything so far leaves one gap: if an attacker obtains a corrupted return address or function pointer without tripping a canary, nothing checks where control goes. Control-flow integrity closes it in two halves. On the forward edge, Intel's Indirect Branch Tracking (IBT) requires every indirect branch to land on an endbr64 instruction, which is why greet above begins with one; -fcf-protection=branch emits the landing pads, and clang's -fsanitize=cfi goes further by checking at link time, with LTO, that each indirect call targets a function of the matching type. On the backward edge, a shadow stack (Intel SHSTK, -fcf-protection=return) makes the CPU push every return address to a second, hardware-protected stack and fault on ret if the two copies disagree. Linux gained user-space shadow stacks in 6.6, and glibc enables them when the hardware, the kernel and every loaded object agree.

That last condition is where deployments fail, and this build is an example. The compiler marked each object with a .note.gnu.property saying "IBT and SHSTK safe", but the linker ANDs those bits across all inputs, and the C runtime objects of this cross toolchain were not built with -fcf-protection. Result: no property note in the final binary, and no CET even on a capable CPU.

$ readelf -nW greet-hardened | grep -A1 "x86 feature"
$ # nothing: one unmarked input object silently disables CET for the whole program

On a distribution whose toolchain and glibc were built with CET, the same command prints Properties: x86 feature: IBT, SHSTK; GNU ld's -z cet-report=error turns the silent downgrade into a link failure. On arm64 the equivalents are pointer authentication and BTI, enabled together with -mbranch-protection=standard. What it stops: return-oriented programming (shadow stack) and jumping into the middle of functions (IBT). What it does not stop: calling a legitimate, correctly typed function with attacker-chosen arguments, overwriting data that decides whether a privileged path runs, and any bug whose payoff needs no control-flow hijack at all. Enforcement also needs a CPU that has it (Intel Tiger Lake and AMD Zen 3 onward) plus a kernel and libc that enable it.

Flags per mitigation

TIP

Do not assume the defaults; ask the compiler. gcc -v 2>&1 | grep -o -- '--enable-default-[a-z]*' shows whether PIE and the stack protector are on by default, and echo | gcc -O2 -dM -E - | grep FORTIFY whether the distribution injects _FORTIFY_SOURCE. Then verify the artefact rather than the flags with checksec --file=./binary or Debian's hardening-check, confirm that cat /proc/sys/kernel/randomize_va_space prints 2, and look for user_shstk in /proc/cpuinfo to learn whether a shadow stack is even possible on the machine.

$ checksec --file=greet-weak
    Arch:       amd64-64-little
    RELRO:      No RELRO
    Stack:      No canary found
    NX:         NX enabled
    PIE:        No PIE (0x1000000)
    Stripped:   No
    Debuginfo:  Yes

$ checksec --file=greet-hardened
    Arch:       amd64-64-little
    RELRO:      Full RELRO
    Stack:      Canary found
    NX:         NX enabled
    PIE:        PIE enabled
    Stripped:   No
    Debuginfo:  Yes

$ readelf -lW greet-hardened
Program Headers:
  Type           Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
  PHDR           0x000040 0x0000000000000040 0x0000000000000040 0x000230 0x000230 R   0x8
  INTERP         0x000270 0x0000000000000270 0x0000000000000270 0x00001c 0x00001c R   0x1
      [Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
  LOAD           0x000000 0x0000000000000000 0x0000000000000000 0x000574 0x000574 R   0x1000
  LOAD           0x000580 0x0000000000001580 0x0000000000001580 0x000270 0x000270 R E 0x1000
  LOAD           0x0007f0 0x00000000000027f0 0x00000000000027f0 0x000188 0x000810 RW  0x1000
  DYNAMIC        0x0007f0 0x00000000000027f0 0x00000000000027f0 0x000150 0x000150 RW  0x8
  GNU_RELRO      0x0007f0 0x00000000000027f0 0x00000000000027f0 0x000188 0x000810 R   0x1
  GNU_EH_FRAME   0x00049c 0x000000000000049c 0x000000000000049c 0x00002c 0x00002c R   0x4
  GNU_STACK      0x000000 0x0000000000000000 0x0000000000000000 0x000000 0x1000000 RW  0
  NOTE           0x00028c 0x000000000000028c 0x000000000000028c 0x000020 0x000020 R   0x4

WARNING

Mitigations raise the cost of exploitation; they do not lower the number of bugs. A program with every row green still has the strcpy in it, and a good information leak plus an arbitrary write walks through most of these layers in a real target. Treat the table as the floor for anything that parses untrusted input, keep the sanitizers (-fsanitize=address,undefined) in test builds, and write the next parser in a memory-safe language if you can.

Further reading

• GCC, Program Instrumentation Options, the reference for -fstack-protector-*, -fstack-clash-protection and -fcf-protection, and Code Generation Options for -fPIE.
• GNU binutils, ld command-line options, including every -z keyword used above.
• The GNU C Library manual, Fortification of function calls, and Red Hat, How to improve application security using FORTIFY_SOURCE=3.
• Linux kernel, sysctl documentation for randomize_va_space and Control-flow Enforcement Technology (CET) Shadow Stack.
• Intel, A Technical Look at Intel's Control-flow Enforcement Technology, and Clang, Control Flow Integrity.
• OpenSSF, Compiler Options Hardening Guide for C and C++, the current recommended flag set with rationale per option.
• PaX Team, Address Space Layout Randomization, the 2001 design document that the kernel implementation follows, and the Debian wiki's Hardening page on which flags a distribution turns on and why.
• checksec, the shell script; the output above is from the pwntools reimplementation, which prints the same columns.