A Complete Guide to Virtual Memory, Allocators, Paging, and Performance in the Modern Linux Kernel
A Complete Guide to Virtual Memory, Allocators, Paging, and Performance in the Modern Linux Kernel
Memory management is the most intricate, performance-critical, and ubiquitous subsystem within the Linux kernel. Every memory read, instruction fetch, process spawn, buffer allocation, network packet transfer, and disk write directly interacts with the virtual memory (VM) and physical memory management layers.
Operating at the tight intersection between CPU hardware microarchitecture (MMUs, TLBs, cache hierarchies) and abstract user-space runtimes, the Linux memory subsystem balances competing engineering priorities: strict process isolation, near-zero translation latency, deterministic physical page allocation, fragmentation avoidance, and resilient reclaim under severe memory pressure.
This section provides an exhaustive, kernel-level curriculum covering modern Linux memory management from first principles to kernel source code walkthroughs, low-level data structures, hardware address translation, kernel allocators, user-space memory engines, page cache mechanics, NUMA placement, cgroups v2 resource limits, OOM resilience, and performance optimization.
┌────────────────────────────────────────────────────────────────────────┐
│ LINUX MEMORY MANAGEMENT TAXONOMY │
│ │
│ User-Space Runtimes & Applications │
│ ├─ glibc ptmalloc3 (Arenas/Bins), jemalloc, tcmalloc │
│ └─ Process VMA Tree: text, data, BSS, heap (brk), mmap, stack │
│ │ │
│ ▼ │
│ Kernel Virtual Memory Subsystem (mm/) │
│ ├─ Virtual Address Translation: 4-Level/5-Level Paging (PGD->P4D-> │
│ │ PUD->PMD->PTE), Transparent Huge Pages (2 MiB / 1 GiB), TLB │
│ ├─ Demand Paging & Faults: do_page_fault(), Copy-on-Write (COW) │
│ ├─ Dynamic Kernel Allocations: vmalloc, vmap, ioremap │
│ └─ Page Cache & VFS: Folios, XArrays, Readahead, Dirty Writeback │
│ │ │
│ ▼ │
│ Object & Slab Allocator Layer │
│ ├─ SLUB Allocator: Per-CPU kmem_cache_cpu (lockless), kmalloc() │
│ └─ Historical Allocators: SLAB (queues), SLOB (embedded list) │
│ │ │
│ ▼ │
│ Physical Page Allocator (Buddy Allocator) │
│ ├─ Free Lists (Orders 0 to 10: 4 KiB to 4 MiB chunks) │
│ ├─ Memory Zones: ZONE_DMA, ZONE_DMA32, ZONE_NORMAL, ZONE_MOVABLE │
│ └─ Memory Nodes: NUMA pg_data_t, Zoned Allocator, Watermarks (wmark) │
│ │ │
│ ▼ │
│ Hardware Architecture (x86-64, ARM64, RISC-V) │
│ └─ MMU, Page Table Walkers, TLB Shootdowns, Caches (L1/L2/L3), DRAM │
└────────────────────────────────────────────────────────────────────────┘Curriculum Breakdown
| Chapter | Title | Scope & Subsystems |
|---|---|---|
| Intro | Introduction | Why memory management governs Linux engineering, core mental models |
| 01 | Foundations of Memory Abstraction | Physical memory sharing problem, address space isolation, MMU role, Linux VM philosophy |
| 02 | Hardware Foundations and Address Translation | MMU, page tables, TLB shootdowns, x86-64 4/5-level paging, ARM64/RISC-V, page walk by hand |
| 03 | Linux Address Spaces and the Virtual Memory Layout | Process layout, direct-map, ASLR, mm_struct, vm_area_struct, mmap/brk/mremap mechanics |
| 04 | Physical Memory Management and the Buddy Allocator | struct page, memmap, zones, buddy algorithm, GFP flags, fragmentation, mm/page_alloc.c |
| 05 | Kernel Memory Allocators: Slab, Slub, and SLOB | SLAB vs SLUB vs SLOB, lockless per-CPU caches, kmalloc(), false sharing, custom slab module |
| 06 | User-Space Memory Allocation and vmalloc | POSIX malloc, glibc ptmalloc (arenas/bins), jemalloc/tcmalloc, vmalloc/ioremap, eBPF tracing |
| 07 | Demand Paging, Page Faults, and Copy-on-Write | Demand paging, major vs minor faults, COW mechanics, fork/exec efficiency, mm/memory.c |
| 08 | The Page Cache and Buffer Head Architecture | Page cache, XArrays, folios, readahead, dirty page writeback, buffer heads, measuring hit rates |
| 09 | Swapping, Reclaim, and Page Replacement Policies | Active/inactive LRU lists, swap-in/swap-out, zswap compressed caching, thrashing, vm.swappiness |
| 10 | Memory Compaction, Defragmentation, and Huge Pages | Compaction scanner, kcompactd, THP promotion/demotion, hugetlbfs pools, benchmarking THP |
| 11 | NUMA, Memory Policy, and Scalability | Non-Uniform Memory Access, NUMA distances, memory policies (bind, interleave), task placement |
| 12 | Memory Cgroups, Controllers, and Resource Limits | cgroups v2 memory controller, high/max limits, hierarchical reclaim, swap limits, container limits |
| 13 | Out-of-Memory Handling, Panic, and Recovery | OOM killer algorithm, badness heuristics, oom_score_adj, panic conditions, post-mortems |
| 14 | Synchronization, Concurrency, and Locking | Memory locking hierarchy, mmap_lock, PTL spinlocks, RCU page lifecycle, memory barriers |
| 15 | Performance Optimization, Profiling, and Debugging | /proc/meminfo, perf memory events, eBPF BCC tools, ftrace, crash/kdump memory dump analysis |
| 16 | Security Implications, Exploits, and Mitigations | Use-after-free, slab spraying, KASLR, SMEP/SMAP, Spectre/Meltdown, kernel hardening flags |
| 17 | Architecture-Specific Implementations and Source Walkthroughs | x86-64 vs ARM64 vs RISC-V (Sv39/48/57), IOMMU DMA mapping, kernel source navigation, custom build |
| Outro | Conclusion: Synthesis and Future Directions | Architectural synthesis, tiered memory (CXL), memory folios evolution, future Linux VM horizons |