Chapter 1: Linux Kernel Fundamentals

Updated

October 31, 2026

Chapter 1: Linux Kernel Fundamentals

Before architecting a system that runs multiple kernels, we must understand why the standard Linux kernel expects to operate as an absolute monopolist over the host’s physical resources. By examining the hardware initialization path, CPU topology discovery, and physical memory allocation algorithms, we can identify exactly which hardware structures must be partitioned, isolated, or virtualized to support concurrent kernel execution.

The Kernel as Resource Monopolist

The Linux kernel is historically a monolithic supervisor program designed to execute at the highest processor privilege level (Ring 0 on x86_64, EL1 on ARM64). In this design:

  • Single Address Space Authority: The kernel assumes complete ownership of the physical address space. Physical addresses \(0x00000000\) through the highest physical memory address are tracked by the kernel’s internal page structures (struct page and large memory folios).
  • Interrupt Routing Monopoly: The interrupt controller (such as the x86 Advanced Programmable Interrupt Controller - APIC, or ARM Generic Interrupt Controller - GIC) has its interrupt descriptor table (IDT) or vector table mapped and managed solely by the primary kernel.
  • Hardware Register Arbitrator: System management registers, MSRs (Model Specific Registers), clock generators, and PCIe configuration space accesses are assumed to have a single authoritative master.

When two uncoordinated kernels attempt to write to the same hardware registers, program the same interrupt vectors, or allocate the same physical memory pages, immediate memory corruption and unrecoverable processor state exceptions occur.

Boot and Initialization Sequence

The Linux boot process follows a deterministic sequence from processor reset vector to userspace:

[Firmware / UEFI] ──► [Bootloader (GRUB/systemd-boot)] ──► [Kernel Entry (_start)]
                                                                  │
┌─────────────────────────────────────────────────────────────────┘
▼
[setup_arch()] ──► Initialize architecture, detect memory map from UEFI
[trap_init()]  ──► Configure exception handling and interrupt vectors
[mm_init()]    ──► Set up page allocator, zones, and buddy system
[sched_init()] ──► Initialize runqueues and CFS/EEVDF scheduler
[rest_init()]  ──► Spawn kernel_init (PID 1) and kthreadd (PID 2)

At setup_arch(), the kernel parses the EFI memory map, claims all usable memory regions, and marks ACPI reserved tables. Any secondary kernel attempting to boot must receive its own clean, non-overlapping memory map.

Hardware Discovery and Enumeration

During early initialization, the kernel identifies and configures connected peripherals:

  • ACPI Table Parsing: The kernel parses the Root System Description Pointer (RSDP) to discover the Multiple APIC Description Table (MADT), Differentiated System Description Table (DSDT), and SRAT (System Resource Affinity Table).
  • PCIe Bus Scanning: The kernel reads the PCIe Extended Configuration Space (via MMCONFIG / ECAM) to enumerate all devices, functions, and bridges starting at Bus 0, Device 0, Function 0.
  • Resource Allocation: The kernel assigns Base Address Registers (BARs) and allocates memory-mapped I/O (MMIO) and I/O port windows to each discovered PCIe device.

In a multi-kernel environment, peripheral enumeration must either be strictly virtualized or physically isolated using hardware IOMMU boundaries so that each kernel discovers only its assigned hardware subset.

CPU Topology and NUMA Architecture

Modern enterprise servers feature Non-Uniform Memory Access (NUMA) topologies where physical memory is distributed across multiple memory controllers attached directly to specific CPU sockets or dies:

  • NUMA Nodes: A physical CPU socket or die and its directly attached local memory channels form a NUMA node. Accessing local memory yields minimal latency (~80ns), while accessing remote memory over the inter-socket bus (Intel UPI or AMD Infinity Fabric) incurs significant latency penalties (~140ns+).
  • Cores, Packages, and SMT: A physical processor package contains multiple physical execution cores, each often split into two Symmetric Multithreading (SMT) logical threads. SMT threads share L1/L2 caches and execution execution units.
  • Core Complexes (CCX): Modern AMD EPYC architectures group 8 cores into Core Complexes sharing an L3 cache slice.

Multi-kernel architectures leverage NUMA boundaries naturally: a dedicated kernel can be bound completely to NUMA Node 1 (cores and local RAM), while the control kernel executes exclusively on NUMA Node 0, avoiding all cross-interconnect memory traffic.

Memory Management Fundamentals

The kernel organizes physical memory into distinct allocation tiers:

  • Zones: Memory is partitioned into functional zones: ZONE_DMA (first 16MB for legacy ISA), ZONE_DMA32 (memory under 4GB), and ZONE_NORMAL (directly mapped 64-bit physical memory).
  • Buddy Allocator: The primary allocator manages memory in power-of-two page blocks (\(4\text{ KB} \times 2^n\)) up to MAX_ORDER (typically 4MB).
  • Slab/Slub Allocators: Small kernel object caches (e.g., task_struct, mm_struct) are carved out of buddy pages to prevent external fragmentation.
  • Folios: Modern Linux kernels (v6.x+) utilize large memory folios to manage compound pages efficiently across the page cache and anonymous memory.

To safely execute multiple kernels, memory isolation requires static physical carving—either via bootloader memory reservation parameters (memmap=), hypervisor-enforced Extended Page Tables (EPT), or hardware physical memory partitioning.