Chapter 2: The Linux Security Model
Chapter 2: The Linux Security Model
To defend any operating system, an engineer must first understand the fundamental abstractions that define its security boundaries. Linux does not use an ad-hoc collection of security rules; its defenses are anchored in hardware privilege levels, kernel virtual memory management, process credentials, and discretionary access control primitives.
This chapter explores the core mechanisms of the Linux security architecture, detailing how the kernel isolates processes, enforces authorization, segments privileges, and presents runtime information.
Kernel Architecture and Privilege Levels
Modern CPU architectures (such as x86_64 and ARM64) provide hardware-enforced protection rings that strictly separate unprivileged application code from privileged operating system operations.
┌────────────────────────────────────────────────────────────────────────┐
│ HARDWARE RINGS & THE SYSCALL BOUNDARY │
│ │
│ Ring 3: Userspace │
│ ┌──────────────────────────────────────────────────────────────┐ │
│ │ Applications: Web servers, databases, shells, worker tasks │ │
│ │ Virtual Memory: Low canonical addresses (0x0000...) │ │
│ │ Restricted Instructions: Cannot access I/O ports or MMU maps │ │
│ └──────────────────────────────┬───────────────────────────────┘ │
│ │ syscall (SYSENTER / SVC) │
│ ▼ │
│ Ring 0: Kernel Space (Supervisor Mode) │
│ ┌──────────────────────────────────────────────────────────────┐ │
│ │ Kernel Subsystems: Memory allocator, VFS, Netfilter, Drivers │ │
│ │ Virtual Memory: High canonical addresses (0xFFFF...) │ │
│ │ Unrestricted: Direct access to physical memory & peripherals │ │
│ └──────────────────────────────────────────────────────────────┘ │
└────────────────────────────────────────────────────────────────────────┘- Ring 3 (Userspace): Ordinary user applications execute in Ring 3. In this ring, the CPU restricts direct access to physical memory addresses, I/O ports, and privileged CPU control registers (such as CR0, CR3, and CR4 on x86). Code running in Ring 3 cannot modify page tables or disable CPU interrupts.
- Ring 0 (Supervisor Mode / Kernel Space): The Linux kernel executes in Ring 0. It enjoys full, unrestricted access to the entire physical address space, CPU control structures, and hardware devices.
- Hardware Mitigations: SMEP and SMAP:
- Supervisor Mode Execution Prevention (SMEP): Prevents kernel-mode code from executing instructions located in userspace memory addresses. This blocks classic kernel exploitation techniques where an attacker overwrites a kernel function pointer to jump directly to shellcode stored in userspace.
- Supervisor Mode Access Prevention (SMAP): Prevents the kernel from reading or writing userspace memory unless explicitly permitted via controlled helper routines (such as
copy_from_user()andcopy_to_user()).
- Kernel Page Table Isolation (KPTI): Implemented to mitigate the Meltdown vulnerability (CVE-2017-5754), KPTI unmaps almost all kernel memory while running in userspace, maintaining two separate sets of page tables per process.
# Check CPU protection flags on a live Linux system
grep -E 'smep|smap|pti' /proc/cpuinfo | head -n 1Processes, User IDs, and Isolation
Every running task in Linux is represented internally by the task_struct structure in the kernel. A process possesses several User Identifiers (UIDs) and Group Identifiers (GIDs) that dictate what actions it can perform:
- Real UID (RUID) / Real GID (RGID): Identifies the actual user and group who launched the process. Used primarily for signal dispatch accounting and ownership auditing.
- Effective UID (EUID) / Effective GID (EGID): Evaluated by the kernel during permission checks when the process accesses files, binds sockets, or executes privileged syscalls.
- Saved UID (SUID) / Saved GID (SGID): Preserves the previous effective UID during privilege transitions, allowing setuid programs (e.g.,
passwd) to drop privileges temporarily and regain them later. - Filesystem UID (FSUID): Linux-specific identifier used exclusively for filesystem access checks, matching the EUID unless overridden by NFS or specialized daemons.
# Inspect the active UIDs and GIDs of your current shell
grep -E 'Uid|Gid' /proc/$$/status
# Output format: Real, Effective, Saved, Filesystem
# Uid: 1000 1000 1000 1000
# Gid: 1000 1000 1000 1000When a process executes the fork() and execve() system calls, it inherits the caller’s credentials unless the target binary possesses special permissions or flags.
File Permissions, ACLs, and Special Modes
Standard Discretionary Access Control (DAC) in Linux relies on a 12-bit mode field stored within the file’s inode:
┌────────────────────────────────────────────────────────────────────────┐
│ INODE PERMISSION BITS (12-BIT) │
│ │
│ Special (3 bits) Owner (3 bits) Group (3 bits) Others (3 bits)│
│ ┌───┬───┬───┐ ┌───┬───┬───┐ ┌───┬───┬───┐ ┌───┬───┬───┐ │
│ │SU │SG │St │ │ r │ w │ x │ │ r │ w │ x │ │ r │ w │ x │ │
│ └───┴───┴───┘ └───┴───┴───┘ └───┴───┴───┘ └───┴───┴───┘ │
│ 4 2 1 4 2 1 4 2 1 4 2 1 │
└────────────────────────────────────────────────────────────────────────┘The Standard Permissions
- Read (
r, 4): View file contents or list directory entries. - Write (
w, 2): Modify file contents or create/delete files inside a directory. - Execute (
x, 1): Execute a binary or traverse (cd) into a directory.
The Special Modes
- Setuid (SUID, octal 4000): When set on an executable binary, the operating system executes the process with the permissions of the file’s owner rather than the calling user. Classic example:
/usr/bin/passwdis owned byroot, allowing unprivileged users to update their password hashes in/etc/shadow. SUID binaries represent a primary vector for local privilege escalation. - Setgid (SGID, octal 2000): On binaries, executes the process with the file’s group permissions. On directories, forces any newly created files within the directory to inherit the directory’s group rather than the creating user’s primary group.
- Sticky Bit (octal 1000): Applied to shared world-writable directories (such as
/tmp). Restricts file deletion or renaming: a user can only delete or rename files they personally own, preventing malicious users from clearing files created by other tenants.
# Audit all SUID and SGID binaries on the host
find / -xdev \( -perm -4000 -o -perm -2000 \) -type f -exec ls -la {} + 2>/dev/nullCapabilities and Privilege Separation
Historically, the Unix permission model suffered from the “all-or-nothing” root problem: if a program needed to bind to a low-numbered port (< 1024), it had to run with full superuser privileges, granting it the power to format disks, read any file, and load kernel modules.
Linux Capabilities dismantle the monolithic authority of UID 0 into approximately 40 distinct privileges:
| Capability | Scope of Authority |
|---|---|
CAP_NET_BIND_SERVICE |
Bind a socket to privileged ports (< 1024) |
CAP_NET_ADMIN |
Configure interfaces, routing tables, and firewall rules |
CAP_SYS_ADMIN |
Overly broad privilege: mount filesystems, configure IPC, handle namespaces |
CAP_SYS_CHROOT |
Invoke chroot() to alter process root |
CAP_SYS_PTRACE |
Trace arbitrary processes via ptrace (dangerous) |
CAP_DAC_OVERRIDE |
Bypass file read, write, and execute permission checks |
Capabilities exist in five distinct sets within a process: 1. Permitted: The absolute maximum set of capabilities the process may assume. 2. Effective: The capabilities currently active and evaluated by the kernel for operations. 3. Inheritable: Capabilities preserved across an execve() call. 4. Bounding: A system-wide or per-process ceiling that restricts what capabilities can ever be gained. 5. Ambient: Capabilities that survive execve() without requiring file capabilities on the target binary.
# Inspect capabilities of the current process
capsh --print
# Set capability on a binary to avoid setuid root (e.g., binding port 80/443)
sudo setcap 'cap_net_bind_service=+ep' /opt/my-app/bin/server
# Verify file capabilities
getcap /opt/my-app/bin/serverNamespaces and Cgroups: The Foundation of Isolation
Linux does not contain a single kernel object called a “container”. Containers are built entirely from two foundational kernel primitives: Namespaces and Control Groups (cgroups).
┌────────────────────────────────────────────────────────────────────────┐
│ CONTAINER PRIMITIVES │
│ │
│ Namespaces: What a process can SEE │
│ ├─ Mount (mnt): Isolated mount points and filesystems │
│ ├─ Process ID (pid): Independent PID numbering (PID 1 in container) │
│ ├─ Network (net): Dedicated interfaces, IP routing, iptables tables │
│ ├─ Inter-Process Comm (ipc): Isolated System V IPC and POSIX queues │
│ ├─ UTS: Independent hostname and domain name │
│ ├─ User (user): Map container UID 0 to an unprivileged host UID │
│ └─ Cgroup: Virtualized view of cgroup hierarchies │
│ │
│ Control Groups (cgroups v2): What a process can USE │
│ ├─ cpu: Bandwidth limits, CFS quotas, thread weight │
│ ├─ memory: Max memory limit, swap limits, OOM killer triggers │
│ ├─ io: Read/write IOPS and byte limits per block device │
│ └─ pids: Max process count to prevent fork bombs │
└────────────────────────────────────────────────────────────────────────┘User namespaces (CLONE_NEWUSER) are particularly significant for security: they allow a process to have root privileges (UID 0) inside its private namespace while remaining an unprivileged user (e.g., UID 100000) on the underlying host. However, exposed unprivileged user namespaces have also been a frequent source of kernel privilege escalation vulnerabilities when interacting with buggy filesystem or network drivers.
System Calls, Procfs, Sysfs, and the Attack Surface
The system call interface is the boundary between untrusted userspace applications and the privileged kernel:
Userspace (glibc / musl) ──► syscall instruction (e.g., #0x80, SYSENTER) ──► Kernel Syscall Dispatch TableLinux provides over 300 unique system calls. Vulnerabilities arise when attackers pass malformed parameters, induce race conditions (TOCTOU), or exploit logic errors inside complex syscalls like io_uring_enter(), bpf(), or perf_event_open().
The Pseudo-Filesystem Attack Surface (/proc and /sys)
The virtual filesystems /proc and /sys expose kernel data structures, process memories, and device drivers to userspace:
/proc/[pid]/mem: Direct interface to process virtual memory. Historically abused to inject code into running processes./proc/kallsyms: Kernel symbol table exposing internal kernel memory addresses. Essential for attackers to bypass KASLR unless restricted./sys/class/&/sys/kernel/debug/: Hardware controls and debug interfaces. If writable, attackers can manipulate device registers or trigger debug faults.
# Harden kernel pointer leaks in /proc/kallsyms
sudo sysctl -w kernel.kptr_restrict=2
# Restrict dmesg kernel log access to processes with CAP_SYSLOG
sudo sysctl -w kernel.dmesg_restrict=1