mmap vs pread in Go Storage

Updated

September 8, 2026

mmap vs pread in Go Storage

Overview

Storage engines eventually ask: how should we read bytes from files? In Go the answer is rarely absolute—“always mmap” or “always pread”. The interesting design is an abstraction that prefers mmap when pages are warm and falls back when a fault would block an M.

Guest-systems depth: mmap vs pread in a real Go storage engine (Phuong Le / Internals for Interns). Patterns below are teaching synthesis.

Diagram: two I/O styles

  mmap path:  map pages → load byte
                 │ page fault?
                 └── may block M on major fault

  pread path: syscall ReadAt → copy into []byte
                 └── G park rules better understood

Tradeoffs

	mmap	pread
API	pointer-like access	explicit `ReadAt`
Warm data	very fast	syscall + copy
Cold data	major fault risk	controlled syscall
Go runtime	fault on M is painful	netpoll/syscall paths understood
Portability	OS-specific edges	straightforward

Go-shaped recommendation

abstraction ReadAt(off, buf)
  if region cached / known resident → mmap view
  else → pread into buf
never let page faults surprise a hot P/M without budget

Prefer measuring tail latency under cold cache and concurrent queries, not only average throughput.

Experiment

# conceptual microbench outline
# 1) os.ReadAt loop over file
# 2) unix.Mmap + index loads
# compare under: warm cache vs drop_caches (Linux ops only)

What to notice: Warm mmap wins; cold fault storms can look like “Go scheduler freeze” when Ms stuck in fault handling.

Try next: Read the guest post’s VictoriaLogs-oriented rationale end-to-end; map it to your own on-disk layout.