Resource Limits, rlimit, and Go Knobs

Updated

September 8, 2026

Resource Limits, rlimit, and Go Knobs

Overview

Production processes run under resource ceilings: open files, memory, CPU, process count. Some are OS-enforced (rlimit, cgroups); some are Go runtime knobs (GOMAXPROCS, GOMEMLIMIT). Systems tools should fail clearly when limits bite—and avoid creating the conditions that hit them.

Discovering limits (Unix)

ulimit -n    # open files
ulimit -u    # max user processes

In Go (via x/sys/unix):

//go:build unix

var lim unix.Rlimit
if err := unix.Getrlimit(unix.RLIMIT_NOFILE, &lim); err == nil {
    fmt.Printf("nofile soft=%d hard=%d\n", lim.Cur, lim.Max)
}

Raising NOFILE (when allowed)

lim.Cur = lim.Max
_ = unix.Setrlimit(unix.RLIMIT_NOFILE, &lim)

Containers may still enforce a lower cgroup/nproc limit—raising soft rlimit cannot exceed hard or cgroup.

Common failure modes

Symptom Typical limit
too many open files NOFILE / FD leak
cannot allocate memory / OOMKill RSS cgroup / host RAM
Slow scheduling CPU quota / throttling
fork: resource temporarily unavailable nproc / PIDs

Go runtime knobs

Knob Role
GOMAXPROCS OS threads for Go code (auto-tuned in containers on recent Go)
GOMEMLIMIT Soft memory limit; GC aims to stay under
GOGC GC percent trigger
runtime/debug.SetMemoryLimit Same family as GOMEMLIMIT 
import "runtime/debug"

debug.SetMemoryLimit(2 << 30) // 2 GiB soft limit

See deep dives for pacer/GC details; here the systems takeaway is: set limits intentionally in containers.

Bound your own resource use

// concurrency cap
sem := make(chan struct{}, 32)

// HTTP client
t := http.DefaultTransport.(*http.Transport).Clone()
t.MaxIdleConns = 100
t.MaxConnsPerHost = 10

// file workers: don't open 100k files at once

Minimal tool: limits

func printLimits(w io.Writer) {
    fmt.Fprintf(w, "gomaxprocs\t%d\n", runtime.GOMAXPROCS(0))
    fmt.Fprintf(w, "numcpu\t%d\n", runtime.NumCPU())
    fmt.Fprintf(w, "numgoroutine\t%d\n", runtime.NumGoroutine())
    var ms runtime.MemStats
    runtime.ReadMemStats(&ms)
    fmt.Fprintf(w, "heap_alloc\t%d\n", ms.HeapAlloc)
    // optional: Getrlimit NOFILE
}

Ship in debug endpoints (GET /debug/limits) carefully—auth or localhost only.

cgroups (conceptual)

Kubernetes sets CPU/memory requests/limits via cgroups. Go 1.22+ improved GOMAXPROCS container awareness. Still verify:

pod memory limit < GOMEMLIMIT strategy
liveness does not restart during GC thrash

Rules of thumb

Do Don’t
Cap concurrency near I/O boundaries Unbounded go func per request/file
Close FDs; monitor NOFILE “Just raise ulimit” as only fix
Set GOMEMLIMIT in memory-capped pods Assume unlimited heap in sidecars
Log clear errors on EMFILE Busy-loop retry without backoff

Try next

  1. Print soft/hard NOFILE in a binary.
  2. Open files until failure; confirm error type.
  3. Run with GOMEMLIMIT=100MiB and allocate; observe GC behavior under load.