Day 67 — GC, GOGC & GOMEMLIMIT

Updated

July 30, 2026

Day 67 — GC, GOGC & GOMEMLIMIT

Stage VII · ~3h
Goal: Build a correct mental model of Go’s garbage collector, measure a workload under default settings, then experiment with GOGC and GOMEMLIMIT—recording latency/memory trade-offs without cargo-cult tuning.

Why this day exists

Go’s GC is excellent for most services. Problems appear when:

Containers have a hard memory limit and the process OOMs
Latency spikes correlate with GC CPU
Someone set GOGC=off “for performance” and created a time bomb

You need awareness and measurement, not a permanent custom GC config for every binary.

Note

Go 1.26 / recent GC: release notes discuss ongoing GC work (including the “Green Tea” GC direction in the 1.25–1.26 era). Defaults change; measure on your toolchain (go version) and prefer official release notes over blog posts. For this volume, treat knobs below as still current unless your notes say otherwise.

Theory 1 — What the GC does

Go is garbage-collected. The runtime tracks heap pointers and reclaims unreachable objects. Cost shows up as:

CPU spent marking/sweeping
Latency effects (STW is short in modern Go, but assist and CPU steal still matter)
Memory retained as heap grows before next collection

You influence behavior primarily via:

Knob	Role
Allocation rate / live set	Your code (pools, reuse, fewer objects)
`GOGC`	Target heap growth relative to live data
`GOMEMLIMIT`	Soft memory limit the runtime tries to respect
`GOMEMLIMIT` + container cgroup	Align process with actual limit

Best optimization remains fewer allocations (Days 65–66)—tuning is second-line.

Live set vs garbage

Live set: objects still reachable
Garbage: unreachable; reclaimed on next cycles
High allocation rate with small live set → frequent GC work
Large live set → large heap baseline regardless of GOGC

Theory 2 — `GOGC`

GOGC is a percentage. Rough mental model:

GC aims to keep the heap near live_set * (1 + GOGC/100).

Value	Effect (simplified)
`100`	Default: allow ~100% growth over live set before collecting more eagerly
`50`	More frequent GC → lower peak heap, more CPU on GC
`200`	Larger heap, fewer GC cycles, more memory
`off`	Disables GC (dangerous; only for special tools with care)

GOGC=50  ./server
GOGC=200 ./server
GOGC=off ./server   # do not use in real services without a hard plan

Programmatic (rare in apps):

import "runtime/debug"

debug.SetGCPercent(50)
// return previous percent

Prefer environment configuration for ops-owned knobs so you can change without rebuild.

Theory 3 — `GOMEMLIMIT` (Go 1.19+)

GOMEMLIMIT sets a soft memory limit. The runtime becomes more aggressive about GC and returning memory as you approach the limit—reducing OOM kills when the limit matches the container.

# 256 MiB soft limit
GOMEMLIMIT=256MiB ./server

Practice	Detail
Set near cgroup limit	Leave headroom for non-Go memory, stacks, caches
Combine with container	Day 68: memory limits in Docker/K8s
Not a substitute for fixing leaks	Limit delays death; leak still wins eventually

Also:

debug.SetMemoryLimit(256 << 20)

Interaction with `GOGC`

With a memory limit, the runtime can collect earlier than pure GOGC would suggest when under pressure. Read the official GC guide when tuning for production. Rule of thumb for containers: set GOMEMLIMIT to a fraction of the cgroup limit (e.g. 90%) and leave GOGC default unless measured otherwise.

Theory 4 — Observing GC

`GODEBUG=gctrace=1`

GODEBUG=gctrace=1 go run .

Lines show GC cycles, heap sizes, CPU fractions. Dense but authoritative. Redirect stderr to a file for comparison.

`runtime.ReadMemStats`

var m runtime.MemStats
runtime.ReadMemStats(&m)
fmt.Printf("Alloc=%d KiB Sys=%d KiB NumGC=%d\n",
    m.Alloc/1024, m.Sys/1024, m.NumGC)

Useful fields for a lab table: HeapAlloc, HeapSys, PauseTotalNs, NumGC, GCCPUFraction (where available / meaningful).

Note: ReadMemStats stops the world briefly—fine for lab sampling, not for tight hot loops in production.

Metrics you will wire on Day 69

Prometheus go_* collectors already expose GC stats. Today, prefer direct MemStats / gctrace so the concept is clear.

`runtime/metrics` (modern)

import "runtime/metrics"

func sampleGCCycles() uint64 {
    samples := []metrics.Sample{
        {Name: "/gc/cycles/total:gc-cycles"},
    }
    metrics.Read(samples)
    return samples[0].Value.Uint64()
}

Exact metric names can vary by Go version—check metrics.All() on your toolchain. Optional stretch: sample one value instead of MemStats.

Theory 5 — What not to do

Anti-pattern	Why
`GOGC=off` on a server	Heap grows until OOM
Limit = exact container bytes	Non-heap + spikes → kill
Tuning every microservice differently without data	Ops complexity
Ignoring leaks because “GC will handle it”	GC does not free reachable leaks

Worked example — allocator churn workload

package main

import (
    "flag"
    "fmt"
    "os"
    "runtime"
    "time"
)

func main() {
    sec := flag.Int("sec", 10, "run seconds")
    size := flag.Int("size", 4096, "alloc size bytes")
    flag.Parse()

    deadline := time.Now().Add(time.Duration(*sec) * time.Second)
    var junk [][]byte
    var ops int
    for time.Now().Before(deadline) {
        b := make([]byte, *size)
        b[0] = 1
        // retain some to grow live set; drop some to create garbage
        if ops%3 == 0 {
            junk = append(junk, b)
            if len(junk) > 1000 {
                junk = junk[len(junk)/2:]
            }
        }
        ops++
    }

    var m runtime.MemStats
    runtime.ReadMemStats(&m)
    fmt.Fprintf(os.Stdout,
        "ops=%d HeapAlloc=%d MiB NumGC=%d PauseTotal=%v\n",
        ops, m.HeapAlloc>>20, m.NumGC, time.Duration(m.PauseTotalNs),
    )
}

Run matrix:

go version | tee GC.md
go build -o churn .
/usr/bin/time -l ./churn -sec=15            # macOS time -l; Linux: /usr/bin/time -v
GOGC=50         ./churn -sec=15
GOGC=200        ./churn -sec=15
GOMEMLIMIT=64MiB ./churn -sec=15
GODEBUG=gctrace=1 GOGC=50 ./churn -sec=5 2> gc50.txt

Fill a table in GC.md:

Config	HeapAlloc end	NumGC	Notes
default
GOGC=50
GOGC=200
GOMEMLIMIT=64MiB

Worked example — periodic MemStats in a service

When you already have an HTTP service, sample under synthetic load instead of only the churn toy:

func logMem(log *slog.Logger, every time.Duration, stop <-chan struct{}) {
    t := time.NewTicker(every)
    defer t.Stop()
    for {
        select {
        case <-stop:
            return
        case <-t.C:
            var m runtime.MemStats
            runtime.ReadMemStats(&m)
            log.Info("memstats",
                "heap_alloc_mib", m.HeapAlloc>>20,
                "heap_sys_mib", m.HeapSys>>20,
                "num_gc", m.NumGC,
                "pause_total", time.Duration(m.PauseTotalNs).String(),
            )
        }
    }
}

Drive traffic with a loop of curl or a tiny client; change only one env knob between runs. Paste three log lines into GC.md next to the config name.

Interpreting a gctrace line (lab reading guide)

You do not need to memorize every field. For each run, note roughly:

How often cycles appear (density of lines)
Whether heap goal / live sizes trend upward unbounded (leak smell)
Whether CPU fraction for GC dominates wall time under light load (unusual—check allocation rate)

If GOGC=50 shows more cycles and lower peak heap than default, your model is working. If numbers are noise-identical, extend -sec or increase allocation pressure.

Lab

Suggested workspace: ~/lab/90daysofx/01-go/day67

Implement the churn program or instrument your service with periodic MemStats logs under load.
Run at least three configs (default + GOGC change + GOMEMLIMIT).
Capture either gctrace snippets or MemStats rows.
Write a short recommendation: default is fine or set GOMEMLIMIT to match container with numbers.
Explicitly note why you would not set GOGC=off in a long-running service.
Record go env GOGC GOMEMLIMIT (often empty → defaults) next to results.

Stretch

Align GOMEMLIMIT with a Docker --memory experiment (pairs with Day 68).
Read the official GC guide section on memory limit soft vs hard OOM.
Force a GC with runtime.GC() between phases to stabilize lab readings (document that production rarely needs manual GC).
Compare GOGC=100 vs debug.SetGCPercent(100) equivalence with a one-line note.

Common gotchas

Gotcha	Fix
Tuning without alloc reduction	Fix hot allocs first (Days 65–66)
`GOGC=off` in server	Almost never; prefer limits + good code
Limit = container limit exactly	Leave headroom for non-heap memory
Comparing different workloads	Same `-sec`, `-size`, machine quiet
Ignoring cgroup vs process	In K8s, limit must match runtime awareness
Reading one MemStats sample	Sample under steady load; note GC mid-cycle variance
Mixing Go versions in a table	Record `go version` once per table

Checkpoint

Explain GOGC in one sentence with trade-off
Explain GOMEMLIMIT in one sentence
Measurement table with ≥3 configs
Written recommendation for your workload
go version recorded next to results
Explicit “no GOGC=off” rationale

Commit

git add .
git commit -m "day67: gc gogc gomemlimit measurements"

Write three personal gotchas before continuing.

Tomorrow

Day 68 — Containers & distroless: package a Go binary for production-like images with minimal attack surface.

Day 67 — GC, GOGC & GOMEMLIMIT

Why this day exists

Theory 1 — What the GC does

Live set vs garbage

Theory 2 — GOGC

Theory 3 — GOMEMLIMIT (Go 1.19+)

Interaction with GOGC

Theory 4 — Observing GC

GODEBUG=gctrace=1

runtime.ReadMemStats

Metrics you will wire on Day 69

runtime/metrics (modern)

Theory 5 — What not to do

Worked example — allocator churn workload

Worked example — periodic MemStats in a service

Interpreting a gctrace line (lab reading guide)

Lab

Stretch

Common gotchas

Checkpoint

Commit

Tomorrow

Theory 2 — `GOGC`

Theory 3 — `GOMEMLIMIT` (Go 1.19+)

Interaction with `GOGC`

`GODEBUG=gctrace=1`

`runtime.ReadMemStats`

`runtime/metrics` (modern)