Day 65 — pprof CPU & heap

Updated

July 30, 2026

Day 65 — pprof CPU & heap

Stage VII · ~3h
Goal: Capture CPU and heap profiles from a running program or test, interpret the top frames with go tool pprof, and write a short evidence note with one optimization hypothesis.

Why this day exists

Without profiles, “performance work” is cosplay. pprof turns gut feel into:

  • Where CPU time went
  • What allocated on the heap
  • Which call stacks dominate

You already built services and CLIs (Stages V–VI). Today you learn to observe them the way production teams do.

Theory 1 — What pprof is

The runtime/pprof and net/http/pprof packages expose samples of program state:

Profile Answers
CPU Where did execution time go (sampled stacks)?
heap What is in memory / what allocated?
goroutine What are goroutines doing / blocked on?
mutex / block Contention and blocking (enable rates carefully)
allocs Allocation sites (related to heap view)

Profiles are samples and aggregates. They are not a perfect wall-clock trace (that is closer to tracing / otel on Day 70).

Theory 2 — Two capture styles

A) HTTP endpoints (net/http/pprof)

package main

import (
    "log"
    "net/http"
    _ "net/http/pprof" // registers on DefaultServeMux
)

func main() {
    // Prefer a dedicated debug mux/server in real services — never expose pprof publicly.
    go func() {
        log.Println(http.ListenAndServe("localhost:6060", nil))
    }()
    // ... real server on :8080 with its own mux
    select {}
}

Safer pattern: separate mux on loopback only:

mux := http.NewServeMux()
mux.HandleFunc("/debug/pprof/", pprof.Index)
mux.HandleFunc("/debug/pprof/cmdline", pprof.Cmdline)
mux.HandleFunc("/debug/pprof/profile", pprof.Profile)
mux.HandleFunc("/debug/pprof/symbol", pprof.Symbol)
mux.HandleFunc("/debug/pprof/trace", pprof.Trace)
// heap, goroutine, etc. via pprof.Handler
go http.ListenAndServe("127.0.0.1:6060", mux)

Capture:

# 30s CPU profile
curl -o cpu.pb.gz "http://127.0.0.1:6060/debug/pprof/cpu?seconds=30"
# heap snapshot
curl -o heap.pb.gz "http://127.0.0.1:6060/debug/pprof/heap"

B) Tests and benchmarks

go test -cpuprofile=cpu.out -memprofile=mem.out -bench=. ./pkg
go test -cpuprofile=cpu.out -run=^$ -bench=BenchmarkHot ./...

Or programmatic:

f, _ := os.Create("cpu.prof")
pprof.StartCPUProfile(f)
defer pprof.StopCPUProfile()
// work...

Theory 3 — Reading profiles with go tool pprof

go tool pprof cpu.pb.gz
# interactive:
#   top
#   top -cum
#   list FuncName
#   web          # needs graphviz for full graph
#   png > cpu.png

Non-interactive:

go tool pprof -top cpu.pb.gz
go tool pprof -top -cum cpu.pb.gz
go tool pprof -list=hotFunction cpu.pb.gz
go tool pprof -http=:0 cpu.pb.gz   # local UI

How to read top

Column Meaning
flat Time/bytes in this function alone
cum Including callees
sum% Running total of flat

Start with top, then list the hottest symbols that are your code (not only runtime). Runtime frames are real but not always actionable on Day 1 of profiling.

Heap views

go tool pprof -top heap.pb.gz
# in interactive mode, common toggles:
#   inuse_space / inuse_objects
#   alloc_space / alloc_objects
  • inuse_*: currently held
  • alloc_*: cumulative allocations (great for “who allocates a lot even if freed”)

Theory 4 — A sane profiling workflow

  1. Reproduce load (loop, vegeta later on Day 80, or a benchmark).
  2. Capture CPU under that load.
  3. Capture heap under that load (and maybe at idle for comparison).
  4. Write the top 3 frames that look like your code.
  5. Form one hypothesis (“JSON marshal dominates; try pooling / smaller DTOs”).
  6. Change one thing tomorrow (Day 66 benchmarks) and re-measure.
Warning

Do not expose /debug/pprof on a public interface. It leaks internals and can be abused as a DoS vector. Bind to localhost or protect with network policy / auth.

Worked example — deliberate hot path

package hot

import (
    "encoding/json"
    "strings"
)

type Event struct {
    ID   string            `json:"id"`
    Tags map[string]string `json:"tags"`
}

func Process(raw []byte) (string, error) {
    var e Event
    if err := json.Unmarshal(raw, &e); err != nil {
        return "", err
    }
    // intentionally chatty string work
    var b strings.Builder
    for k, v := range e.Tags {
        b.WriteString(k)
        b.WriteByte('=')
        b.WriteString(v)
        b.WriteByte(';')
    }
    return b.String(), nil
}
// hot_test.go
package hot

import (
    "encoding/json"
    "testing"
)

func BenchmarkProcess(b *testing.B) {
    raw, _ := json.Marshal(Event{
        ID:   "x",
        Tags: map[string]string{"a": "1", "b": "2", "c": "3"},
    })
    b.ReportAllocs()
    b.ResetTimer()
    for i := 0; i < b.N; i++ {
        if _, err := Process(raw); err != nil {
            b.Fatal(err)
        }
    }
}
go test -bench=BenchmarkProcess -benchmem -cpuprofile=cpu.out -memprofile=mem.out .
go tool pprof -top cpu.out
go tool pprof -top mem.out

Write FINDINGS.md:

# day65 findings
## load
go test -bench=BenchmarkProcess ...
## CPU top (mine)
1. encoding/json.Unmarshal
2. hot.Process
3. strings.(*Builder).WriteString
## hypothesis
JSON dominates; if this is a real hot path, consider a tighter encoding or reuse buffers.

Lab

Suggested workspace: ~/lab/90daysofx/01-go/day65

  1. Pick a target: Stage VI service or the mini hot package above.
  2. Capture a CPU profile (HTTP 20–30s under load or via -cpuprofile on a benchmark).
  3. Capture a heap profile.
  4. Produce FINDINGS.md with top-3 symbols + one hypothesis + commands used.
  5. Optional: go tool pprof -http=:0 screenshot or note of the flame-ish UI.

Service load without Day 80 tools

# crude load while CPU profile runs
while true; do curl -s localhost:8080/api/health >/dev/null; done

Common gotchas

Gotcha Fix
Profile of idle process Generate realistic load during capture
Optimizing runtime frames only Look for your package prefixes
Public pprof Localhost / auth / separate port
Confusing alloc vs inuse State which view you used in FINDINGS
Huge profiles in git Add *.pb.gz / *.out to .gitignore; keep notes
“I need to rewrite everything” One hypothesis; measure on Day 66

Checkpoint

  • CPU profile file produced
  • Heap profile file produced
  • go tool pprof -top interpreted for both
  • FINDINGS.md with top-3 + hypothesis
  • pprof exposure policy understood

Commit

git add .
git commit -m "day65: pprof cpu heap findings"

Write three personal gotchas before continuing.

Tomorrow

Day 66 — Benchmarks & allocs: turn today’s hypothesis into a measured microbenchmark and an evidence-based micro-optimization.