Day 11 — Race Detector, Concurrency Patterns & errgroup

Updated

July 30, 2025

Day 11 — Race Detector, Concurrency Patterns & errgroup

Day 25 — Race detector, leaks & deadlocks

Stage III · ~3h (theory-heavy)
Goal: Read and fix data races with the race detector, recognize deadlock classes, and develop a practical leak-hunting habit—including awareness of Go 1.26 goroutine leak profiles.

Important

A program that “works” without -race can still be wrong. Data races are undefined behavior in Go’s memory model. Make -race non-optional for concurrent packages.

Why this day exists

Symptoms of concurrency bugs:

  • Flaky tests
  • Impossible counter values
  • Heisenbugs under load
  • Gradual memory growth (leaked goroutines)
  • Full process freeze (deadlock)

Tools:

Tool Catches
go test -race / go run -race Data races at runtime
Runtime deadlock detector All goroutines blocked
runtime.NumGoroutine Coarse leak signal
Go 1.26 goroutineleak profile (experiment) Many blocked-forever Gs
Manual review Logic races, lost wakeups

Theory 1 — What is a data race?

A data race occurs when:

  1. Two goroutines access the same memory location concurrently
  2. At least one access is a write
  3. No happens-before synchronization orders them

Not every bug is a data race (wrong lock ordering can be race-free yet deadlocked). Not every race detector silence means correct (logical races, atomic misuse across fields).

Example

var n int
go func() { n = 1 }()
go func() { fmt.Println(n) }() // race: concurrent read/write

Theory 2 — Enabling the race detector

go run -race .
go test -race ./...
go build -race -o app .

Properties:

  • Instrumentation slows CPU-bound code significantly (often 2–20×)
  • Uses more memory
  • Supported on major platforms (Go 1.26: linux/riscv64 race support added)
  • Not a substitute for production always-on race detection (too slow)—use in CI and local tests

CI recommendation for concurrent modules:

go test -race -count=1 ./...

-count=1 disables test caching so flaky races reappear.

Theory 3 — Reading a race report

Typical report sections:

  1. Read at stack A
  2. Previous write at stack B
  3. Goroutine creation stacks
  4. Your code lines highlighted

Method:

  1. Identify the shared variable
  2. Identify the missing synchronization
  3. Choose fix: mutex, atomic, channel ownership, or redesign
  4. Re-run until clean under load (-count=10 or higher)

False confidence

Absence of a race report on one run does not prove freedom—only that that execution didn’t trip instrumentation. Still, -race finds the vast majority of classic races.

Theory 4 — Deadlocks

Global deadlock

All goroutines are asleep; runtime prints:

fatal error: all goroutines are asleep - deadlock!

Partial deadlock / livelock

Some Gs stuck; process still “alive.” Harder—needs timeouts, metrics, leak profiles.

Classic lock ordering

G1: lock A → lock B
G2: lock B → lock A

Fix: global lock order (always A then B).

Channel deadlock patterns

  • Send without receiver
  • WaitGroup mismatch
  • Mutual wait on two channels

Theory 5 — Goroutine leaks

A leaked goroutine typically blocks forever on:

  • Channel op
  • Mutex
  • select without exit case
  • Condition wait

Go 1.26 experimental profile:

GOEXPERIMENT=goroutineleakprofile go test -race ./...
# pprof: goroutineleak profile / debug endpoint when enabled

Theory (reachability): if a G blocks on primitive P and nothing reachable from runnable code can unblock P, the G is leaked.

You do not need production enablement today—know the tool exists and prefer designs that cancel and join.

Coarse check in tests

func TestNoLeak(t *testing.T) {
    before := runtime.NumGoroutine()
    runScenario(t)
    // allow scheduler to settle
    deadline := time.Now().Add(2 * time.Second)
    for time.Now().Before(deadline) {
        if runtime.NumGoroutine() <= before+1 { // slack
            return
        }
        time.Sleep(20 * time.Millisecond)
    }
    t.Fatalf("goroutines before=%d after=%d", before, runtime.NumGoroutine())
}

Fragile but educational; prefer structured cancel (errgroup) over heuristics.

Theory 6 — Fix strategies catalog

Bug Fix
Shared counter atomic or mutex
Shared map mutex / sync.Map (know tradeoffs)
Init race sync.Once
Message ownership unclear channel transfer exclusive ownership
Early return leaves workers blocked on send buffered results, context cancel, or drain
Timer/ticker leak Stop + drain pattern

Worked example — race then fix

package bank

import "sync"

type Account struct {
    mu      sync.Mutex
    balance int
}

func (a *Account) Deposit(v int) {
    a.mu.Lock()
    defer a.mu.Unlock()
    a.balance += v
}

func (a *Account) Balance() int {
    a.mu.Lock()
    defer a.mu.Unlock()
    return a.balance
}

Test:

func TestAccountConcurrent(t *testing.T) {
    var a Account
    var wg sync.WaitGroup
    for i := 0; i < 1000; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            a.Deposit(1)
        }()
    }
    wg.Wait()
    if a.Balance() != 1000 {
        t.Fatalf("got %d", a.Balance())
    }
}
go test -race ./...

Remove the mutex temporarily and watch the race report—then restore.

Lab 1 — Break five programs

Workspace: ~/lab/90daysofx/01-go/day25

Create racy/ with intentional races:

  1. Concurrent map write (map[string]int without lock)
  2. Concurrent slice append from two goroutines
  3. Check-then-act without lock (if !exists { m[k]=v })
  4. Publishing a struct field without sync
  5. WaitGroup-add race (optional)

For each:

go test -race ./racy -run TestName

Save one full race report in notes/race-report.txt (or journal).

Lab 2 — Fix all five

Implement production-shaped fixes. Suite must pass:

go test -race -count=20 ./...

Lab 3 — Deadlock museum

Write programs that:

  1. Double-lock same mutex in one goroutine (self-deadlock)
  2. Lock order inversion between two goroutines (use timeouts in a supervisor test so the test suite doesn’t hang forever—or run manually)
  3. Unbuffered channel self-send in one goroutine

Document stack traces from runtime where applicable.

Lab 4 — Leak the pool, then plug it

  1. Start workers that send on unbuffered results
  2. Main returns early on first error without draining or cancelling
  3. Observe NumGoroutine growth

Fix with either:

  • Context cancel (Day 24), or
  • Drain remaining results after error, or
  • Buffered channel sized to in-flight

Re-check goroutine counts. Optionally experiment with Go 1.26 leak profile if available:

GOEXPERIMENT=goroutineleakprofile go test -race ./...

Common gotchas

Gotcha Fix
Only running -race once CI + -count
Ignoring races in benchmarks Don’t ship racy code even if “fast”
sync.Map as default Often plain map+mutex is clearer
Fixing races with time.Sleep Never—use real synchronization
Assuming deadlock detector finds partial hangs It won’t

Checkpoint

  • Can define data race in three conditions
  • Read a real race report and fixed the root cause
  • go test -race -count=20 clean on labs
  • Documented one deadlock and one leak scenario
  • Knows Go 1.26 goroutine leak profile exists
  • Added -race to personal checklist for Stage III

Commit

git add .
git commit -m "day25: race detector labs — break, fix, leak plug"

Write three personal gotchas before continuing.

Tomorrow

Day 26 — Concurrency patterns catalog: fan-out/fan-in, or-done, tee, bridge, and error propagation shapes you will reuse for years.

Day 26 — Concurrency patterns

Stage III · ~3h (theory-heavy)
Goal: Internalize a catalog of reusable concurrency patterns—fan-out/fan-in, or-done, tee, bounded parallelism, and error-aware pipelines—and implement fan-out with correct error propagation.

Note

Patterns are vocabulary. Name them in code reviews: “this is fan-out with an or-done parent,” not “a bunch of goroutines.”

Why this day exists

Days 19–25 gave primitives. Without patterns you re-invent:

  • How to stop a pipeline mid-stream
  • How to duplicate a stream
  • How to limit in-flight work while collecting errors

Today is a map of the territory before errgroup (Day 27) packages the most common structure.

Theory 1 — Fan-out / fan-in

Fan-out: multiple workers pull from one jobs channel (or each gets a subset).
Fan-in: merge multiple result channels into one (Day 21).

          ┌→ worker1 ─┐
jobs  →───┼→ worker2 ─┼→ results → consumer
          └→ worker3 ─┘

Use when work is independent and CPU/I/O parallelizable.

Theory 2 — Or-done (cancel-aware receive)

Problem: for v := range in ignores cancellation until the producer closes.

func orDone[T any](done <-chan struct{}, in <-chan T) <-chan T {
    out := make(chan T)
    go func() {
        defer close(out)
        for {
            select {
            case <-done:
                return
            case v, ok := <-in:
                if !ok {
                    return
                }
                select {
                case out <- v:
                case <-done:
                    return
                }
            }
        }
    }()
    return out
}

Modern code often uses ctx.Done() instead of a raw done channel—the shape is identical.

Theory 3 — Tee (split a stream)

func tee[T any](done <-chan struct{}, in <-chan T) (<-chan T, <-chan T) {
    out1 := make(chan T)
    out2 := make(chan T)
    go func() {
        defer close(out1)
        defer close(out2)
        for v := range orDone(done, in) {
            // careful: both sends must not block forever
            o1, o2 := out1, out2
            for i := 0; i < 2; i++ {
                select {
                case <-done:
                    return
                case o1 <- v:
                    o1 = nil // disable case
                case o2 <- v:
                    o2 = nil
                }
            }
        }
    }()
    return out1, out2
}

Use for “log and process” dual consumers. Prefer careful nil-case disabling so one slow tee side doesn’t deadlock forever without cancel.

Theory 4 — Bridge (chan of chans)

Flatten <-chan <-chan T into <-chan T:

func bridge[T any](done <-chan struct{}, chanCh <-chan <-chan T) <-chan T {
    out := make(chan T)
    go func() {
        defer close(out)
        for {
            var stream <-chan T
            select {
            case <-done:
                return
            case maybe, ok := <-chanCh:
                if !ok {
                    return
                }
                stream = maybe
            }
            for v := range orDone(done, stream) {
                select {
                case out <- v:
                case <-done:
                    return
                }
            }
        }
    }()
    return out
}

Appears when each job produces a channel of chunks.

Theory 5 — Bounded parallelism pattern

func mapLimit[T any, R any](ctx context.Context, items []T, limit int, fn func(context.Context, T) (R, error)) ([]R, error) {
    // semaphore + results collection; first error cancels ctx
}

Skeleton:

  1. Derive cancellable ctx
  2. Semaphore capacity limit
  3. Store results by index (avoid append races)
  4. First error → cancel
  5. Wait all in-flight
  6. Return first error or results

This is the heart of Day 27’s errgroup + SetLimit.

Theory 6 — Error propagation styles

Style Behavior
Best effort Collect all errors (errors.Join)
Fail fast Cancel siblings on first error
All-or-nothing Fail fast + no partial commit

Channels often carry:

type result[T any] struct {
    v   T
    err error
    idx int
}

Never lose the index if order matters.

Theory 7 — Pipeline with cancel (composed)

ctx → gen → transform → sink
         ↘ cancel on error

Each stage takes ctx and exits on Done, closing outbound channels so downstream ranges finish.

Worked example — fan-out with error awareness

package main

import (
    "context"
    "fmt"
    "sync"
)

func fanOut(ctx context.Context, jobs <-chan int, workers int, fn func(context.Context, int) (int, error)) <-chan result {
    out := make(chan result)
    var wg sync.WaitGroup
    wg.Add(workers)
    for i := 0; i < workers; i++ {
        go func() {
            defer wg.Done()
            for {
                select {
                case <-ctx.Done():
                    return
                case j, ok := <-jobs:
                    if !ok {
                        return
                    }
                    v, err := fn(ctx, j)
                    select {
                    case out <- result{v: v, err: err}:
                    case <-ctx.Done():
                        return
                    }
                }
            }
        }()
    }
    go func() {
        wg.Wait()
        close(out)
    }()
    return out
}

type result struct {
    v   int
    err error
}

func main() {
    ctx, cancel := context.WithCancel(context.Background())
    defer cancel()

    jobs := make(chan int)
    go func() {
        defer close(jobs)
        for i := 1; i <= 20; i++ {
            select {
            case jobs <- i:
            case <-ctx.Done():
                return
            }
        }
    }()

    out := fanOut(ctx, jobs, 4, func(ctx context.Context, n int) (int, error) {
        if n == 13 {
            return 0, fmt.Errorf("unlucky")
        }
        return n * n, nil
    })

    for r := range out {
        if r.err != nil {
            fmt.Println("err", r.err)
            cancel() // fail fast signal; workers exit
            // continue draining out until closed
            continue
        }
        fmt.Println(r.v)
    }
}

Improve tomorrow with errgroup so cancel + wait is less manual.

Lab 1 — Catalog implementations

Workspace: ~/lab/90daysofx/01-go/day26

Implement in package patterns:

  1. OrDone
  2. FanIn
  3. Tee (two outputs)
  4. MapLimit (bounded)
go test -race ./...

Table-driven tests with small channels and timeouts (t.Fatal if >2s).

Lab 2 — Fan-out square with fail-fast

Process numbers 1..100; function fails on multiples of 17. On first error, cancel remaining work. Assert:

  • Error returned to caller
  • Not all squares necessarily computed
  • No leaked goroutines after short settle (coarse)
  • Race-clean

Lab 3 — Best-effort mode

Same as Lab 2 but collect all errors with errors.Join and do not cancel. Compare counts of successes.

Journal when each mode is appropriate (payments vs metrics).

Lab 4 — Name the pattern

Take your Day 23/24 fetcher and add comments:

// Pattern: worker pool (fixed fan-out)
// Pattern: or-done via ctx.Done

Refactor one piece toward the catalog helpers.

Common gotchas

Gotcha Fix
Fan-in without waiting senders WaitGroup then close
Tee deadlock on slow consumer ctx cancel + nil cases
Losing error index order Result with index
Infinite goroutines in mapLimit Semaphore / fixed workers
Closing shared jobs from workers Only producer closes

Checkpoint

  • Implemented OrDone, FanIn, Tee, MapLimit
  • Explained fail-fast vs best-effort
  • Fan-out lab race-clean with cancel
  • Can sketch bridge pattern verbally
  • Annotated an older lab with pattern names

Commit

git add .
git commit -m "day26: concurrency patterns catalog + fan-out errors"

Write three personal gotchas before continuing.

Tomorrow

Day 27 — errgroup: structured concurrency with golang.org/x/sync/errgroup—first-error cancel and SetLimit.

Day 27 — errgroup

Stage III · ~3h (theory-heavy)
Goal: Use golang.org/x/sync/errgroup for structured concurrent work: wait for a group, return the first error, cancel siblings, and bound parallelism with SetLimit.

Note

errgroup is not in the standard library—it lives in golang.org/x/sync. It is the de-facto companion for production Go concurrency.

Why this day exists

Manual patterns from Day 26 work, but:

  • Error channels are easy to mishandle
  • Cancel + WaitGroup boilerplate repeats
  • Limits are reinvented poorly

errgroup.Group encodes a battle-tested contract:

Run N functions; wait; return the first non-nil error; optionally cancel the rest.

Theory 1 — Module setup

go get golang.org/x/sync/errgroup
import "golang.org/x/sync/errgroup"

Pin versions via go.mod like any other dependency. Keep x/sync reasonably current; API used here is stable.

Theory 2 — Basic Group

var g errgroup.Group

g.Go(func() error {
    return taskA()
})
g.Go(func() error {
    return taskB()
})

if err := g.Wait(); err != nil {
    return err
}

Rules:

  1. Go starts a goroutine immediately.
  2. Wait blocks until all finish.
  3. First non-nil error is returned; others are discarded (they still run unless using cancel context).
  4. If a function panics, Wait panics with the same value (don’t use panic for control flow).

Without a derived context, siblings keep running after the first error—they just don’t change the returned error.

Theory 3 — WithContext for fail-fast cancel

g, ctx := errgroup.WithContext(parent)

g.Go(func() error {
    return work(ctx, a)
})
g.Go(func() error {
    return work(ctx, b)
})

err := g.Wait() // ctx cancelled when first Go func returns error

Semantics:

  • WithContext returns a child context cancelled when the first Go function returns a non-nil error or when Wait returns
  • Pass ctx into all blocking ops
  • After Wait, resources associated with the group are done

This is the structured form of Day 26 fail-fast.

Theory 4 — SetLimit

g, ctx := errgroup.WithContext(ctx)
g.SetLimit(8) // max concurrent Go funcs running

for _, item := range items {
    item := item
    g.Go(func() error {
        return process(ctx, item)
    })
}
if err := g.Wait(); err != nil {
    return err
}
  • SetLimit(n) caps concurrent goroutines inside the group
  • Additional Go calls block until a slot frees (or panic if limit is 0? — limit must be positive; 0 means unlimited historically—check docs for your version; use positive n)
  • Call SetLimit before Go

This replaces many hand-rolled worker pools for the “run this function per item” shape.

Theory 5 — What errgroup is not

Not for Prefer
Streaming pipelines with continuous channels Channel stages
Long-lived worker servers Dedicated pool + lifecycle
Collecting all errors Mutex slice + errors.Join, or custom
Shared mutable aggregation without sync Still need mutex/atomic

Wait returns one error. For multierror:

var (
    mu   sync.Mutex
    errs []error
)
// inside Go: on error append under mu; return nil to not cancel
// after Wait: errors.Join(errs...)

Or return nil from workers and join manually—know you give up fail-fast cancel unless you cancel yourself.

Theory 6 — Panic and recovery policy

Prefer errors over panics. If a dependency panics:

  • errgroup will surface panic on Wait
  • Recover inside Go only if you have a clear policy:
g.Go(func() error {
    defer func() {
        if r := recover(); r != nil {
            // convert to error if required by policy
        }
    }()
    return risky()
})

Usually: fix the panic source.

Worked example — parallel fetch with limit

package main

import (
    "context"
    "fmt"
    "net/http"
    "time"

    "golang.org/x/sync/errgroup"
)

func headOK(ctx context.Context, url string) error {
    req, err := http.NewRequestWithContext(ctx, http.MethodHead, url, nil)
    if err != nil {
        return err
    }
    res, err := http.DefaultClient.Do(req)
    if err != nil {
        return fmt.Errorf("%s: %w", url, err)
    }
    res.Body.Close()
    if res.StatusCode >= 400 {
        return fmt.Errorf("%s: status %d", url, res.StatusCode)
    }
    return nil
}

func checkAll(ctx context.Context, urls []string, limit int) error {
    g, ctx := errgroup.WithContext(ctx)
    g.SetLimit(limit)
    for _, u := range urls {
        u := u
        g.Go(func() error {
            return headOK(ctx, u)
        })
    }
    return g.Wait()
}

func main() {
    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
    defer cancel()

    urls := []string{
        "https://example.com",
        "https://example.com/404", // may fail
    }
    if err := checkAll(ctx, urls, 4); err != nil {
        fmt.Println("failed:", err)
        return
    }
    fmt.Println("all ok")
}

Use -race in tests; live network optional—prefer fake servers (httptest) in automated tests.

Worked example — pure CPU with cancel

func squares(ctx context.Context, n int) ([]int, error) {
    out := make([]int, n)
    g, ctx := errgroup.WithContext(ctx)
    g.SetLimit(runtime.GOMAXPROCS(0))

    for i := 0; i < n; i++ {
        i := i
        g.Go(func() error {
            if err := ctx.Err(); err != nil {
                return err
            }
            if i == n/2 {
                return fmt.Errorf("boom at %d", i)
            }
            out[i] = i * i // each index exclusive — no race
            return nil
        })
    }
    if err := g.Wait(); err != nil {
        return nil, err
    }
    return out, nil
}

Index-exclusive writes need no mutex.

Lab 1 — Install and basic Wait

Workspace: ~/lab/90daysofx/01-go/day27

mkdir -p ~/lab/90daysofx/01-go/day27 && cd ~/lab/90daysofx/01-go/day27
go mod init example.com/day27
go get golang.org/x/sync/errgroup
  1. Run three tasks: two succeed, one fails.
  2. Print error from Wait.
  3. Race-clean.
go test -race ./...

Lab 2 — WithContext fail-fast

  1. Tasks sleep 50ms, 100ms, 200ms.
  2. The 50ms task returns an error.
  3. Instrument longer tasks to exit early via ctx.Done().
  4. Assert they did not run to full sleep (timing with tolerance).

Lab 3 — SetLimit correctness

Spawn 100 tasks each holding a critical section counter:

var current atomic.Int64
var max atomic.Int64
// on enter: c := current.Add(1); update max; sleep; current.Add(-1)

SetLimit(5) ⇒ observed max concurrent ≤ 5. Race-clean.

Lab 4 — Refactor Day 23/24 pool to errgroup

Replace hand-rolled worker pool for batch jobs with errgroup + SetLimit. Keep context timeout. Compare line count and clarity in journal.

Optional: multierror mode that does not cancel—document tradeoff.

Common gotchas

Gotcha Fix
Forgetting to use returned ctx All work must observe it
SetLimit after many Go Set limit first
Capturing loop variable wrong item := item before Go
Assuming all errors returned Only first; design multierror explicitly
Mixing panic with Wait Convert or prevent panics
Not go get x/sync Module dependency required

Checkpoint

  • Added golang.org/x/sync/errgroup to a module
  • Explains Group vs WithContext
  • Uses SetLimit and proved concurrency bound
  • Fail-fast lab shows sibling cancellation
  • Refactored one older pool toward errgroup
  • go test -race ./... clean

Commit

git add .
git commit -m "day27: errgroup WithContext + SetLimit labs"

Write three personal gotchas before continuing.

Tomorrow

Day 28 — Memory model lite: happens-before, what races really mean, and documenting synchronization so future you trusts the code.