Day 12 — Memory Model, Integration & Stage III Gate

Updated

July 30, 2025

Day 12 — Memory Model, Integration & Stage III Gate

Day 28 — Memory model lite

Stage III · ~3h (theory-heavy)
Goal: Build a working happens-before intuition for Go’s memory model—enough to reason about races, atomics, channel sync, and mutexes without becoming a formal methods researcher.

Note

You do not need to recite the full Go memory model document. You need to know when a read is guaranteed to observe a write.

Why this day exists

Without a model:

  • “It worked on my laptop” becomes architecture
  • time.Sleep becomes synchronization
  • Atomics get sprinkled randomly
  • Channel closes feel magical

With a model:

  • You can document why a program is race-free
  • You can design simpler ownership
  • Race detector results make sense

Theory 1 — Sequential consistency within a goroutine

Inside one goroutine, reads and writes behave as if executed in program order (compiler/CPU reorderings are constrained so you cannot observe them in a single G without concurrency).

The hard part is cross-goroutine visibility.

Theory 2 — Happens-before (HB)

If event A happens before event B, then B is guaranteed to observe the effects of A (for the relevant memory).

Go establishes HB through synchronization events, including:

Synchronization HB edge (simplified)
Mutex Unlock → Lock Unlock HB subsequent Lock
Channel send → receive Send completes HB receive completes
Channel close → receive zero Close HB receive that sees closed
sync.Once Do Do’s f() HB return of any Do
sync.WaitGroup Done → Wait Done HB Wait return
Atomic ops Provide ordering depending on type (see docs)
Goroutine start go f initiation HB start of f

If two accesses to the same location race (no HB either way) and one is a write → data race.

Theory 3 — Initialization guarantees

  • Package-level variable initialization completes before init functions
  • All init complete before main
  • A package’s imported dependencies initialize first

You can rely on:

var global = compute() // finished before main

You cannot rely on unsynchronized writes from other goroutines without HB.

Theory 4 — Incorrect “synchronization”

These do not establish a portable HB for your data:

// NOT synchronization for shared data
time.Sleep(time.Millisecond)
runtime.Gosched()
fmt.Println("debug") // may seem to "fix" races via timing

They may change interleaving probability; they do not make a race correct.

Theory 5 — Channel as memory barrier (practical)

// G1
data = 42
ch <- struct{}{}

// G2
<-ch
fmt.Println(data) // guaranteed to see 42

Send/receive pair synchronizes. Similarly, closing and ranging establishes ordering for published data before close.

Mutex pattern

// writer
mu.Lock()
data = 42
mu.Unlock()

// reader
mu.Lock()
v := data
mu.Unlock()

Unlock HB next Lock ⇒ reader sees writer’s updates.

Theory 6 — Atomics and flags

var ready atomic.Bool
var data int

// G1
data = 42
ready.Store(true) // publish

// G2
if ready.Load() {
    fmt.Println(data) // sees 42 under atomic's ordering for this pattern
}

Use atomics deliberately; for multi-field structs prefer:

type snap struct{ /* immutable fields */ }
var p atomic.Pointer[snap]
p.Store(&snap{...}) // publish immutable snapshot

Readers Load pointer and use fields without further locks if truly immutable.

Theory 7 — Documenting synchronization

In code reviews, leave a breadcrumb:

// Sync: data is written only before ch is closed; readers range ch.
// Sync: mu protects m and n together.
// Sync: ready atomic publishes immutable cfg pointer.

If you cannot write one sentence naming the HB edge, the design is probably wrong.

Worked example — broken then proven

Broken:

var a, b int

func f() {
    a = 1
    b = 1
}

func g() {
    for b == 0 {
        runtime.Gosched()
    }
    fmt.Println(a) // may print 0: race + no HB
}

Fixed with channel:

done := make(chan struct{})
go func() {
    a = 1
    b = 1
    close(done)
}()
<-done
fmt.Println(a, b) // 1 1

Worked example — WaitGroup as publication

var data int
var wg sync.WaitGroup
wg.Add(1)
go func() {
    defer wg.Done()
    data = 7
}()
wg.Wait()
fmt.Println(data) // 7 — Done HB Wait

Lab 1 — Annotate old code

Workspace: ~/lab/90daysofx/01-go/day28

  1. Reopen Day 19–27 labs.
  2. Add // Sync: comments on every shared variable.
  3. If you cannot annotate, rewrite until you can.
go test -race ./...

Lab 2 — Prove one race you fixed

Write doc/race-story.md (or journal section) with:

  1. The racy snippet
  2. Race detector excerpt
  3. The HB edge you introduced
  4. Why sleep was not an acceptable fix

Lab 3 — Publication patterns matrix

Implement three versions of “publish config to readers”:

Version Mechanism
A Mutex around config struct
B Channel close after write
C atomic.Pointer to immutable config

Table tests: many readers, occasional writers. Race-clean. Journal tradeoffs (latency, API shape).

Lab 4 — Read the official model (targeted)

Open https://go.dev/ref/mem. Skim:

  • Introduction
  • Synchronization (channels, mutexes)
  • Incorrect synchronization

Write five bullet takeaways in your own words—no paste.

Common gotchas

Gotcha Fix
Double-checked locking without atomics/fences Use Once or proper atomic publish
Reading fields after free-unlock copy mistakes Keep lock scope correct
Believing race detector is the memory model Detector implements checks; model is the law
“Single writer so no sync needed” for readers Readers still need HB with writer
Over-using atomics for complex state Mutex or channel ownership

Checkpoint

  • Defines happens-before in plain language
  • Lists ≥4 Go primitives that create HB edges
  • Documented Sync comments on a real lab
  • Wrote a race-fix story with evidence
  • Compared three publication patterns
  • All day28 tests pass with -race

Commit

git add .
git commit -m "day28: memory model lite + sync annotations + race story"

Write three personal gotchas before continuing.

Tomorrow

Day 29 — Concurrency integration: combine pipeline + context + tests + race-clean design into one mini-tool.

Day 29 — Concurrency integration

Stage III · ~3h (theory-heavy + build)
Goal: Integrate Stage III skills into one race-clean concurrent tool: pipeline or pool + context + tests + documented synchronization—ready for tomorrow’s gate.

Note

Integration day is not “new primitives.” It is composition under constraints: cancellation, bounds, errors, tests, and readability.

Why this day exists

Isolated days create isolated muscles. Production code needs:

  • Bounded work
  • Cooperative cancel
  • Observable errors
  • Tests that fail if races return
  • A README a teammate can run

Day 30 grades this composition. Build it seriously today.

Theory 1 — Choose a product shape

Pick one mini-tool (do not boil the ocean):

Option Description
A. Concurrent line processor Read paths/URLs/strings; transform with W workers; print/json results
B. Site prober HEAD/GET list of URLs with limit + timeout; summarize codes
C. Directory walker pipeline Enumerate files → hash/count → report (careful with real FS size)

All options must include:

  1. context timeout or cancel
  2. Bounded concurrency (errgroup.SetLimit or pool)
  3. Race-clean tests
  4. Non-zero exit on hard failure mode

Theory 2 — Package layout for a tiny tool

day29/
  go.mod
  README.md
  cmd/tool/main.go      # flags, os.Exit
  internal/engine/      # concurrent core
  internal/engine/engine_test.go

Rules:

  • Keep concurrency out of main as much as possible—test engine
  • Flags parse in main; pure functions take structs
  • No global mutable state

Theory 3 — API sketch (engine)

package engine

type Config struct {
    Workers int
    Timeout time.Duration
    // Mode: fail-fast vs best-effort
}

type Item struct {
    ID   string
    // payload
}

type Result struct {
    ID  string
    Err error
    // fields
}

func Run(ctx context.Context, cfg Config, items []Item) ([]Result, error)

Implementation preference for Stage III:

g, ctx := errgroup.WithContext(ctx)
g.SetLimit(cfg.Workers)
// per item: g.Go(...)

Or channel pipeline if you want streaming—document why.

Theory 4 — Testing concurrent engines

Deterministic unit tests

  • Fake work function injected (Processor interface or func field)
  • No real network in unit tests (use httptest if HTTP)

Race tests

go test -race -count=10 ./...

Cancel tests

ctx, cancel := context.WithCancel(context.Background())
// start Run in goroutine with slow processor
cancel()
// assert returns quickly with context.Canceled or wrapped

Limit tests

Atomic in-flight counter under fake processor sleep—max ≤ workers.

Theory 5 — CLI ergonomics (minimal)

tool -workers 8 -timeout 3s inputs.txt
  • Default workers: runtime.GOMAXPROCS(0) or 8
  • Timeout: wraps root context
  • Signals (optional stretch): signal.NotifyContext

Print summary to stderr; machine-readable results optional.

Theory 6 — Definition of done (integration)

Check Pass criteria
Race -race -count=10 clean
Cancel Timeout stops work
Bound Never exceeds workers in-flight
Errors Fail-fast or documented best-effort
Sync docs // Sync: or design note in README
UX README with run instructions

Worked skeleton — engine with errgroup

package engine

import (
    "context"
    "fmt"

    "golang.org/x/sync/errgroup"
)

type Config struct {
    Workers int
}

type Item struct{ N int }
type Result struct {
    N int
    Square int
}

func Run(ctx context.Context, cfg Config, items []Item, fn func(context.Context, Item) (Result, error)) ([]Result, error) {
    if cfg.Workers < 1 {
        return nil, fmt.Errorf("workers must be >= 1")
    }
    out := make([]Result, len(items))
    g, ctx := errgroup.WithContext(ctx)
    g.SetLimit(cfg.Workers)

    for i, it := range items {
        i, it := i, it
        g.Go(func() error {
            res, err := fn(ctx, it)
            if err != nil {
                return err
            }
            out[i] = res // exclusive index — Sync: each i written once
            return nil
        })
    }
    if err := g.Wait(); err != nil {
        return nil, err
    }
    return out, nil
}

Lab 1 — Scaffold

Workspace: ~/lab/90daysofx/01-go/day29

mkdir -p ~/lab/90daysofx/01-go/day29/{cmd/tool,internal/engine}
cd ~/lab/90daysofx/01-go/day29
go mod init example.com/day29
go get golang.org/x/sync/errgroup

Write README goals in 10 lines before coding.

Lab 2 — Implement core + tests

  1. Implement Run with injection point for work function.
  2. Table tests for happy path.
  3. Test fail-fast error.
  4. Test cancel.
  5. Test worker limit with atomic in-flight.
go test -race -count=10 ./...

Lab 3 — CLI wiring

cmd/tool/main.go:

  • Parse flags
  • Build items from args or file
  • context.WithTimeout
  • Print results / exit codes
go run -race ./cmd/tool -- -workers 4 -timeout 2s ...

Lab 4 — Chaos pass

  1. Set timeout shorter than work—expect clean cancel.
  2. Raise workers to 1 vs 64—behavior correct both ways.
  3. Empty input—returns nil error and empty results.
  4. Deliberately introduce a race; confirm -race catches; revert.

Common gotchas

Gotcha Fix
Testing only main Extract engine package
Global http.Client without timeout Use ctx on requests + client timeouts
Ignoring SetLimit Bound always
Partial results on error undefined Document: nil results vs partial slice
README missing Gate day will require it

Checkpoint

  • One integrated tool/module exists
  • Context + bounds + errors designed deliberately
  • go test -race -count=10 ./... clean
  • Cancel/timeout demonstrated
  • README run steps work from clean clone path
  • Sync strategy documented

Commit

git add .
git commit -m "day29: concurrency integration — engine + CLI + race tests"

Write three personal gotchas before continuing.

Tomorrow

Day 30 — Stage III gate: formal self-check—race-clean concurrent tool with graceful shutdown/cancel and oral/written explanations of GMP, channels, and context.

Day 30 — Stage III gate

Stage III · Gate day · ~3h
Goal: Prove concurrency competence: a race-clean concurrent tool with cancellation, bounded workers, tests, and clear explanations of the Stage III mental model.

Important

Gates are pass/fail against a rubric, not calendar guilt. If something fails, fix it today or schedule a re-gate before Stage IV.

Why this day exists

Stage III (Days 19–29) stacked:

Day Skill
19 Goroutines, GMP lite, join
20 Channels, close ownership
21 Select, timeouts
22 Mutex, atomic, WaitGroup
23 Worker pools, pipelines
24 Context cancel/deadlines
25 Race detector, leaks
26 Patterns catalog
27 errgroup
28 Memory model lite
29 Integration build

Isolated labs create isolated muscles. The gate asks whether these compose under pressure—not whether you memorized slogans. Generics, fuzzing, and I/O depth in Stage IV will multiply concurrent bugs if your mental model is still hand-wavy.

Theory 1 — Gate artifact (definition of done)

You need a single module (Day 29 project refined is ideal) that:

  1. Performs concurrent work on ≥ 20 items
  2. Bounds concurrency (pool or errgroup.SetLimit)
  3. Accepts a root context (timeout flag and/or cancel)
  4. Propagates errors intentionally (fail-fast or best-effort—documented)
  5. Passes go test -race -count=20 ./... on Go 1.26
  6. Ships README with: purpose, run, test, design notes (sync + cancel)

Stretch (not required, impressive)

  • signal.NotifyContext for Ctrl-C
  • Structured logging of worker IDs (slog)
  • Benchmark with/without race detector noted
  • Metrics counter for completed jobs (even a simple atomic)

Explicit non-goals

  • No HTTP server requirement (that is Stage V+)
  • No database
  • No perfect CLI UX framework
  • No “must use every primitive from Days 19–28”

Theory 2 — What “race-clean under pressure” means

A single green go test -race is weak evidence. Flaky races show up under:

  • Higher -count
  • Parallel tests (t.Parallel)
  • Contended maps without mutex
  • Closing channels from the wrong side

Gate standard:

go test -race -count=20 ./...

If it fails once in twenty, it fails the gate. Fix the happens-before edge; do not paper over with time.Sleep.

Mini mental model (must be explainable)

Concept One-liner you must own
Data race Concurrent write/write or write/read without sync
Happens-before Channel, mutex, WaitGroup, atomics, etc. create edges
Cooperative cancel ctx.Done() observed; work stops voluntarily
Bound Limit in-flight work so you cannot create unbounded goroutines

Theory 3 — Oral/written explanation board

Answer in writing (half page each) in GATE.md. Use your project as the example, not textbook prose alone.

A. GMP

Explain G, M, P and what GOMAXPROCS limits. One misconception you no longer hold (e.g. “more goroutines always means more cores used”).

B. Channel ownership

Who closes? What happens on send-to-closed? When is a nil channel useful in a select?

C. Context

Why cancellation is cooperative. Why defer cancel() matters. When not to use context values for optional config.

D. Race

Define data race. Paste or paraphrase one race you fixed and the happens-before edge you introduced (mutex, channel send/receive, atomic, wait).

E. Mutex vs channel vs errgroup

One sentence each for when you pick them in this project.

F. Error policy (added for gate clarity)

Fail-fast (cancel siblings, return first error) vs best-effort (collect errors, continue). Quote which you chose and why.

Theory 4 — Rubric (score yourself)

Criterion Weight 0 1 2
Race-clean under -count=20 25% fails flaky solid
Bounded concurrency 15% unbounded informal explicit limit
Context cancel works 15% none partial demonstrated
Error policy clear 10% silent ad-hoc documented
Tests quality 15% none smoke cancel+limit+happy
Explanations A–F 15% missing shallow crisp
README / UX 5% none minimal clear

Pass: ≥ 80% weighted with no zero in race-clean or explanations.

Self-score honestly in GATE.md:

## Rubric

| Criterion | Score (0–2) | Notes |
|-----------|-------------|-------|
| Race-clean | 2 | count=20 green |
| Bounded concurrency | 2 | SetLimit(8) |
| Context cancel | 2 | -timeout works |
| Error policy | 1 | documented late |
| Tests | 2 | cancel + happy |
| Explanations | 2 | A–F written |
| README | 2 | demo script OK |
| **Weighted** | **≈ 95%** | PASS |

Theory 5 — Graceful shutdown checklist

For the gate tool (CLI/worker, not full HTTP drain):

  • Timeout flag or signal cancels root context
  • Workers observe ctx.Done() on each unit of work
  • Wait / errgroup / join completes after cancel
  • No permanent hang on empty input or immediate error
  • Process exits with meaningful code (0 success, non-zero hard fail)

“Graceful” here means cooperative cancel + join, not Stage V+ http.Server.Shutdown.

Pattern sketch (fail-fast pool)

func Run(ctx context.Context, jobs []Job, limit int) error {
    g, ctx := errgroup.WithContext(ctx)
    g.SetLimit(limit)
    for _, job := range jobs {
        job := job
        g.Go(func() error {
            if err := ctx.Err(); err != nil {
                return err
            }
            return process(ctx, job)
        })
    }
    return g.Wait()
}

Pattern sketch (best-effort with mutex)

var (
    mu   sync.Mutex
    errs []error
)
// each worker:
if err := process(ctx, job); err != nil {
    mu.Lock()
    errs = append(errs, err)
    mu.Unlock()
}
// after WaitGroup: join errs with errors.Join

Document which shape you shipped.

Theory 6 — Anti-patterns that fail the gate

Anti-pattern Why it fails
Sleep-based “sync” Hides races; flaky under load
Ignoring -race failures Gate criterion is race-clean
Unbounded go in a loop on the production path OOM / scheduler thrash
Closing channels from multiple goroutines unsafely Panic / races
Panic as control flow Unreadable; recover is not a strategy
Giant untested main No evidence under -count=20
Sharing a map without mutex or confinement Classic race
Forgetting defer cancel() Context leak; children outlive intent

Theory 7 — Minimum test matrix

Your tests should cover at least:

Case Intent
Happy path ≥20 jobs complete correctly
Cancel / timeout Work stops; no hang; returns context error or wrapped
Bound smoke Limit is set (even if hard to assert exactly)
Empty input No deadlock 
func TestRun_Cancel(t *testing.T) {
    ctx, cancel := context.WithCancel(context.Background())
    cancel()
    err := Run(ctx, makeJobs(50), 4)
    if !errors.Is(err, context.Canceled) && err != nil {
        // accept wrapped cancel depending on design
        t.Logf("err=%v (document policy)", err)
    }
}

Prefer asserting no hang with t.Deadline / short timeout over perfect error equality if your design aggregates.

Lab 1 — Harden Day 29 into gate shape

Workspace: Day 29 module or ~/lab/90daysofx/01-go/day30

  1. Confirm go 1.26 in go.mod and go version.
  2. Re-run full tests with race, count 20.
  3. Add any missing cancel/limit/empty-input tests.
  4. Fix README gaps (flags, examples, error policy paragraph).
  5. Record rubric scores honestly in GATE.md.
go version   # go1.26.x
go test -race -count=20 ./...
go vet ./...

Lab 2 — Live demo script

Write DEMO.md (or scripts/demo.sh) with commands a stranger can run:

# happy — process enough work to exercise the pool
go run ./cmd/tool --workers 4 --input ./testdata/jobs.txt

# timeout — must exit promptly
go run ./cmd/tool -timeout 1ms --workers 8 --input ./testdata/jobs.txt

# tests
go test -race -count=20 ./...

Run the demo once end-to-end without editing mid-flight. If a command fails, fix code/docs, then re-run the full script.

Lab 3 — Explanation document

Create GATE.md with answers A–F and the rubric table. Timebox 45 minutes. Clarity > length. Every answer should mention a concrete file or type in your module.

Template:

# Stage III Gate — day30

Date:
Module path:
Go version:

## Artifact
- Binary/cmd:
- Bound mechanism:
- Cancel mechanism:
- Error policy: fail-fast | best-effort

## Explanations
### A. GMP
...
### B. Channel ownership
...
### C. Context
...
### D. Race fixed
...
### E. Mutex vs channel vs errgroup
...
### F. Error policy
...

## Rubric
...

## Appendix — race detector drill

Lab 4 — Deliberate regression drill

  1. Break a synchronization on purpose (e.g. remove a mutex around a shared counter).
  2. Show -race failing; capture output snippet.
  3. Restore the fix.
  4. Note the detector output in GATE.md appendix.

This proves you can use the detector under stress—not that you never introduce races.

# after deliberate break:
go test -race ./internal/... -count=5
# restore, re-run count=20

Lab 5 — Exit codes and UX polish (short)

Scenario Expected exit
All jobs OK 0
Hard failure / fail-fast error non-zero
Cancel via timeout non-zero (or documented zero—be consistent) 
func main() {
    ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt)
    defer cancel()
    // or: context.WithTimeout from -timeout flag
    if err := realMain(ctx); err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
}

Hour-by-hour suggestion

Time Focus
0:00–0:30 Inventory Day 29 gaps vs gate bar
0:30–1:15 Harden code + tests (race, cancel)
1:15–1:45 DEMO.md dry run
1:45–2:30 GATE.md explanations A–F
2:30–2:50 Regression drill + rubric
2:50–3:00 Commit; list Stage IV risks

Common gotchas

Gotcha Fix
“Works on my machine” once -count=20 with -race
README outdated flags Re-run demo from zero
Explanations copy docs Use your project as example
Partial gate Schedule re-sit; don’t enter Stage IV blind
Timeout too long in tests Use short WithTimeout in unit tests
errgroup without SetLimit Add explicit bound
Logging from many goroutines without care slog is safe; still avoid huge allocations in hot paths
Treating cancel as optional Gate requires demonstrated cancel

Checkpoint

  • Rubric ≥ 80% with no zero on race-clean or explanations
  • go test -race -count=20 ./... pass on Go 1.26
  • Cancel/timeout demonstrated in demo
  • GATE.md explanations A–F complete
  • DEMO.md / demo script commands work cold
  • Regression drill appendix present
  • Honest list of remaining weaknesses

Commit

git add .
git commit -m "day30: stage III gate — race-clean concurrent tool + GATE notes"

Write three personal gotchas before continuing.

Tomorrow

Stage IV begins — Day 31 Generics: type parameters and constraints after you have felt real duplication pain in concurrent and sequential code. Enter Stage IV only with a race-clean mental model—or schedule a re-gate first.