Day 29 — Concurrency integration

Updated

July 30, 2026

Day 29 — Concurrency integration

Stage III · ~3h (theory-heavy + build)
Goal: Integrate Stage III skills into one race-clean concurrent tool: pipeline or pool + context + tests + documented synchronization—ready for tomorrow’s gate.

Note

Integration day is not “new primitives.” It is composition under constraints: cancellation, bounds, errors, tests, and readability.

Why this day exists

Isolated days create isolated muscles. Production code needs:

  • Bounded work
  • Cooperative cancel
  • Observable errors
  • Tests that fail if races return
  • A README a teammate can run

Day 30 grades this composition. Build it seriously today.

Theory 1 — Choose a product shape

Pick one mini-tool (do not boil the ocean):

Option Description
A. Concurrent line processor Read paths/URLs/strings; transform with W workers; print/json results
B. Site prober HEAD/GET list of URLs with limit + timeout; summarize codes
C. Directory walker pipeline Enumerate files → hash/count → report (careful with real FS size)

All options must include:

  1. context timeout or cancel
  2. Bounded concurrency (errgroup.SetLimit or pool)
  3. Race-clean tests
  4. Non-zero exit on hard failure mode

Theory 2 — Package layout for a tiny tool

day29/
  go.mod
  README.md
  cmd/tool/main.go      # flags, os.Exit
  internal/engine/      # concurrent core
  internal/engine/engine_test.go

Rules:

  • Keep concurrency out of main as much as possible—test engine
  • Flags parse in main; pure functions take structs
  • No global mutable state

Theory 3 — API sketch (engine)

package engine

type Config struct {
    Workers int
    Timeout time.Duration
    // Mode: fail-fast vs best-effort
}

type Item struct {
    ID   string
    // payload
}

type Result struct {
    ID  string
    Err error
    // fields
}

func Run(ctx context.Context, cfg Config, items []Item) ([]Result, error)

Implementation preference for Stage III:

g, ctx := errgroup.WithContext(ctx)
g.SetLimit(cfg.Workers)
// per item: g.Go(...)

Or channel pipeline if you want streaming—document why.

Theory 4 — Testing concurrent engines

Deterministic unit tests

  • Fake work function injected (Processor interface or func field)
  • No real network in unit tests (use httptest if HTTP)

Race tests

go test -race -count=10 ./...

Cancel tests

ctx, cancel := context.WithCancel(context.Background())
// start Run in goroutine with slow processor
cancel()
// assert returns quickly with context.Canceled or wrapped

Limit tests

Atomic in-flight counter under fake processor sleep—max ≤ workers.

Theory 5 — CLI ergonomics (minimal)

tool -workers 8 -timeout 3s inputs.txt
  • Default workers: runtime.GOMAXPROCS(0) or 8
  • Timeout: wraps root context
  • Signals (optional stretch): signal.NotifyContext

Print summary to stderr; machine-readable results optional.

Theory 6 — Definition of done (integration)

Check Pass criteria
Race -race -count=10 clean
Cancel Timeout stops work
Bound Never exceeds workers in-flight
Errors Fail-fast or documented best-effort
Sync docs // Sync: or design note in README
UX README with run instructions

Worked skeleton — engine with errgroup

package engine

import (
    "context"
    "fmt"

    "golang.org/x/sync/errgroup"
)

type Config struct {
    Workers int
}

type Item struct{ N int }
type Result struct {
    N int
    Square int
}

func Run(ctx context.Context, cfg Config, items []Item, fn func(context.Context, Item) (Result, error)) ([]Result, error) {
    if cfg.Workers < 1 {
        return nil, fmt.Errorf("workers must be >= 1")
    }
    out := make([]Result, len(items))
    g, ctx := errgroup.WithContext(ctx)
    g.SetLimit(cfg.Workers)

    for i, it := range items {
        i, it := i, it
        g.Go(func() error {
            res, err := fn(ctx, it)
            if err != nil {
                return err
            }
            out[i] = res // exclusive index — Sync: each i written once
            return nil
        })
    }
    if err := g.Wait(); err != nil {
        return nil, err
    }
    return out, nil
}

Lab 1 — Scaffold

Workspace: ~/lab/90daysofx/01-go/day29

mkdir -p ~/lab/90daysofx/01-go/day29/{cmd/tool,internal/engine}
cd ~/lab/90daysofx/01-go/day29
go mod init example.com/day29
go get golang.org/x/sync/errgroup

Write README goals in 10 lines before coding.

Lab 2 — Implement core + tests

  1. Implement Run with injection point for work function.
  2. Table tests for happy path.
  3. Test fail-fast error.
  4. Test cancel.
  5. Test worker limit with atomic in-flight.
go test -race -count=10 ./...

Lab 3 — CLI wiring

cmd/tool/main.go:

  • Parse flags
  • Build items from args or file
  • context.WithTimeout
  • Print results / exit codes
go run -race ./cmd/tool -- -workers 4 -timeout 2s ...

Lab 4 — Chaos pass

  1. Set timeout shorter than work—expect clean cancel.
  2. Raise workers to 1 vs 64—behavior correct both ways.
  3. Empty input—returns nil error and empty results.
  4. Deliberately introduce a race; confirm -race catches; revert.

Common gotchas

Gotcha Fix
Testing only main Extract engine package
Global http.Client without timeout Use ctx on requests + client timeouts
Ignoring SetLimit Bound always
Partial results on error undefined Document: nil results vs partial slice
README missing Gate day will require it

Checkpoint

  • One integrated tool/module exists
  • Context + bounds + errors designed deliberately
  • go test -race -count=10 ./... clean
  • Cancel/timeout demonstrated
  • README run steps work from clean clone path
  • Sync strategy documented

Commit

git add .
git commit -m "day29: concurrency integration — engine + CLI + race tests"

Write three personal gotchas before continuing.

Tomorrow

Day 30 — Stage III gate: formal self-check—race-clean concurrent tool with graceful shutdown/cancel and oral/written explanations of GMP, channels, and context.