Day 19 — Templates, httptest & Stage V Gate

Day 19 — Templates, httptest & Stage V Gate

Day 48 — templates & regexp

Stage V · ~3h
Goal: Use text/template and html/template correctly, compile regexp once, and complete one real text task (report rendering + field extraction) with tests.

Why this day exists

Not every interface is JSON. You will:

• Emit emails, CLI reports, config snippets → text/template
  
• Emit HTML → html/template (auto-escaping)
  
• Validate/extract tokens → regexp

Mixing up HTML vs text templates is an XSS footgun. Compiling regexps in hot loops is a performance footgun.

---

Theory 1 — text/template

const reportTmpl = `Items: {{len .Items}}
{{range .Items}}- {{.Name}} ({{.ID}})
{{end}}`

t, err := template.New("report").Parse(reportTmpl)
if err != nil {
    return err
}
var buf bytes.Buffer
if err := t.Execute(&buf, data); err != nil {
    return err
}

Actions (core)

Action	Meaning
{{.Field}}	Print field
{{if .OK}}...{{end}}	Conditional
{{range .Items}}...{{end}}	Iterate
{{with .User}}...{{end}}	Set dot
{{define "x"}}...{{end}}	Named template
{{template "x" .}}	Invoke

FuncMap

t, err := template.New("r").Funcs(template.FuncMap{
    "upper": strings.ToUpper,
    "ms": func(d time.Duration) int64 { return d.Milliseconds() },
}).Parse(`{{upper .Name}} took {{ms .Dur}}ms`)

Functions must be registered before Parse.

---

Theory 2 — html/template auto-escape

import htmltemplate "html/template"

t, err := htmltemplate.New("page").Parse(`<h1>{{.Title}}</h1><p>{{.Body}}</p>`)

If Body is <script>alert(1)</script>, it is escaped for HTML context. Context-aware escaping also handles attributes, JS, CSS URLs—still do not disable escaping casually.

// Dangerous — only for trusted, already-sanitized HTML:
htmltemplate.HTML("<b>trusted</b>")

Rule: user content → normal pipeline; never cast to HTML without a sanitizer policy.

---

Theory 3 — regexp

var tokenRE = regexp.MustCompile(`^[a-z][a-z0-9_-]{2,31}$`)

if !tokenRE.MatchString(s) {
    return fmt.Errorf("invalid token")
}

API	Use
Compile / MustCompile	Once at init
MatchString	Boolean
FindStringSubmatch	Capture groups
ReplaceAllString	Rewrite
Split	Divide

Submatches

re := regexp.MustCompile(`(?P<user>[^@]+)@(?P<host>[^@]+)`)
m := re.FindStringSubmatch("ada@example.com")
// m[0] full; m[1] user; m[2] host
// names: re.SubexpNames()

Performance & safety

• Prefer simple strings functions when regex is overkill.
  
• Avoid catastrophic backtracking patterns on untrusted input (pathological regex DoS).
  
• MustCompile at package init panics on bad patterns—good for static patterns.

---

Theory 4 — When not to use either

Problem	Prefer
Structured API output	encoding/json
Simple prefix/suffix	strings.Cut*
Binary protocols	custom codecs / protobuf later
SQL	query builders / sqlc—not templates for values

Never assemble SQL with templates/string concat for user input.

---

Worked example — inventory report + ID extract

package report

import (
    "bytes"
    "fmt"
    "regexp"
    "text/template"
    "time"
)

var idRE = regexp.MustCompile(`\bid=([A-Za-z0-9_-]+)\b`)

type Item struct {
    ID   string
    Name string
    Qty  int
}

type Data struct {
    GeneratedAt time.Time
    Items       []Item
}

const tmpl = `Inventory report {{.GeneratedAt.Format "2006-01-02 15:04:05 MST"}}
Count: {{len .Items}}
{{range .Items}}- {{.Name}} [{{.ID}}] qty={{.Qty}}
{{end}}`

func Render(data Data) (string, error) {
    t, err := template.New("inv").Parse(tmpl)
    if err != nil {
        return "", err
    }
    var buf bytes.Buffer
    if err := t.Execute(&buf, data); err != nil {
        return "", err
    }
    return buf.String(), nil
}

func ExtractIDs(text string) []string {
    matches := idRE.FindAllStringSubmatch(text, -1)
    out := make([]string, 0, len(matches))
    for _, m := range matches {
        if len(m) > 1 {
            out = append(out, m[1])
        }
    }
    return out
}

func ValidateID(id string) error {
    if !regexp.MustCompile(`^[A-Za-z0-9_-]+$`).MatchString(id) {
        return fmt.Errorf("bad id")
    }
    return nil
}

Improve ValidateID by lifting the regexp to package level (lab requirement).

---

Lab 1 — Setup

mkdir -p ~/lab/90daysofx/01-go/day48
cd ~/lab/90daysofx/01-go/day48
go mod init example.com/day48

---

Lab 2 — Text report

1. Parse template once (sync.Once or package init / Must).
   
2. Render sample inventory.
   
3. Table test: empty items still prints count 0.
   
4. FuncMap: qtyStatus returns "low" if qty < 3 else "ok".

---

Lab 3 — HTML page (escape proof)

// title/body from "user"
// assert output does not contain raw <script>

Render with html/template; input Body=<script>alert(1)</script>; assert escaped entities in output.

---

Lab 4 — Regexp extract

Given log lines:

level=info id=abc-01 msg=created
level=error id=xyz_2 msg=failed

Extract all ids with compiled regexp; test expected slice.

---

Lab 5 — CLI

go run . -in items.json -format text|html > out

JSON in → template out. Reuse Day 41 skills lightly.

---

Common gotchas

Gotcha	Fix
text/template for HTML	Use html/template
HTML cast on user data	Don’t
FuncMap after Parse	Register before Parse
Compile regexp per request	Package-level MustCompile
Greedy catastrophic patterns	Simplify; bound input
Template SQL	Parameterized queries only
Forgetting Execute error	Always check

---

Checkpoint

• Text report template with range/if
  
• FuncMap used
  
• HTML auto-escape proven by test
  
• Regexp compiled once; extract + validate
  
• Can explain text vs html package choice
  
• CLI or library API documented

Commit

git add .
git commit -m "day48: templates and regexp report pipeline"

Write three personal gotchas before continuing.

---

Tomorrow

Day 49 — httptest: table-driven handler tests for your API—status, headers, body, and middleware behavior without a real port.

---

Day 49 — httptest handler tests

Stage V · ~3h
Goal: Fully test HTTP handlers with net/http/httptest—table-driven routes, status/body/header assertions, middleware checks—without flaky port binds.

Why this day exists

Manual curl does not scale. Regression suites need:

• Fast in-process requests
  
• Deterministic status and JSON bodies
  
• Coverage of 400/404/405/500 paths
  
• No race on :8080 already in use

httptest is the stdlib way. Combined with Day 47 slog and Day 45 routing, this is the testing backbone for Stage V gate and beyond.

---

Theory 1 — ResponseRecorder

req := httptest.NewRequest(http.MethodGet, "/healthz", nil)
rec := httptest.NewRecorder()
handler.ServeHTTP(rec, req)

res := rec.Result()
defer res.Body.Close()
body, err := io.ReadAll(res.Body)
if err != nil {
    t.Fatal(err)
}
if res.StatusCode != http.StatusOK {
    t.Fatalf("status %d body=%s", res.StatusCode, body)
}

Recorder implements http.ResponseWriter and records code, headers, body. Prefer rec.Result() when you need a real *http.Response (trailers, etc.); rec.Code / rec.Body are fine for simple asserts.

---

Theory 2 — NewRequest vs context

req := httptest.NewRequest(http.MethodPost, "/items", bytes.NewReader(body))
req.Header.Set("Content-Type", "application/json")

ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
req = req.WithContext(ctx)

Path and query: "/items?limit=10"—available as r.URL.Query(). For Go 1.22+ method muxes, the path must match the registered pattern (GET /items/{id}).

httptest.NewRequest panics on bad method/URL in older mental models—still validate inputs in helpers if you build URLs dynamically.

---

Theory 3 — Table-driven handler matrix

tests := []struct {
    name       string
    method     string
    path       string
    body       string
    wantStatus int
    wantSubstr string
}{
    {"health", http.MethodGet, "/healthz", "", 200, `"status"`},
    {"missing", http.MethodGet, "/items/nope", "", 404, `not found`},
    {"bad json", http.MethodPost, "/items", `{`, 400, ``},
}

for _, tc := range tests {
    t.Run(tc.name, func(t *testing.T) {
        t.Parallel()
        var r io.Reader
        if tc.body != "" {
            r = strings.NewReader(tc.body)
        }
        req := httptest.NewRequest(tc.method, tc.path, r)
        if tc.body != "" {
            req.Header.Set("Content-Type", "application/json")
        }
        rec := httptest.NewRecorder()
        h.ServeHTTP(rec, req)
        if rec.Code != tc.wantStatus {
            t.Fatalf("code=%d body=%s", rec.Code, rec.Body.String())
        }
        if tc.wantSubstr != "" && !strings.Contains(rec.Body.String(), tc.wantSubstr) {
            t.Fatalf("body=%s", rec.Body.String())
        }
    })
}

Rule: each t.Run gets its own handler/server state when handlers are mutable.

---

Theory 4 — Testing Server vs Recorder

Tool	When
httptest.NewRecorder + ServeHTTP	Unit/handler tests (default)
httptest.NewServer	Need real URL, clients, redirects, TLS optional
httptest.NewTLSServer	TLS client tests

ts := httptest.NewServer(handler)
t.Cleanup(ts.Close)
resp, err := ts.Client().Get(ts.URL + "/healthz")
if err != nil {
    t.Fatal(err)
}
defer resp.Body.Close()

ts.Client() is preconfigured for that server (including TLS when used). Prefer Recorder unless you need a real http.Client path (Day 46 skill).

---

Theory 5 — Middleware and path values

Exercise middleware by asserting headers:

if rec.Header().Get("X-Request-ID") == "" {
    t.Fatal("missing request id")
}

For Go 1.22+ mux, path patterns work through the same mux you productionize—test the assembled handler, not only leaf functions, so routing stays covered.

Leaf unit tests still useful for pure decoding logic. Setting PathValue manually is awkward; prefer full mux: GET /items/abc.

// Good: production wiring
h := api.New(log).Handler() // includes mux + middleware
// Avoid: only testing leaf with hand-built PathValue unless necessary

---

Theory 6 — Assertions that age well

Prefer	Avoid
Status code + JSON fields	Exact full body strings only
json.Unmarshal into structs	Brittle substring soup for every field
Header canonical names	Typos in mixed-case keys
Stable error code	Free-form message text alone

var env struct {
    Error struct {
        Code string `json:"code"`
    } `json:"error"`
}
if err := json.Unmarshal(rec.Body.Bytes(), &env); err != nil {
    t.Fatal(err)
}
if env.Error.Code != "not_found" {
    t.Fatalf("code=%q", env.Error.Code)
}

---

Worked example — test helper package

package api_test

import (
    "encoding/json"
    "io"
    "net/http"
    "net/http/httptest"
    "strings"
    "testing"

    "example.com/day49/api"
)

func newTestHandler(t *testing.T) http.Handler {
    t.Helper()
    return api.New().Handler()
}

func doJSON(t *testing.T, h http.Handler, method, path, body string) (int, map[string]any) {
    t.Helper()
    var r io.Reader
    if body != "" {
        r = strings.NewReader(body)
    }
    req := httptest.NewRequest(method, path, r)
    if body != "" {
        req.Header.Set("Content-Type", "application/json")
    }
    rec := httptest.NewRecorder()
    h.ServeHTTP(rec, req)
    var m map[string]any
    if rec.Body.Len() > 0 {
        if err := json.Unmarshal(rec.Body.Bytes(), &m); err != nil {
            t.Fatalf("json: %v body=%s", err, rec.Body.String())
        }
    }
    return rec.Code, m
}

func TestCreateAndGet(t *testing.T) {
    h := newTestHandler(t)
    code, m := doJSON(t, h, http.MethodPost, "/items", `{"name":"widget"}`)
    if code != http.StatusCreated {
        t.Fatalf("create code=%d m=%v", code, m)
    }
    id, _ := m["id"].(string)
    if id == "" {
        t.Fatal("missing id")
    }
    code, m = doJSON(t, h, http.MethodGet, "/items/"+id, "")
    if code != http.StatusOK || m["name"] != "widget" {
        t.Fatalf("get code=%d m=%v", code, m)
    }
}

---

Lab 1 — Project

mkdir -p ~/lab/90daysofx/01-go/day49
cd ~/lab/90daysofx/01-go/day49
go mod init example.com/day49

Bring forward (or reimplement) Day 45/47 API surface: health, list, get, create + middleware (request ID + access log optional).

---

Lab 2 — Coverage matrix

Minimum cases:

1. Health 200
   
2. Create 201/200 + returned id
   
3. Get existing 200
   
4. Get missing 404
   
5. Create invalid body 400
   
6. Create empty name 400
   
7. Method not allowed on patterned route (e.g. DELETE /healthz)
   
8. Request ID header present after middleware

go test ./... -count=1
go test ./... -cover

Aim for meaningful cover on api package (not vanity 100%).

---

Lab 3 — Parallel tests

t.Parallel()

Ensure handler state is either per-test server instance (preferred) or mutex-safe. Catch races:

go test ./... -race

If a test shares a global map store, either protect it or construct a fresh Server per test.

---

Lab 4 — httptest.NewServer client path

One test uses NewServer + real http.Client Get to prove client/server wiring (ties Day 46). Assert status and a JSON field. Cleanup with t.Cleanup(ts.Close).

---

Lab 5 — Golden JSON (optional)

For stable list responses, compare pretty-printed JSON with testdata/list.golden (Day 35 skill). Update with -update flag pattern if you like. Prefer golden for stable sorted lists; avoid for timestamps unless you stub clocks.

---

Common gotchas

Gotcha	Fix
Forgetting Content-Type on POST	Set header in helper
Reusing one Server mutable state across parallel tests	New server per test
Testing only happy path	Matrix includes 4xx
Asserting exact log format only	Assert status/body first
Using live port in unit tests	Recorder/Server
Ignoring middleware in tests	Hit full Handler() chain
Not closing res.Body from Result()	defer res.Body.Close()
Path mismatch with Go 1.22 patterns	Register and request the same shape

---

Checkpoint

• Table-driven tests for all routes
  
• 4xx paths covered
  
• Middleware header asserted
  
• -race clean
  
• Helper reduces boilerplate
  
• Optional NewServer client test

Commit

git add .
git commit -m "day49: httptest table-driven API tests"

Write three personal gotchas before continuing.

---

Tomorrow

Day 50 — Stage V gate: ship a coherent HTTP service with slog, tests, and graceful shutdown—the bar for leaving stdlib I/O stage.

---

Day 50 — Stage V gate

Stage V · ~3h (integration / ship)
Goal: Close Stage V by shipping a stdlib HTTP service with structured logging, handler tests, timeouts, and signal-based graceful shutdown.

Why this day exists

Days 39–49 were skills in isolation. Gates force integration:

• Does the service start and stop cleanly?
  
• Can someone else run tests and understand the README?
  
• Are I/O, JSON, slog, and http habits consistent?

You do not need a database yet (Stage VI). You need a trustworthy process that looks like something you would leave running overnight.

---

Gate bar (definition of done)

Requirement	Evidence
HTTP JSON API (≥3 endpoints)	Code + curl examples
Go 1.22+ method patterns	ServeMux routes
Middleware (≥2): request ID + access log or recover	Code
log/slog (JSON flag optional)	Logs on request
http.Server timeouts set	ReadHeaderTimeout required
Graceful shutdown on SIGINT/SIGTERM	Demo or documented script
Tests via httptest	go test ./... green
README: run, test, endpoints	File present
Race clean	go test -race

Stretch: limit request bodies; DisallowUnknownFields; /readyz vs /healthz.

---

Theory 1 — Graceful shutdown

srv := &http.Server{
    Addr:              addr,
    Handler:           h,
    ReadHeaderTimeout: 5 * time.Second,
}

errCh := make(chan error, 1)
go func() {
    errCh <- srv.ListenAndServe()
}()

ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()

select {
case err := <-errCh:
    if err != nil && err != http.ErrServerClosed {
        return err
    }
case <-ctx.Done():
    shutdownCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
    defer cancel()
    if err := srv.Shutdown(shutdownCtx); err != nil {
        _ = srv.Close() // force
        return err
    }
}

What Shutdown does

1. Stops accepting new conns
   
2. Waits for in-flight handlers (until context done)
   
3. Returns; then process can exit

Long handlers should respect r.Context() cancellation so Shutdown can finish.

Why timeouts on the Server

Field	Protects against
ReadHeaderTimeout	Slowloris-style header drip
ReadTimeout	Slow body uploads
WriteTimeout	Stuck clients on response
IdleTimeout	Lingering keep-alives

Never ship ListenAndServe without at least ReadHeaderTimeout.

---

Theory 2 — Health endpoints

Path	Meaning
/healthz or /livez	Process up
/readyz	Ready to take traffic (deps OK later)

For Stage V, both can be static OK; structure them separately so Stage VI can diverge (DB ping on ready only).

func (s *Server) healthz(w http.ResponseWriter, _ *http.Request) {
    writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
}

func (s *Server) readyz(w http.ResponseWriter, _ *http.Request) {
    // later: ping DB; today: same as live
    writeJSON(w, http.StatusOK, map[string]string{"status": "ready"})
}

---

Theory 3 — README as interface

Minimum sections:

1. What it is
   
2. Go version (go 1.26 in go.mod)
   
3. go run / go test
   
4. Endpoint table
   
5. Shutdown behavior
   
6. Design notes (in-memory store, etc.)

A gate without a README fails the “someone else can run it” bar.

---

Theory 4 — Process boundaries

cmd/service/main.go   → wiring only
api/                  → HTTP + middleware
internal/store/       → in-memory repo (mutex)

Rules of thumb:

• main parses flags, builds logger, constructs server, blocks on signal
  
• No business logic in main beyond wiring
  
• Tests import api / store, not main

---

Architecture sketch

cmd/service/main.go
  → config (flags/env)
  → slog setup
  → api.New(...)
  → http.Server + signal shutdown

api/
  server.go   // routes, middleware
  handlers.go
  server_test.go

Stdlib-first: no web framework.

---

Lab — Ship the gate service

mkdir -p ~/lab/90daysofx/01-go/day50
cd ~/lab/90daysofx/01-go/day50
go mod init example.com/day50

Suggested endpoint set

Method	Path	Behavior
GET	/healthz	liveness
GET	/readyz	readiness
GET	/items	list
GET	/items/{id}	get
POST	/items	create
DELETE	/items/{id}	delete (optional)

Shutdown drill

go run ./cmd/service &
pid=$!
curl -s localhost:8080/healthz
kill -TERM $pid
wait $pid
echo exit:$?

Confirm process exits without needing kill -9, and in-flight requests get a chance to finish (optional sleep handler test).

Test drill

go test ./... -race -count=1

Self-review checklist

• No http.DefaultClient outbound without timeouts (if any client exists)
  
• No unlimited ReadAll on request bodies (io.LimitReader)
  
• Mutex around in-memory maps
  
• JSON Content-Type consistent
  
• Errors are JSON, not only plain text
  
• ReadHeaderTimeout set

---

Worked main skeleton

package main

import (
    "context"
    "flag"
    "log/slog"
    "net/http"
    "os"
    "os/signal"
    "syscall"
    "time"

    "example.com/day50/api"
)

func main() {
    addr := flag.String("addr", ":8080", "listen address")
    jsonLogs := flag.Bool("json-log", false, "JSON logs")
    flag.Parse()

    var handler slog.Handler
    if *jsonLogs {
        handler = slog.NewJSONHandler(os.Stdout, nil)
    } else {
        handler = slog.NewTextHandler(os.Stdout, nil)
    }
    log := slog.New(handler)

    srv := &http.Server{
        Addr:              *addr,
        Handler:           api.New(log).Handler(),
        ReadHeaderTimeout: 5 * time.Second,
        ReadTimeout:       10 * time.Second,
        WriteTimeout:      20 * time.Second,
        IdleTimeout:       60 * time.Second,
    }

    go func() {
        log.Info("listening", "addr", *addr)
        if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
            log.Error("listen", "err", err)
            os.Exit(1)
        }
    }()

    ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
    defer stop()
    <-ctx.Done()
    log.Info("shutting down")

    shCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
    defer cancel()
    if err := srv.Shutdown(shCtx); err != nil {
        log.Error("shutdown", "err", err)
        os.Exit(1)
    }
    log.Info("bye")
}

Recover middleware (stretch / recommended)

func recoverMW(log *slog.Logger) func(http.Handler) http.Handler {
    return func(next http.Handler) http.Handler {
        return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            defer func() {
                if rec := recover(); rec != nil {
                    log.Error("panic", "recover", rec)
                    http.Error(w, `{"error":{"code":"internal","message":"internal error"}}`,
                        http.StatusInternalServerError)
                }
            }()
            next.ServeHTTP(w, r)
        })
    }
}

---

Common gotchas

Gotcha	Fix
Only ListenAndServe in main	Signal + Shutdown
Shutdown timeout too short	10s+ or tune
Handlers ignore r.Context()	Check ctx on slow work
Tests require manual server	httptest
README missing run steps	Write them last, verify once
Leaking goroutines on exit	Shutdown; avoid bare go without lifecycle
Missing ReadHeaderTimeout	Always set
Data races on map store	sync.Mutex / RWMutex

---

Checkpoint (gate)

• All gate bar rows satisfied
  
• go test ./... -race green
  
• SIGTERM demo works
  
• README complete
  
• Personal note: three Stage V skills you will reuse in Stage VI

Commit

git add .
git commit -m "day50: stage V gate HTTP service slog tests shutdown"

Write a short retrospective (5–10 lines) in your journal: what was hardest—io composition, http routing, or testing?

---

Tomorrow

Stage VI begins — Day 51 database/sql: persistence, pooling, QueryContext, and a repository over SQLite or Postgres.