Day 17 — os/exec, Time & Network Basics

Updated

July 30, 2025

Day 17 — os/exec, Time & Network Basics

Day 42 — os & os/exec

Stage V · ~3h
Goal: Cross the process boundary safely—read env and files, spawn subprocesses with timeouts, capture stdout/stderr, and never shell-inject yourself.

Why this day exists

Services and CLIs constantly touch the OS:

  • Config paths, PID files, temporary dirs
  • Feature flags via environment
  • Wrapping git, ffmpeg, cloud CLIs, or health probes

os/exec is powerful and easy to misuse (hangs, leaked processes, injection via bash -c).

Theory 1 — os essentials

Files

data, err := os.ReadFile("config.json")     // whole file
err = os.WriteFile("out.txt", data, 0o644) // create/truncate

f, err := os.Open("in.txt")               // read-only
defer f.Close()
f, err = os.OpenFile("a.log", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0o644)

Paths & dirs

dir := os.TempDir()
err := os.MkdirAll("state/cache", 0o755)
info, err := os.Stat(path)
if errors.Is(err, os.ErrNotExist) { /* ... */ }
if info.IsDir() { /* ... */ }

Environment

val, ok := os.LookupEnv("API_TOKEN")
port := os.Getenv("PORT") // "" if unset — often ambiguous
os.Setenv("DEBUG", "1")  // process-local

Prefer explicit LookupEnv when empty string is valid.

Args & exit

os.Args[0] // program name
os.Exit(1) // skips deferred calls in this goroutine's stack — prefer returning errors from main

Pattern:

func main() {
    if err := run(); err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
}

Theory 2 — os/exec model

cmd := exec.Command("echo", "hello", "world") // argv list — no shell
out, err := cmd.Output() // stdout; err may be *exec.ExitError
API Captures Notes
Run nothing Start+wait
Output stdout stderr discarded unless set
CombinedOutput stdout+stderr merged Handy for logs
Start/Wait manual Streaming pipes

Always prefer argv over shell

// BAD — injection if user controls s
exec.Command("sh", "-c", "echo "+s)

// GOOD
exec.Command("echo", s)

If you truly need a shell, document why and sanitize ruthlessly.

Theory 3 — Context deadlines

ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()

cmd := exec.CommandContext(ctx, "sleep", "30")
err := cmd.Run()
// ctx exceeded → process killed (implementation: kill process group nuances on Unix)

CommandContext sends kill when ctx is done. Always set timeouts on untrusted or external tools.

Capture streams with pipes

cmd := exec.CommandContext(ctx, "git", "status", "--porcelain")
var stdout, stderr bytes.Buffer
cmd.Stdout = &stdout
cmd.Stderr = &stderr
err := cmd.Run()
if err != nil {
    return fmt.Errorf("git status: %w\nstderr=%s", err, stderr.String())
}

Exit codes

var ee *exec.ExitError
if errors.As(err, &ee) {
    code := ee.ExitCode()
    _ = code
}

Theory 4 — Stdin and interactive-ish tools

cmd := exec.Command("grep", "foo")
cmd.Stdin = strings.NewReader("foo\nbar\n")
out, err := cmd.Output()

For large data, use cmd.StdinPipe() and write in a goroutine; close the pipe when done so the child sees EOF.

Theory 5 — Security & operations checklist

  1. Fixed binary path when possible (/usr/bin/git) or resolve once.
  2. No user-controlled executable name without allowlist.
  3. Timeouts always for network-bound tools.
  4. Env scrubbing: cmd.Env = append(os.Environ(), "KEY=val") or minimal env.
  5. Working directory: cmd.Dir = dir.
  6. Signals: understand that killing parent may leave children unless process groups are set (SysProcAttr on Unix—advanced).

Worked example — run with timeout helper

package runx

import (
    "bytes"
    "context"
    "errors"
    "fmt"
    "os/exec"
    "time"
)

type Result struct {
    Stdout   string
    Stderr   string
    ExitCode int
}

func Command(ctx context.Context, timeout time.Duration, name string, args ...string) (Result, error) {
    if timeout > 0 {
        var cancel context.CancelFunc
        ctx, cancel = context.WithTimeout(ctx, timeout)
        defer cancel()
    }
    cmd := exec.CommandContext(ctx, name, args...)
    var stdout, stderr bytes.Buffer
    cmd.Stdout = &stdout
    cmd.Stderr = &stderr
    err := cmd.Run()
    res := Result{Stdout: stdout.String(), Stderr: stderr.String()}
    if err != nil {
        var ee *exec.ExitError
        if errors.As(err, &ee) {
            res.ExitCode = ee.ExitCode()
        }
        if errors.Is(ctx.Err(), context.DeadlineExceeded) {
            return res, fmt.Errorf("timeout after %s running %s: %w", timeout, name, err)
        }
        return res, fmt.Errorf("%s: %w (stderr=%q)", name, err, res.Stderr)
    }
    return res, nil
}

Lab 1 — Setup

mkdir -p ~/lab/90daysofx/01-go/day42
cd ~/lab/90daysofx/01-go/day42
go mod init example.com/day42

Implement runx and a small CLI cmd/runxdemo.

Lab 2 — Happy path tests

Use portable commands:

func TestEcho(t *testing.T) {
    res, err := Command(context.Background(), 2*time.Second, "echo", "hi")
    if err != nil {
        t.Fatal(err)
    }
    if got := strings.TrimSpace(res.Stdout); got != "hi" {
        t.Fatalf("got %q", got)
    }
}

On Windows, adjust binary names; this volume assumes a Unix-like lab shell.

Lab 3 — Timeout behavior

func TestTimeout(t *testing.T) {
    _, err := Command(context.Background(), 200*time.Millisecond, "sleep", "5")
    if err == nil {
        t.Fatal("expected timeout error")
    }
}

Confirm the error chain mentions timeout. Optionally check no long-lived sleep remains (pgrep manually).

Lab 4 — File + env integration

Write a tool that:

  1. Reads APP_ENV (default dev)
  2. Loads ./config.$APP_ENV.json via os.ReadFile
  3. Runs echo with a summary line of config size and env

Reject path traversal if you accept a user -config flag (filepath.Clean, stay under workdir).

Lab 5 — Combined stderr (stretch)

Run a command that writes to stderr (ls of missing path). Ensure your helper surfaces stderr in the returned error string. Assert with errors.As on exit code ≠ 0.

Common gotchas

Gotcha Fix
Command(userInput) as shell Argv list only; allowlist
No timeout CommandContext + deadline
Ignoring stderr on failure Capture and wrap
os.Exit in libraries Return error
Relative binary relying on PATH Document PATH; consider absolute
Deadlock with full pipe buffers Copy stdout/stderr concurrently if large
Assuming Output includes stderr It doesn’t—set cmd.Stderr

Checkpoint

  • ReadFile / WriteFile / MkdirAll used intentionally
  • Subprocess helper with timeout + stdout/stderr capture
  • Test proves timeout kills long sleep
  • Can explain why sh -c is dangerous
  • Exit codes inspected via ExitError
  • main returns errors without losing defers carelessly

Commit

git add .
git commit -m "day42: os/exec with timeouts and safe argv"

Write three personal gotchas before continuing.

Tomorrow

Day 43 — time: durations, timers, tickers, locations, monotonic clocks, and a cancellable mini-scheduler.

Day 43 — time, timers & tickers

Stage V · ~3h
Goal: Use time correctly for scheduling and deadlines—locations, monotonic measurements, timers/tickers that you actually stop—and build a context-aware mini-scheduler.

Why this day exists

Distributed systems fail on time:

  • “Works in UTC, breaks in local TZ”
  • Comparing wall clocks across machines
  • Leaking time.Ticker goroutines
  • Using time.Sleep in hot loops without cancellation

time is small API surface, large operational impact.

Theory 1 — Instant, duration, location

now := time.Now()              // wall + monotonic reading
d := 500 * time.Millisecond
later := now.Add(d)
elapsed := time.Since(now)     // uses monotonic when possible
delta := later.Sub(now)

Duration

const day = 24 * time.Hour
d, err := time.ParseDuration("1h30m") // "300ms", "2h45m"

Format for humans: d.String() → "1h30m0s".

Wall vs monotonic

time.Now() stores wall clock and monotonic clock. Subtractions between Time values from Now ignore wall adjustments (NTP step) for elapsed measurement. Serializing to JSON loses monotonic reading—only wall remains after Marshal/Unmarshal.

Theory 2 — Parsing and formatting

Go’s reference time is not YYYY-MM-DD. It is:

Mon Jan 2 15:04:05 MST 2006
t, err := time.Parse(time.RFC3339, "2026-07-27T15:04:05Z")
s := t.UTC().Format(time.RFC3339Nano)
// custom:
s = t.Format("2006-01-02 15:04:05")

Prefer predefined layouts (RFC3339, DateOnly since 1.20, etc.) when possible.

Locations

loc, err := time.LoadLocation("America/New_York")
local := t.In(loc)

time.Local depends on machine config. Servers: store UTC, display with explicit location.

time.Date(2026, 7, 27, 12, 0, 0, 0, time.UTC)

Theory 3 — Timer and Ticker

Timer (once)

timer := time.NewTimer(2 * time.Second)
defer timer.Stop()
select {
case <-timer.C:
    // fired
case <-ctx.Done():
    if !timer.Stop() {
        <-timer.C // drain if stop lost the race
    }
    return ctx.Err()
}

Or time.After (convenient, but cannot be garbage-collected until it fires—avoid in tight loops).

Ticker (periodic)

ticker := time.NewTicker(time.Second)
defer ticker.Stop()
for {
    select {
    case t := <-ticker.C:
        _ = t // tick time
    case <-ctx.Done():
        return ctx.Err()
    }
}

Always Stop tickers you create when the loop ends.

time.Tick

Discouraged for long-lived processes—no way to stop. Prefer NewTicker.

Theory 4 — Sleep and cancellation

// cancellable sleep
func Sleep(ctx context.Context, d time.Duration) error {
    timer := time.NewTimer(d)
    defer timer.Stop()
    select {
    case <-timer.C:
        return nil
    case <-ctx.Done():
        return ctx.Err()
    }
}

Never block shutdown paths on uncancellable time.Sleep.

Theory 5 — Deadlines for I/O (preview)

deadline := time.Now().Add(5 * time.Second)
_ = conn.SetDeadline(deadline)

Day 44 uses this on sockets; the idea starts here: absolute deadlines often beat stacked relative sleeps.

Worked example — mini scheduler

package sched

import (
    "context"
    "log/slog"
    "time"
)

// Every runs fn immediately (optional) then every interval until ctx cancel.
func Every(ctx context.Context, interval time.Duration, runImmediately bool, fn func(context.Context) error) error {
    if interval <= 0 {
        return errNonPositive
    }
    if runImmediately {
        if err := fn(ctx); err != nil {
            return err
        }
    }
    t := time.NewTicker(interval)
    defer t.Stop()
    for {
        select {
        case <-ctx.Done():
            return ctx.Err()
        case <-t.C:
            if err := fn(ctx); err != nil {
                slog.Error("tick failed", "err", err)
                // choose: continue or return — document policy
            }
        }
    }
}

Define errNonPositive with errors.New. Production schedulers add jitter, overlap guards (sync.Mutex or skip if previous still running), and metrics—keep today focused.

Overlap-safe variant sketch

var gate sync.Mutex
// on tick:
if !gate.TryLock() {
    return // skip overlapping run
}
defer gate.Unlock()
_ = fn(ctx)

(TryLock is available in modern Go sync.Mutex.)

Lab 1 — Setup

mkdir -p ~/lab/90daysofx/01-go/day43
cd ~/lab/90daysofx/01-go/day43
go mod init example.com/day43

Lab 2 — Duration & location drills

Write tests:

  1. ParseDuration("1500ms") equals 1500 * time.Millisecond
  2. Round-trip RFC3339 UTC
  3. Convert a fixed instant between UTC and Asia/Kolkata (or your TZ)—assert hour offset
func TestRFC3339RoundTrip(t *testing.T) {
    in := "2026-01-02T03:04:05Z"
    ts, err := time.Parse(time.RFC3339, in)
    if err != nil {
        t.Fatal(err)
    }
    if ts.UTC().Format(time.RFC3339) != in {
        t.Fatal(ts)
    }
}

Lab 3 — Scheduler with cancel

Implement Every and test with a fake clock or short real intervals:

func TestEveryCancel(t *testing.T) {
    ctx, cancel := context.WithCancel(context.Background())
    var n atomic.Int32
    done := make(chan error, 1)
    go func() {
        done <- Every(ctx, 20*time.Millisecond, true, func(context.Context) error {
            n.Add(1)
            return nil
        })
    }()
    time.Sleep(70 * time.Millisecond)
    cancel()
    err := <-done
    if !errors.Is(err, context.Canceled) {
        t.Fatalf("err=%v", err)
    }
    if n.Load() < 2 {
        t.Fatalf("ticks=%d", n.Load())
    }
}

Lab 4 — Cancellable sleep helper

Unit test that Sleep(ctx, time.Hour) returns quickly when cancel() is called.

Lab 5 — Wall clock journaling (stretch)

Log time.Now().Format(time.RFC3339Nano) every second for 5s. Change system TZ mid-lab if you can; note display changes vs UTC storage practice.

Common gotchas

Gotcha Fix
time.Tick leak NewTicker + Stop
time.After in loop NewTimer + reset/stop carefully
Local TZ in APIs UTC on wire
Wrong layout (YYYY) Use 2006 reference
Ignoring ticker.Stop Always defer Stop
Sleep without ctx Cancellable helper
Comparing times from different zones carelessly Normalize with .UTC() or .Equal

Checkpoint

  • Explain monotonic vs wall in one sentence
  • Format/parse RFC3339 correctly
  • Scheduler stops on cancel and stops ticker
  • Overlap policy documented (skip vs queue)
  • No time.Tick in long-lived code
  • Cancellable sleep tested

Commit

git add .
git commit -m "day43: time scheduler with tickers and cancel"

Write three personal gotchas before continuing.

Tomorrow

Day 44 — net basics: TCP listen/dial, net.Conn deadlines, and an echo server + client you can test on localhost.

Day 44 — net basics (TCP)

Stage V · ~3h
Goal: Understand sockets under HTTP—listen, dial, read/write with deadlines, and ship a TCP echo server + client with tests.

Why this day exists

net/http will dominate Days 45–50, but HTTP rides on net.Conn. When debugging resets, half-open connections, or custom protocols, you need the TCP layer:

  • Listen / Accept
  • Dial / DialContext
  • Deadlines and buffer reads
  • Graceful close vs abort

Theory 1 — Listen and Accept

ln, err := net.Listen("tcp", "127.0.0.1:0") // :0 → ephemeral port
if err != nil {
    return err
}
defer ln.Close()

addr := ln.Addr().String()

for {
    conn, err := ln.Accept()
    if err != nil {
        return err // or continue on temporary errors
    }
    go serve(conn)
}

Address forms

Address Meaning
"127.0.0.1:8080" IPv4 loopback
"[::1]:8080" IPv6 loopback
":8080" All interfaces, port 8080
"localhost:8080" Resolver-dependent

Prefer loopback in tests.

Theory 2 — Dial

d := net.Dialer{Timeout: 3 * time.Second}
conn, err := d.DialContext(ctx, "tcp", addr)
// or: net.DialTimeout("tcp", addr, 3*time.Second)
defer conn.Close()

Always bound dial with timeout/context—hanging dials freeze clients.

Theory 3 — Conn I/O and deadlines

net.Conn implements io.ReadWriteCloser plus:

conn.SetDeadline(time.Now().Add(5 * time.Second))
conn.SetReadDeadline(...)
conn.SetWriteDeadline(...)

Without deadlines, a stuck peer can block a goroutine forever.

Line-oriented echo pattern

func serve(conn net.Conn) {
    defer conn.Close()
    _ = conn.SetDeadline(time.Now().Add(30 * time.Second))
    r := bufio.NewReader(conn)
    for {
        line, err := r.ReadString('\n')
        if err != nil {
            return
        }
        if _, err := io.WriteString(conn, line); err != nil {
            return
        }
        // extend deadline per message if desired
        _ = conn.SetDeadline(time.Now().Add(30 * time.Second))
    }
}

Theory 4 — Temporary errors & close

var ne net.Error
if errors.As(err, &ne) && ne.Timeout() {
    // deadline hit
}

Close unblocks pending I/O with an error. For half-close: CloseWrite on *net.TCPConn when you need FIN while still reading—advanced protocol design.

Theory 5 — net.Pipe for tests

In-memory full-duplex connection without sockets:

client, server := net.Pipe()
go func() {
    defer server.Close()
    _, _ = io.Copy(server, server) // silly echo; use your serve()
}()

Great for unit-testing protocol framing without ports.

Worked example — package echo

package echo

import (
    "bufio"
    "context"
    "fmt"
    "io"
    "net"
    "time"
)

type Server struct {
    ln net.Listener
}

func Listen(addr string) (*Server, error) {
    ln, err := net.Listen("tcp", addr)
    if err != nil {
        return nil, err
    }
    s := &Server{ln: ln}
    go s.loop()
    return s, nil
}

func (s *Server) Addr() string { return s.ln.Addr().String() }

func (s *Server) Close() error { return s.ln.Close() }

func (s *Server) loop() {
    for {
        conn, err := s.ln.Accept()
        if err != nil {
            return
        }
        go handle(conn)
    }
}

func handle(conn net.Conn) {
    defer conn.Close()
    _ = conn.SetDeadline(time.Now().Add(10 * time.Second))
    br := bufio.NewReader(conn)
    for {
        line, err := br.ReadBytes('\n')
        if len(line) > 0 {
            if _, werr := conn.Write(line); werr != nil {
                return
            }
        }
        if err != nil {
            return
        }
        _ = conn.SetDeadline(time.Now().Add(10 * time.Second))
    }
}

func Echo(ctx context.Context, addr, msg string) (string, error) {
    var d net.Dialer
    conn, err := d.DialContext(ctx, "tcp", addr)
    if err != nil {
        return "", err
    }
    defer conn.Close()
    if deadline, ok := ctx.Deadline(); ok {
        _ = conn.SetDeadline(deadline)
    }
    if _, err := io.WriteString(conn, msg); err != nil {
        return "", err
    }
    br := bufio.NewReader(conn)
    line, err := br.ReadString('\n')
    if err != nil {
        return "", fmt.Errorf("read: %w", err)
    }
    return line, nil
}

Lab 1 — Setup

mkdir -p ~/lab/90daysofx/01-go/day44
cd ~/lab/90daysofx/01-go/day44
go mod init example.com/day44

Lab 2 — Integration test on localhost

func TestEcho(t *testing.T) {
    s, err := Listen("127.0.0.1:0")
    if err != nil {
        t.Fatal(err)
    }
    t.Cleanup(func() { _ = s.Close() })

    ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
    defer cancel()

    got, err := Echo(ctx, s.Addr(), "ping\n")
    if err != nil {
        t.Fatal(err)
    }
    if got != "ping\n" {
        t.Fatalf("got %q", got)
    }
}
go test ./...

Lab 3 — Concurrent clients

Spawn 20 goroutines each sending unique lines. Use errgroup (from Stage III) or sync.WaitGroup + error channel. Assert all echoes match.

Lab 4 — Deadline failure

Set a very short read deadline on a client that dials but never receives a newline from a hung peer (write a “slow” server that sleeps). Assert timeout error.

Lab 5 — Manual demo

go run ./cmd/echod -addr 127.0.0.1:9000
# other terminal:
printf 'hello\n' | nc 127.0.0.1 9000

Theory note — from TCP to HTTP

HTTP/1.1 is text framed on a net.Conn: request line, headers, body. net/http manages connection pools, parsing, and TLS. After today you should be able to sketch where Accept sits under http.Server.Serve.

Common gotchas

Gotcha Fix
No dial timeout Dialer{Timeout} / DialContext
No read deadline SetDeadline / SetReadDeadline
Forgetting \n in line protocol Document framing
Accept loop dies on one error Log temporary; exit on close
Goroutine per conn without limits Bound concurrency later
Testing on :8080 fixed Use :0 + Addr()
IPv6/localhost surprises Prefer 127.0.0.1 in tests

Checkpoint

  • Echo server listens on ephemeral port
  • Client round-trip tested
  • Deadlines demonstrated
  • Concurrent clients succeed
  • Can explain net.Conn vs http.ResponseWriter relationship
  • Listener closed in t.Cleanup

Commit

git add .
git commit -m "day44: TCP echo server and client with deadlines"

Write three personal gotchas before continuing.

Tomorrow

Day 45 — net/http server: Handler, Go 1.22+ ServeMux patterns, middleware chains, and a small JSON API.