022 Project 22: Log Shipper CLI

022 Build a Log Shipper CLI

Tail a file and POST new lines to an HTTP ingestion endpoint.

file tail -> line channel -> batch -> HTTP POST /ingest

Full main.go

package main

import (
    "bufio"
    "bytes"
    "encoding/json"
    "flag"
    "fmt"
    "io"
    "net/http"
    "os"
    "time"
)

func main() {
    file := flag.String("file", "app.log", "log file")
    endpoint := flag.String("endpoint", "http://localhost:8080/ingest", "ingest URL")
    flag.Parse()

    f, err := os.Open(*file)
    if err != nil {
        panic(err)
    }
    defer f.Close()

    _, _ = f.Seek(0, io.SeekEnd)
    r := bufio.NewReader(f)
    client := &http.Client{Timeout: 3 * time.Second}

    for {
        line, err := r.ReadString('\n')
        if err == io.EOF {
            time.Sleep(300 * time.Millisecond)
            continue
        }
        if err != nil {
            fmt.Println("read error:", err)
            continue
        }

        payload, _ := json.Marshal(map[string]any{
            "ts":   time.Now().Format(time.RFC3339Nano),
            "line": line,
        })

        resp, err := client.Post(*endpoint, "application/json", bytes.NewReader(payload))
        if err != nil {
            fmt.Println("post failed:", err)
            continue
        }
        _ = resp.Body.Close()
    }
}

Run

go run . -file app.log -endpoint http://localhost:8080/ingest

Step-by-Step Explanation

  1. Define request and response contracts first.
  2. Validate inbound input before doing any state changes.
  3. Keep handler logic short and move reusable logic into helper functions.
  4. Add timeouts and clear error paths.
  5. Return consistent responses and status codes.

Code Anatomy

  • Handlers parse input, call domain logic, write response.
  • Shared state uses synchronization where needed.
  • Transport concerns stay separate from business rules.

Learning Goals

  • Build reliable service endpoints in Go.
  • Understand API ergonomics and operational safety.
  • Prepare code structure for tests and persistence later.